From c2de397a8674b1546fad0f6f01088a42efde98cf Mon Sep 17 00:00:00 2001 From: Deyao Zhu Date: Mon, 23 Oct 2023 21:31:13 +0300 Subject: [PATCH] update refcoco preparation --- dataset/README_MINIGPTv2_FINETUNE.md | 46 ++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/dataset/README_MINIGPTv2_FINETUNE.md b/dataset/README_MINIGPTv2_FINETUNE.md index 2d5c825..be1e09b 100644 --- a/dataset/README_MINIGPTv2_FINETUNE.md +++ b/dataset/README_MINIGPTv2_FINETUNE.md @@ -2,16 +2,56 @@ ### COCO captions - ### RefCOCO, RefCOCO+, RefCOCOg -### Visual genome +Makesure you have the COCO 2014 images first. + +Then, +download RefCOCO, RefCOCO+, and RefCOCOg annotation files in the following links. + +- https://bvisionweb1.cs.unc.edu/licheng/referit/data/refcoco.zip +- https://bvisionweb1.cs.unc.edu/licheng/referit/data/refcoco+.zip +- https://bvisionweb1.cs.unc.edu/licheng/referit/data/refcocog.zip + +Unzip these files to the location you like. It should have the structure like the following + +``` +Location_you_like +├── refcoco +│ ├── instances.json +│ ├── refs(google).p +│ └── refs(unc).p +├── refcoco+ +│ ├── instances.json +│ └── refs(unc).p +└── refcocog + ├── instances.json + ├── refs(google).p + └── refs(umd).p +``` + +Set **image_path** in all the following dataset configuration files to the COCO 2014 image folder. +Similarly, set **ann_path** in all the following configs to the above folder (Location_you_like) that contains refcoco, refcoco+, and refcocog. + +- [minigpt4/configs/refcoco.yaml](../minigpt4/configs/refcoco.yaml) +- [minigpt4/configs/refcocog.yaml](../minigpt4/configs/refcocog.yaml) +- [minigpt4/configs/refcocop.yaml](../minigpt4/configs/refcocop.yaml) +- [minigpt4/configs/invrefcoco.yaml](../minigpt4/configs/invrefcoco.yaml) +- [minigpt4/configs/invrefcocog.yaml](../minigpt4/configs/invrefcocog.yaml) +- [minigpt4/configs/invrefcocop.yaml](../minigpt4/configs/invrefcocop.yaml) + + + +### Visual Genome ### textcaps ### LLaVA -### gqa + + + +### GQA ### OKVQA