chris/MiniGPT-4
1
0
Fork 0
mirror of https://github.com/Vision-CAIR/MiniGPT-4.git synced 2025-04-05 02:20:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

update evaluation readme

Browse Source
This commit is contained in:
junchen14 2023-11-01 09:40:27 +03:00
parent 4daac0d4d2
commit e3e5257482
1 changed files with 10 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
eval_scripts/EVAL_README.md
View File

@ -57,14 +57,15 @@ ${MINIGPTv2_EVALUATION_DATASET}
export PYTHONPATH=$PYTHONPATH:/path/to/directory/of/MiniGPT-4 export PYTHONPATH=$PYTHONPATH:/path/to/directory/of/MiniGPT-4
``` ```
### evaluation config files ### config file setup
Set **llama_model** to the path of LLaMA model. Set **llama_model** to the path of LLaMA model.
Set **ckpt** to the path of our pretrained model. Set **ckpt** to the path of our pretrained model.
Set **eval_file_path** to the path of the annotation files for the evaluation data. Set **eval_file_path** to the path of the annotation files for the evaluation data.
Set **img_path** to the path of the images. Set **img_path** to the path of the images.
Set **save_path** to the path of saving evaluation output. Set **save_path** to the path of saving evaluation output.
- [minigpt4/eval_configs/minigptv2_benchmark_evaluation.yaml](../minigpt4/eval_configs/minigptv2_benchmark_evaluation.yaml) in [minigpt4/eval_configs/minigptv2_benchmark_evaluation.yaml](../minigpt4/eval_configs/minigptv2_benchmark_evaluation.yaml)
@ -73,12 +74,9 @@ Set **save_path** to the path of saving evaluation output.
port=port_number port=port_number
cfg_path=/path/to/eval_configs/minigptv2_benchmark_evaluation.yaml cfg_path=/path/to/eval_configs/minigptv2_benchmark_evaluation.yaml
dataset names:
dataset | | refcoco | refcoco+ | refcocog |
--- | | ------- | -------- | -------- |
refcoco |
refcoco+ |
refcocog |
``` ```
torchrun --master-port ${port} --nproc_per_node 1 eval_ref.py \ torchrun --master-port ${port} --nproc_per_node 1 eval_ref.py \
@ -90,29 +88,16 @@ torchrun --master-port ${port} --nproc_per_node 1 eval_ref.py \
port=port_number port=port_number
cfg_path=/path/to/eval_configs/minigptv2_benchmark_evaluation.yaml cfg_path=/path/to/eval_configs/minigptv2_benchmark_evaluation.yaml
eval_file_path=/path/to/eval/annotation/path
image_path=/path/to/eval/image/path
save_path=/path/to/save/path
ckpt=/path/to/evaluation/checkpoint
split=evaluation_data_split
dataset=dataset_type dataset=dataset_type
dataset_names | dataset_names:
--- | |okvqa | vizwiz | iconvqa | gqa | vsr | hm |
okvqa | | ------- | -------- | -------- |-------- | -------- |
vizwiz |
iconvqa |
gqa |
vsr |
hm |
``` ```
torchrun --master-port ${port} --nproc_per_node 1 eval_vqa.py \ torchrun --master-port ${port} --nproc_per_node 1 eval_vqa.py \
--cfg-path ${cfg_path} --img_path ${image_path} --eval_file_path ${eval_file_path} --save_path ${save_path} \ --cfg-path ${cfg_path} --dataset dataset_name
--ckpt ${ckpt} --split ${split} --dataset ${dataset} --lora_r 64 --lora_alpha 16 \
--batch_size 10 --max_new_tokens 20 --resample
``` ```