MiniGPT-4/dataset
SamimAB a8eb69ecd1 Using ijson to avoid loading full json in memory.
Using ijson to load item by item, so it is possible to load dataset
using dataset/convert_cc_sbu.py and dataset/convert_laion.py on machines
with low RAM.
2023-09-20 23:32:58 +05:30
..
convert_cc_sbu.py Using ijson to avoid loading full json in memory. 2023-09-20 23:32:58 +05:30
convert_laion.py Using ijson to avoid loading full json in memory. 2023-09-20 23:32:58 +05:30
download_cc_sbu.sh Update image download size 2023-08-28 08:31:47 -06:00
download_laion.sh Update image download size 2023-08-28 08:31:47 -06:00
README_1_STAGE.md first commit 2023-04-17 01:04:16 +03:00
README_2_STAGE.md first commit 2023-04-17 01:04:16 +03:00