dr0ptp4kt added a comment.
**Going for the full import**
Further import commenced from there with a `bufferCapacity` of 100000**0**:
ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ date
Mon Mar 4 06:31:06 PM CST 2024
ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ time ./loadData.sh -n
wdq -d /mnt/firehose/munge_on_later_data_set -s 151 -e 2202
Processing wikidump-000000151.ttl.gz
Munge files 151 through 1333 were processed, stopping at Friday, March 8,
2024 12:07:23 AM CST.
So, we have about 4 hours for files 1-150, then another 77.6 hours for files
151-1333. This means about 66% of the full dump was processed in about 3.5 days.
As noted earlier, there may be an opportunity to set the queue capacity
higher and squeeze out even better performance. This will need to wait until
I'm physically at the gaming class desktop.
TASK DETAIL
https://phabricator.wikimedia.org/T359062
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dr0ptp4kt
Cc: ssingh, bking, dr0ptp4kt, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1,
karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi,
Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen,
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
