dr0ptp4kt added a comment.

  **Going for the full import**
  Further import commenced from there with a `bufferCapacity` of 100000**0**:
    ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ date
    Mon Mar  4 06:31:06 PM CST 2024
    ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ time ./loadData.sh -n 
wdq -d /mnt/firehose/munge_on_later_data_set -s 151 -e 2202
    Processing wikidump-000000151.ttl.gz
  Munge files 151 through 1333 were processed, stopping at Friday, March 8, 
2024 12:07:23 AM CST.
  So, we have about 4 hours for files 1-150, then another 77.6 hours for files 
151-1333. This means about 66% of the full dump was processed in about 3.5 days.
  As noted earlier, there may be an opportunity to set the queue capacity 
higher and squeeze out even better performance. This will need to wait until 
I'm physically at the gaming class desktop.



To: dr0ptp4kt
Cc: ssingh, bking, dr0ptp4kt, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, 
karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

Reply via email to