dr0ptp4kt added a comment.
By the way, I'm attempting a run for the first 1332 munged files (one shy of
the 1333 where terminated last time around) with buffer at 100000**0** and heap
size at 31g and queue capacity at 4000 on the gaming-class desktop to see
whether this imports smoothly and whether performance gains are noticeable.
ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ date
Wed Mar 20 02:36:59 PM CDT 2024
ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ time ./loadData.sh -n
wdq -d /mnt/firehose/munge_on_later_data_set -s 1 -e 1332
...screen'ing in to check:
Processing wikidump-000000505.ttl.gz
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd";><html><head><meta
http-equiv="Content-Type"
content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title
></head
><body<p>totalElapsed=13452ms, elapsed=13452ms, connFlush=0ms,
batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
><hr><p>COMMIT: totalElapsed=167329ms, commitTime=1711041930967,
mutationCount=4566497</p
></html
>Thu Mar 21 12:25:35 PM CDT 2024
Processing wikidump-000000506.ttl.gz
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd";><html><head><meta
http-equiv="Content-Type"
content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title
></head
><body<p>totalElapsed=15405ms, elapsed=15405ms, connFlush=0ms,
batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
><hr><p>COMMIT: totalElapsed=203202ms, commitTime=1711042135111,
mutationCount=5262167</p
></html
>Thu Mar 21 12:28:58 PM CDT 2024
Processing wikidump-000000507.ttl.gz
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd";><html><head><meta
http-equiv="Content-Type"
content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title
></head
><body<p>totalElapsed=14701ms, elapsed=14700ms, connFlush=0ms,
batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p
><hr><p>COMMIT: totalElapsed=178754ms, commitTime=1711042314114,
mutationCount=5005853</p
></html
>Thu Mar 21 12:31:57 PM CDT 2024
TASK DETAIL
https://phabricator.wikimedia.org/T359062
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dr0ptp4kt
Cc: ssingh, bking, dr0ptp4kt, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1,
karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi,
Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen,
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
