ArielGlenn added a comment.

  Ran
  
    php /srv/mediawiki/multiversion/MWScript.php 
extensions/Wikibase/repo/maintenance/dumpRdf.php --wiki commonswiki 
--batch-size 500 --format nt --flavor full-dump --entity-type mediainfo 
--no-cache --dbgroupdefault dump  --ignore-missing --first-page-id 78846320 
--last-page-id 79046320 --shard 0 --sharding-factor 1  
2>/var/lib/dumpsgen/mediainfo-log-small-shard-oom.txt | gzip > 
/mnt/dumpsdata/temp/dumpsgen/mediainfo-dumps-test-nt-one-shard-small-oom.gz
  
  which should cover the page range where we had the oom; it ran to completion 
fine. I guess that there is some small memory leak that must accumulate over 
batches, which is what did us in earlier. As long as we limit runs to some 
reasonable number of pages each time, we should be fine.

TASK DETAIL
  https://phabricator.wikimedia.org/T221917

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: ArielGlenn
Cc: Cparle, Abit, Gehel, jleedev, hoo, ArielGlenn, WMDE-leszek, Poyekhali, 
Steinsplitter, Aklapper, Lydia_Pintscher, Bugreporter, Tgr, Ramsey-WMF, Jarekt, 
Addshore, Tpt, Salgo60, Lucas_Werkmeister_WMDE, Smalyshev, Hook696, Daryl-TTMG, 
RomaAmorRoma, 0010318400, E.S.A-Sheild, darthmon_wmde, Meekrab2012, 
joker88john, CucyNoiD, Nandana, NebulousIris, JKSTNK, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, PDrouin-WMF, Gq86, 
Af420, E1presidente, Darkminds3113, Anooprao, SandraF_WMF, Bsandipan, Lordiis, 
GoranSMilovanovic, Adik2382, Lunewa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
EBjune, Tramullas, Acer, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, 
Silverfish, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, 
Susannaanas, Ixocactus, Wong128hk, gnosygnu, Jane023, jkroll, Wikidata-bugs, 
Jdouglas, Base, matthiasmullie, aude, Tobias1984, El_Grafo, Dinoguy1000, 
Manybubbles, Ricordisamoa, Wesalius, Fabrice_Florin, Raymond, Jdforrester-WMF, 
Mbch331, Keegan
_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

Reply via email to