[Wikidata-bugs] [Maniphest] [Commented On] T161577: OOM problems with snapshot1007 due to wikidata dumpRdf jobs taking all 60GB+ of main memory

[hoo]

hoo added a comment.

HHVM on snapshot1007:
22079 datasets 20 0 10.522g 9.948g 22216 R 88.5 15.8 78:49.10 php /srv/mediawiki/multiversion/MWScript.php extensions/Wikidata/extensions/Wikibase/repo/maintenance/dumpRdf.php --wiki wikidatawiki --sharding-factor 5 --shard 0 --snippet
Processed 424796 entities.

Zend on terbium:
16514 www-data 20 0 494840 217436 10144 R 86.9 0.7 82:10.50 php5 /srv/mediawiki-staging/multiversion/MWScript.php extensions/Wikidata/extensions/Wikibase/repo/maintenance/dumpRdf.php --wiki wikidatawiki --sharding-factor 5 --shard 0 --snippet
Processed 313032 entities.

Both processes were started at about the same time, they might have used different database slaves though (that might affect the speed, but not the memory leak).

TASK DETAIL

https://phabricator.wikimedia.org/T161577

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo
Cc: daniel, WMDE-leszek, gerritbot, hoo, ArielGlenn, Gehel, EBernhardson, Adik2382, Lunewa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, D3r1ck01, Izno, gnosygnu, Wikidata-bugs, aude, Mbch331

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs