| Anomie closed this task as "Resolved". Anomie added a comment. |
Commons run with batch-size 4000:
- Average transaction time for the revision table was 1.3709 seconds, max was 3.07866 seconds.
- The initial run took 31 hours 36.5 minutes to process 17896490 rows. At that rate, enwiki would take about 87.2 hours.
- For some reason the Commons run had a much lower MRSS than the eowiki run with the same code. That makes me a bit skeptical about using it as a measure of memory usage for --reuse-content.
- Second run took 722 seconds to process 0 rows.
- Third run with --reuse-content seemed to be using too much memory and took too long. Let's not do it.
- Sanity checks passed.
I think we can call this resolved now. The live runs should probably use a somewhat lower batch size (2000?) to keep the time per transaction down, and shouldn't use --reuse-content.
TASK DETAIL
EMAIL PREFERENCES
To: Anomie
Cc: ops-monitoring-bot, Marostegui, jcrespo, Aklapper, aude, Addshore, Anomie, Jdforrester-WMF, gerritbot, Abit, daniel, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, PDrouin-WMF, Gq86, Baloch007, E1presidente, Ramsey-WMF, Cparle, Darkminds3113, SandraF_WMF, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Tramullas, Acer, LawExplorer, Lewizho99, JJMC89, Maathavan, Agabi10, Susannaanas, Aschroet, Jane023, Wikidata-bugs, PKM, Base, matthiasmullie, Ricordisamoa, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331, Ltrlg
Cc: ops-monitoring-bot, Marostegui, jcrespo, Aklapper, aude, Addshore, Anomie, Jdforrester-WMF, gerritbot, Abit, daniel, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, PDrouin-WMF, Gq86, Baloch007, E1presidente, Ramsey-WMF, Cparle, Darkminds3113, SandraF_WMF, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Tramullas, Acer, LawExplorer, Lewizho99, JJMC89, Maathavan, Agabi10, Susannaanas, Aschroet, Jane023, Wikidata-bugs, PKM, Base, matthiasmullie, Ricordisamoa, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331, Ltrlg
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
