ArielGlenn has submitted this change and it was merged. Change subject: dumps: tweak stages a bit ......................................................................
dumps: tweak stages a bit we want to avoid having a stage claim it's complete when there it's dependent on a previous stage, but not spend a lot of time blocking on waiting for those jobs to get done, where we can move on. this is a first draft compromise. Change-Id: I57940a853749c690bd4ae8f1dd96000d1691515a --- M modules/snapshot/templates/dumpstages.erb 1 file changed, 11 insertions(+), 18 deletions(-) Approvals: ArielGlenn: Looks good to me, approved jenkins-bot: Verified diff --git a/modules/snapshot/templates/dumpstages.erb b/modules/snapshot/templates/dumpstages.erb index 6018834..c6a247f 100644 --- a/modules/snapshot/templates/dumpstages.erb +++ b/modules/snapshot/templates/dumpstages.erb @@ -2,23 +2,18 @@ # slots_used numcommands on_failure error_notify command <% if @stagestype == 'normal' or @stagestype == 'partial' %> -# stubs first -1 8 <%= @stages['smallwikis']['firststage'] -%> --job xmlstubsdump -# tables next so inconsistencies between stubs and tables aren't too huge -1 8 <%= @stages['smallwikis']['rest'] %> --job tables -# stubs for big wikis -4 2 <%= @stages['bigwikis']['firststage'] -%> --job xmlstubsdump -# tables for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job tables -# recombine stubs for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job xmlstubsdumprecombine +# mark the start of the run for all small, big wikis +8 1 <%= @stages['smallwikis']['firststage'] -%> --job createdirs; <%= @stages['bigwikis']['firststage'] %> --job createdirs + +# stubs and then tables so inconsistencies between stubs and tables aren't too huge +1 8 <%= @stages['smallwikis']['rest'] -%> --job xmlstubsdump; <%= @stages['smallwikis']['rest'] %> --job tables +# stubs, recombines, tables for big wikis +4 2 <%= @stages['bigwikis']['rest'] -%> --job xmlstubsdump; <%= @stages['bigwikis']['rest'] -%> --job xmlstubsdumprecombine; <%= @stages['bigwikis']['rest'] -%> --job tables # regular articles 1 8 <%= @stages['smallwikis']['rest'] -%> --job articlesdump -# regular articles for big wikis -4 2 <%= @stages['bigwikis']['rest'] -%> --job articlesdump -# recombine articles for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job articlesdumprecombine +# regular articles, recombines for big wikis +4 2 <%= @stages['bigwikis']['rest'] -%> --job articlesdump; <%= @stages['bigwikis']['rest'] -%> --job articlesdumprecombine # regular articles multistream 1 8 <%= @stages['smallwikis']['rest'] -%> --job articlesmultistreamdump @@ -27,10 +22,8 @@ # articles plus meta pages 1 8 <%= @stages['smallwikis']['rest'] -%> --job metacurrentdump -# articles plus meta pages for big wikis -4 2 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdump -# recombine articles plus meta pages for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdumprecombine +# articles, recombine plus meta pages for big wikis +4 2 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdump; <%= @stages['bigwikis']['rest'] -%> --job metacurrentdumprecombine <% if @stagestype == 'normal' %> # all remaining jobs -- To view, visit https://gerrit.wikimedia.org/r/233418 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I57940a853749c690bd4ae8f1dd96000d1691515a Gerrit-PatchSet: 3 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: ArielGlenn <ar...@wikimedia.org> Gerrit-Reviewer: ArielGlenn <ar...@wikimedia.org> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
