ArielGlenn has uploaded a new change for review. https://gerrit.wikimedia.org/r/233418
Change subject: dumps: tweak stages a bit ...................................................................... dumps: tweak stages a bit we want to avoid having a stage claim it's complete when there it's dependent on a previous stage, but not spend a lot of time blocking on waiting for those jobs to get done, where we can move on. this is a first draft compromise. Change-Id: I57940a853749c690bd4ae8f1dd96000d1691515a --- M modules/snapshot/templates/dumpstages.erb 1 file changed, 8 insertions(+), 18 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/18/233418/1 diff --git a/modules/snapshot/templates/dumpstages.erb b/modules/snapshot/templates/dumpstages.erb index 6018834..e1a9e39 100644 --- a/modules/snapshot/templates/dumpstages.erb +++ b/modules/snapshot/templates/dumpstages.erb @@ -2,23 +2,15 @@ # slots_used numcommands on_failure error_notify command <% if @stagestype == 'normal' or @stagestype == 'partial' %> -# stubs first -1 8 <%= @stages['smallwikis']['firststage'] -%> --job xmlstubsdump -# tables next so inconsistencies between stubs and tables aren't too huge -1 8 <%= @stages['smallwikis']['rest'] %> --job tables -# stubs for big wikis -4 2 <%= @stages['bigwikis']['firststage'] -%> --job xmlstubsdump -# tables for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job tables -# recombine stubs for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job xmlstubsdumprecombine +# stubs and then tables so inconsistencies between stubs and tables aren't too huge +1 8 <%= @stages['smallwikis']['firststage'] -%> --job xmlstubsdump; <%= @stages['smallwikis']['rest'] %> --job tables +# stubs, recombines, tables for big wikis +4 2 <%= @stages['bigwikis']['firststage'] -%> --job xmlstubsdump; <%= @stages['bigwikis']['rest'] -%> --job xmlstubsdumprecombine; <%= @stages['bigwikis']['rest'] -%> --job tables # regular articles 1 8 <%= @stages['smallwikis']['rest'] -%> --job articlesdump -# regular articles for big wikis -4 2 <%= @stages['bigwikis']['rest'] -%> --job articlesdump -# recombine articles for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job articlesdumprecombine +# regular articles, recombines for big wikis +4 2 <%= @stages['bigwikis']['rest'] -%> --job articlesdump; <%= @stages['bigwikis']['rest'] -%> --job articlesdumprecombine # regular articles multistream 1 8 <%= @stages['smallwikis']['rest'] -%> --job articlesmultistreamdump @@ -27,10 +19,8 @@ # articles plus meta pages 1 8 <%= @stages['smallwikis']['rest'] -%> --job metacurrentdump -# articles plus meta pages for big wikis -4 2 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdump -# recombine articles plus meta pages for big wikis -1 8 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdumprecombine +# articles, recombine plus meta pages for big wikis +4 2 <%= @stages['bigwikis']['rest'] -%> --job metacurrentdump; <%= @stages['bigwikis']['rest'] -%> --job metacurrentdumprecombine <% if @stagestype == 'normal' %> # all remaining jobs -- To view, visit https://gerrit.wikimedia.org/r/233418 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I57940a853749c690bd4ae8f1dd96000d1691515a Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: ArielGlenn <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
