Ottomata has submitted this change and it was merged. Change subject: Decrease throttle for webrequest log verification ......................................................................
Decrease throttle for webrequest log verification The cluster default is unnecessarily high for this job, and only bought us jobs that unnecessarily check hdfs again and again. Change-Id: I254e96c4c92c3e842581570c54fc378e609747de --- M oozie/webrequest/partition/add/coordinator.xml 1 file changed, 11 insertions(+), 0 deletions(-) Approvals: Ottomata: Verified; Looks good to me, approved diff --git a/oozie/webrequest/partition/add/coordinator.xml b/oozie/webrequest/partition/add/coordinator.xml index db3577c..ab050ab 100644 --- a/oozie/webrequest/partition/add/coordinator.xml +++ b/oozie/webrequest/partition/add/coordinator.xml @@ -56,6 +56,17 @@ even-though the concurrency is low. --> <concurrency>2</concurrency> + + + <!-- + Since we expect only one incarnation per hourly dataset, the + default throttle of 12 is way to high, and there is not need + to keep that many materialized jobs around. + + By resorting to 2, we keep the hdfs checks on the datasets + low, while still being able to easily feed the concurrency. + --> + <throttle>2</throttle> </controls> <datasets> -- To view, visit https://gerrit.wikimedia.org/r/148700 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I254e96c4c92c3e842581570c54fc378e609747de Gerrit-PatchSet: 2 Gerrit-Project: analytics/refinery Gerrit-Branch: master Gerrit-Owner: QChris <christ...@quelltextlich.at> Gerrit-Reviewer: Ottomata <o...@wikimedia.org> Gerrit-Reviewer: QChris <christ...@quelltextlich.at> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits