Ottomata has submitted this change and it was merged. Change subject: Going back to using single webrequest camus job. Reduce kafka.max.pull.minutes.per.task to 9 ......................................................................
Going back to using single webrequest camus job. Reduce kafka.max.pull.minutes.per.task to 9 Using multiple jobs at this point could get complicated with camus history files. We'd need to coordinate this change much better. Change-Id: Ifc46428948a8011ba2f231108b5879d9754559d5 --- M camus/camus.webrequest.properties 1 file changed, 3 insertions(+), 1 deletion(-) Approvals: Ottomata: Verified; Looks good to me, approved diff --git a/camus/camus.webrequest.properties b/camus/camus.webrequest.properties index 3911202..861ceaf 100644 --- a/camus/camus.webrequest.properties +++ b/camus/camus.webrequest.properties @@ -57,9 +57,11 @@ # events with a timestamp older than this will be discarded. kafka.max.historical.days=7 # Max minutes for each mapper to pull messages (-1 means no limit) -# Let each mapper run for no more than 55 minutes. +# Let each mapper run for no more than 9 minutes. # Camus creates hourly directories, and we don't want a single # long running mapper keep other Camus jobs from being launched. +# We run Camus every 10 minutes, so limiting it to 9 should keep +# runs fresh. kafka.max.pull.minutes.per.task=55 # if whitelist has values, only whitelisted topic are pulled. nothing on the blacklist is pulled -- To view, visit https://gerrit.wikimedia.org/r/231798 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ifc46428948a8011ba2f231108b5879d9754559d5 Gerrit-PatchSet: 1 Gerrit-Project: analytics/refinery Gerrit-Branch: master Gerrit-Owner: Ottomata <[email protected]> Gerrit-Reviewer: Ottomata <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
