Ottomata has uploaded a new change for review.
https://gerrit.wikimedia.org/r/303610
Change subject: Increase retries for eventlogging analytics processor kafka
producers
......................................................................
Increase retries for eventlogging analytics processor kafka producers
Bug: T141285
Change-Id: Iedf0d8664a2c438899043f573daf072b4d86cf87
---
M manifests/role/eventlogging.pp
1 file changed, 10 insertions(+), 2 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet
refs/changes/10/303610/1
diff --git a/manifests/role/eventlogging.pp b/manifests/role/eventlogging.pp
index 8069bc9..70c1447 100644
--- a/manifests/role/eventlogging.pp
+++ b/manifests/role/eventlogging.pp
@@ -155,13 +155,21 @@
'eventlogging_client_side_processors',
['client-side-00', 'client-side-01']
)
+
+ # Increase number and backoff time of retries for async
+ # analytics uses. If metadata changes, we should give
+ # more time to retry. NOTE: testing this in production
+ # STILL yielded some dropped messages. Need to figure
+ # out why and stop it. This either needs to be higher,
+ # or it is a bug in kafka-python.
+ $kafka_producer_args = 'retries=6&retry_backoff_ms=200'
eventlogging::service::processor { $client_side_processors:
format => '%q %{recvFrom}s %{seqId}d %t %o %{userAgent}i',
input => $kafka_client_side_raw_uri,
sid => $kafka_consumer_group,
outputs => [
- $kafka_schema_uri,
- $kafka_mixed_uri,
+ "${kafka_schema_uri}&${kafka_producer_args}",
+ "${kafka_mixed_uri}&${kafka_producer_args}",
],
output_invalid => true,
}
--
To view, visit https://gerrit.wikimedia.org/r/303610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iedf0d8664a2c438899043f573daf072b4d86cf87
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
