Ottomata has submitted this change and it was merged.
Change subject: Increase retries for eventlogging analytics processor kafka
producers
......................................................................
Increase retries for eventlogging analytics processor kafka producers
Bug: T141285
Change-Id: Iedf0d8664a2c438899043f573daf072b4d86cf87
---
M manifests/role/eventlogging.pp
1 file changed, 11 insertions(+), 2 deletions(-)
Approvals:
Ottomata: Verified; Looks good to me, approved
diff --git a/manifests/role/eventlogging.pp b/manifests/role/eventlogging.pp
index 8069bc9..4ccb34c 100644
--- a/manifests/role/eventlogging.pp
+++ b/manifests/role/eventlogging.pp
@@ -155,13 +155,22 @@
'eventlogging_client_side_processors',
['client-side-00', 'client-side-01']
)
+
+ # Increase number and backoff time of retries for async
+ # analytics uses. If metadata changes, we should give
+ # more time to retry. NOTE: testing this in production
+ # STILL yielded some dropped messages. Need to figure
+ # out why and stop it. This either needs to be higher,
+ # or it is a bug in kafka-python.
+ # See: https://phabricator.wikimedia.org/T142430
+ $kafka_producer_args = 'retries=6&retry_backoff_ms=200'
eventlogging::service::processor { $client_side_processors:
format => '%q %{recvFrom}s %{seqId}d %t %o %{userAgent}i',
input => $kafka_client_side_raw_uri,
sid => $kafka_consumer_group,
outputs => [
- $kafka_schema_uri,
- $kafka_mixed_uri,
+ "${kafka_schema_uri}&${kafka_producer_args}",
+ "${kafka_mixed_uri}&${kafka_producer_args}",
],
output_invalid => true,
}
--
To view, visit https://gerrit.wikimedia.org/r/303610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iedf0d8664a2c438899043f573daf072b4d86cf87
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
