Elukey has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/370631 )

Change subject: Force Kafka protocol version
......................................................................

Force Kafka protocol version

This change is the suggested workaround from the librdkafka authors
to limit the effect of https://issues.apache.org/jira/browse/KAFKA-3547.

More info: 
https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility

Bug: T172681
Change-Id: I7a643fe17f9d70718b885e1de9b85494c70cc041
---
M manifests/instance.pp
M templates/varnishkafka_v4.conf.erb
2 files changed, 19 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet/varnishkafka 
refs/changes/31/370631/1

diff --git a/manifests/instance.pp b/manifests/instance.pp
index 0b25c14..bd078eb 100644
--- a/manifests/instance.pp
+++ b/manifests/instance.pp
@@ -72,6 +72,10 @@
 #                                     changes.  Default: true.
 # $conf_template
 #
+# $force_protocol_version           - The Kafka protocol version used to 
produce events.
+#                                     Suggested workaround for 
https://issues.apache.org/jira/browse/KAFKA-3547
+#                                     (Kafka 0.9.0.[0,1] protocol versions 
affected)
+#
 define varnishkafka::instance(
     $brokers                        = ['localhost:9092'],
     $topic                          = 'varnish',
@@ -111,6 +115,7 @@
 
     $should_subscribe               = true,
     $conf_template                  = 'varnishkafka/varnishkafka.conf.erb',
+    $force_protocol_version         = undef,
 ) {
     require ::varnishkafka
 
diff --git a/templates/varnishkafka_v4.conf.erb 
b/templates/varnishkafka_v4.conf.erb
index e0a9d52..9fe9b9f 100644
--- a/templates/varnishkafka_v4.conf.erb
+++ b/templates/varnishkafka_v4.conf.erb
@@ -236,6 +236,20 @@
 # Valid values are 'none', 'gzip', and 'snappy'.
 kafka.compression.codec = <%= @compression_codec %>
 
+<% if @force_protocol_version -%>
+#
+# Kafka 0.9.0.[0,1] protocol versions may cause
+# https://issues.apache.org/jira/browse/KAFKA-3547. This workaround is 
suggested
+# by the author of librdkafka and forces the protocol used to produce events
+# to a specific version. The goal is to avoid any issue like the one described
+# in https://phabricator.wikimedia.org/T172681
+# More info 
https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility
+kafka.api.version.request=false
+kafka.broker.version.fallback=<%= @force_protocol_version %>
+
+<% end -%>
+
+
 #
 # Topic configuration
 #

-- 
To view, visit https://gerrit.wikimedia.org/r/370631
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7a643fe17f9d70718b885e1de9b85494c70cc041
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet/varnishkafka
Gerrit-Branch: master
Gerrit-Owner: Elukey <ltosc...@wikimedia.org>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

Reply via email to