[kafka] branch 2.5 updated: KAFKA-10292: Set min.insync.replicas to 1 of __consumer_offsets (#9286)

[guozhang]

This is an automated email from the ASF dual-hosted git repository.

guozhang pushed a commit to branch 2.5
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/2.5 by this push:
     new 64b24ce  KAFKA-10292: Set min.insync.replicas to 1 of 
__consumer_offsets (#9286)
64b24ce is described below

commit 64b24cef6d3ddba42be07bd5fb0a66c658fa8fc6
Author: Bruno Cadonna <br...@confluent.io>
AuthorDate: Tue Sep 15 20:12:37 2020 +0200

    KAFKA-10292: Set min.insync.replicas to 1 of __consumer_offsets (#9286)
    
    The test StreamsBrokerBounceTest.test_all_brokers_bounce() fails on
    2.5 because in the last stage of the test there is only one broker
    left and the offset commit cannot succeed because the
    min.insync.replicas of __consumer_offsets is set to 2 and acks is
    set to all. This causes a time out and extends the closing of the
    Kafka Streams client to beyond the duration passed to the close
    method of the client.
    
    This affects especially the 2.5 branch since there Kafka Streams
    commits offsets for each task, i.e., close() needs to wait for the
    timeout for each task. In 2.6 and trunk the offset commit is done
    per thread, so close() does only need to wait for one time out per
    stream thread.
    
    I opened this PR on trunk, since the test could also become
    flaky on trunk and we want to avoid diverging system tests across
    branches.
    
    A more complete solution would be to improve the test by defining
    a better success criteria.
    
    Reviewers: Guozhang Wang <wangg...@gmail.com>
---
 tests/kafkatest/tests/streams/streams_broker_bounce_test.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tests/kafkatest/tests/streams/streams_broker_bounce_test.py 
b/tests/kafkatest/tests/streams/streams_broker_bounce_test.py
index 9b624fe..2e3402f 100644
--- a/tests/kafkatest/tests/streams/streams_broker_bounce_test.py
+++ b/tests/kafkatest/tests/streams/streams_broker_bounce_test.py
@@ -275,7 +275,14 @@ class StreamsBrokerBounceTest(Test):
         Start a smoke test client, then kill a few brokers and ensure data is 
still received
         Record if records are delivered
         """
-        self.setup_system() 
+
+        # Set min.insync.replicas to 1 because in the last stage of the test 
there is only one broker left.
+        # Otherwise the last offset commit will never succeed and time out and 
potentially take longer as
+        # duration passed to the close method of the Kafka Streams client.
+        self.topics['__consumer_offsets'] = { 'partitions': 50, 
'replication-factor': self.replication,
+                                              'configs': 
{"min.insync.replicas": 1} }
+
+        self.setup_system()
 
         # Sleep to allow test to run for a bit
         time.sleep(120)