[
https://issues.apache.org/jira/browse/KAFKA-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038146#comment-16038146
]
Apurva Mehta edited comment on KAFKA-5375 at 6/6/17 4:55 AM:
-------------------------------------------------------------
This is a red herring. The problem was with the consumer timeout setting in the
python consumer wrapper. It was set to 5 seconds. When there are broker
bounces, the transactions can't commit for a while. This often crosses the 5
second threshold, at which point the consumer closes, causing this error.
However, there are still failures due to missing messages in the concurrent
consumer only when there are broker hard bounces. So this ticket will be left
open.
was (Author: apurva):
This is a red herring. The problem was with the consumer timeout setting in the
python consumer wrapper. It was set to 5 seconds. When there are broker
bounces, the transactions can't commit for a while. This often crosses the 5
second threshold, at which point the consumer closes, causing this error.
However, there are still failures of this test due to other reasons.
> Transactions: Concurrent transactional consumer loses messages when there are
> broker bounces
> --------------------------------------------------------------------------------------------
>
> Key: KAFKA-5375
> URL: https://issues.apache.org/jira/browse/KAFKA-5375
> Project: Kafka
> Issue Type: Bug
> Reporter: Apurva Mehta
> Assignee: Apurva Mehta
> Priority: Blocker
> Labels: exactly-once
> Fix For: 0.11.0.0
>
>
> With the system test modifications in KAFKA-5366, the concurrent reader
> almost always consumes a fraction of the expected messages when there are
> broker bounces. A consumer running without concurrent writes consumes all the
> messages from the topic in question in the same test.
> {noformat}
> ================================================================================
> test_id:
> kafkatest.tests.core.transactions_test.TransactionsTest.test_transactions.failure_mode=clean_bounce.bounce_target=brokers
> status: FAIL
> run time: 1 minute 59.169 seconds
> Input and concurrently consumed output message sets are not equal. Num
> input messages: 20000. Num concurrently_consumed_messages: 0
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> line 123, in run
> data = self.run_test()
> File
> "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py",
> line 176, in run_test
> return self.test_context.function(self.test)
> File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py", line
> 321, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File "/opt/kafka-dev/tests/kafkatest/tests/core/transactions_test.py", line
> 238, in test_transactions
> (len(input_message_set), len(concurrently_consumed_message_set))
> AssertionError: Input and concurrently consumed output message sets are not
> equal. Num input messages: 20000. Num concurrently_consumed_messages: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
