[
https://issues.apache.org/jira/browse/KAFKA-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905989#comment-15905989
]
Apurva Mehta commented on KAFKA-4574:
-------------------------------------
I posted all the information I could glean from the logs. The tl;dr is :
This seems like something similar to KAFKA-1211, but the current logs offer no
definitive proof. We see log truncation to offset 1496, but this is not
accompanied by a quick succession of leader elections in that timeframe. Nor
are there any 'reset offset' messages from the replicas in the log. So it is
hard to explain the log truncations being observed in this test.
It may make sense to enable trace logging for the controller and the state
change log for this test. Then the next time we see this, we would atleast be
sure that we are seeing _all_ the leader elections, and will give us a clear
idea of whether it is KAFKA-1211 or not.
> Transient failure in ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade
> with security_protocol = SASL_PLAINTEXT, SSL
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-4574
> URL: https://issues.apache.org/jira/browse/KAFKA-4574
> Project: Kafka
> Issue Type: Test
> Components: system tests
> Reporter: Shikhar Bhushan
> Assignee: Apurva Mehta
>
> http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-12-29--001.1483003056--apache--trunk--dc55025/report.html
> {{ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade}} failed with these
> {{security_protocol}} parameters
> {noformat}
> ====================================================================================================
> test_id:
> kafkatest.tests.core.zookeeper_security_upgrade_test.ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol=SASL_PLAINTEXT
> status: FAIL
> run time: 3 minutes 44.094 seconds
> 1 acked message did not make it to the Consumer. They are: [5076]. We
> validated that the first 1 of these missing messages correctly made it into
> Kafka's data files. This suggests they were lost on their way to the consumer.
> Traceback (most recent call last):
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
> line 123, in run
> data = self.run_test()
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
> line 176, in run_test
> return self.test_context.function(self.test)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/mark/_mark.py",
> line 321, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/core/zookeeper_security_upgrade_test.py",
> line 117, in test_zk_security_upgrade
> self.run_produce_consume_validate(self.run_zk_migration)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 101, in run_produce_consume_validate
> self.validate()
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 163, in validate
> assert success, msg
> AssertionError: 1 acked message did not make it to the Consumer. They are:
> [5076]. We validated that the first 1 of these missing messages correctly
> made it into Kafka's data files. This suggests they were lost on their way to
> the consumer.
> {noformat}
> {noformat}
> ====================================================================================================
> test_id:
> kafkatest.tests.core.zookeeper_security_upgrade_test.ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol=SSL
> status: FAIL
> run time: 3 minutes 50.578 seconds
> 1 acked message did not make it to the Consumer. They are: [3559]. We
> validated that the first 1 of these missing messages correctly made it into
> Kafka's data files. This suggests they were lost on their way to the consumer.
> Traceback (most recent call last):
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
> line 123, in run
> data = self.run_test()
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py",
> line 176, in run_test
> return self.test_context.function(self.test)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/mark/_mark.py",
> line 321, in wrapper
> return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/core/zookeeper_security_upgrade_test.py",
> line 117, in test_zk_security_upgrade
> self.run_produce_consume_validate(self.run_zk_migration)
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 101, in run_produce_consume_validate
> self.validate()
> File
> "/var/lib/jenkins/workspace/system-test-kafka/kafka/tests/kafkatest/tests/produce_consume_validate.py",
> line 163, in validate
> assert success, msg
> AssertionError: 1 acked message did not make it to the Consumer. They are:
> [3559]. We validated that the first 1 of these missing messages correctly
> made it into Kafka's data files. This suggests they were lost on their way to
> the consumer.
> {noformat}
> Previously: KAFKA-3985
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
