[jira] [Commented] (KAFKA-5366) Add cases for concurrent transactional reads and writes in system tests
[ https://issues.apache.org/jira/browse/KAFKA-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039916#comment-16039916 ] ASF GitHub Bot commented on KAFKA-5366: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3217 > Add cases for concurrent transactional reads and writes in system tests > --- > > Key: KAFKA-5366 > URL: https://issues.apache.org/jira/browse/KAFKA-5366 > Project: Kafka > Issue Type: Test >Affects Versions: 0.11.0.0 >Reporter: Apurva Mehta >Assignee: Apurva Mehta >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > Currently the transactions system test does a transactional copy while > bouncing brokers and clients, and then does a verifying read on the output > topic to ensure that it exactly matches the input. > We should also have a transactional consumer reading the tail of the output > topic as the writes are happening, and then assert that the values _it_ reads > also exactly match the values in the source topics. > This test really exercises the abort index, and we don't have any of them in > the system or integration tests right now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5366) Add cases for concurrent transactional reads and writes in system tests
[ https://issues.apache.org/jira/browse/KAFKA-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037403#comment-16037403 ] Apurva Mehta commented on KAFKA-5366: - [~ijuma] I originally marked this as a 'Major' item for 0.11.0.1, but am upgrading it to a blocker for 0.11.0.0 . The reason is that with these improvements, we are testing a behavior which would be used at the core of streams EOS (reading and writing transactionally and concurrently at the tail of the log). If this behavior doesn't work, streams EOS won't work. So we need this upgraded tests to pass (have filed KAFKA-5375 and KAFKA-5376 to fix the failures), and then we can merge this. Let me know if you have any objections. > Add cases for concurrent transactional reads and writes in system tests > --- > > Key: KAFKA-5366 > URL: https://issues.apache.org/jira/browse/KAFKA-5366 > Project: Kafka > Issue Type: Test >Affects Versions: 0.11.0.0 >Reporter: Apurva Mehta >Assignee: Apurva Mehta >Priority: Blocker > Labels: exactly-once > Fix For: 0.11.0.0 > > > Currently the transactions system test does a transactional copy while > bouncing brokers and clients, and then does a verifying read on the output > topic to ensure that it exactly matches the input. > We should also have a transactional consumer reading the tail of the output > topic as the writes are happening, and then assert that the values _it_ reads > also exactly match the values in the source topics. > This test really exercises the abort index, and we don't have any of them in > the system or integration tests right now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KAFKA-5366) Add cases for concurrent transactional reads and writes in system tests
[ https://issues.apache.org/jira/browse/KAFKA-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035542#comment-16035542 ] ASF GitHub Bot commented on KAFKA-5366: --- GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3217 KAFKA-5366: Add concurrent reads to transactions system test This currently fails in multiple ways. One of which is most likely KAFKA-5355, where the concurrent consumer reads duplicates. During broker bounces, the concurrent consumer misses messages completely. This is another bug. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apurvam/kafka KAFKA-5366-add-concurrent-reads-to-transactions-system-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3217.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3217 commit cd0990784aaa26fa6485e9f369a600b85c1647f9 Author: Apurva Mehta Date: 2017-06-02T22:25:00Z Add a concurrent consumer in the transactions system tests. This will exercise the abort index commit 71fcad197b403ff7873d646feec287d92793cbe6 Author: Apurva Mehta Date: 2017-06-02T22:29:47Z Bounce brokers as well > Add cases for concurrent transactional reads and writes in system tests > --- > > Key: KAFKA-5366 > URL: https://issues.apache.org/jira/browse/KAFKA-5366 > Project: Kafka > Issue Type: Test >Affects Versions: 0.11.0.0 >Reporter: Apurva Mehta >Assignee: Apurva Mehta > Labels: exactly-once > Fix For: 0.11.1.0 > > > Currently the transactions system test does a transactional copy while > bouncing brokers and clients, and then does a verifying read on the output > topic to ensure that it exactly matches the input. > We should also have a transactional consumer reading the tail of the output > topic as the writes are happening, and then assert that the values _it_ reads > also exactly match the values in the source topics. > This test really exercises the abort index, and we don't have any of them in > the system or integration tests right now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
