[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184574#comment-15184574 ] ASF GitHub Bot commented on KAFKA-3201: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/980 > Add system test for KIP-31 and KIP-32 - Upgrade Test > > > Key: KAFKA-3201 > URL: https://issues.apache.org/jira/browse/KAFKA-3201 > Project: Kafka > Issue Type: Sub-task > Components: system tests >Reporter: Jiangjie Qin >Assignee: Anna Povzner >Priority: Blocker > Fix For: 0.10.0.0 > > > This system test should test the procedure to upgrade a Kafka broker from > 0.8.x and 0.9.0 to 0.10.0 > The procedure is documented in KIP-32: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170091#comment-15170091 ] ASF GitHub Bot commented on KAFKA-3201: --- GitHub user apovzner opened a pull request: https://github.com/apache/kafka/pull/980 KAFKA-3201: Added rolling upgrade system tests from 0.8 and 0.9 to 0.10 Three main tests: 1. Setup: Producer (0.8) → Kafka Cluster → Consumer (0.8) First rolling bounce: Set inter.broker.protocol.version = 0.8 and message.format.version = 0.8 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 2. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 3. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus: use inter.broker.protocol.version = 0.10 and message.format.version = 0.9 Plus couple of variations of these tests using old/new consumer or no compression / snappy compression. Also added optional extra verification to ProduceConsumeValidate test to verify that all acks received by producer are successful. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apovzner/kafka kafka-3201-02 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/980.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #980 commit 35e7362e316419675cf6614787fcc2d12fae6e74 Author: Anna Povzner Date: 2016-02-26T22:09:14Z KAFKA-3201: Added rolling upgrade system tests from 0.8 and 0.9 to 0.10 commit 208a50458ecff8ef1bf9b601c1162e796ad7de28 Author: Anna Povzner Date: 2016-02-26T22:59:22Z Upgrade system tests ensure all producer acks are successful commit dce6ff016c575aae30587c92f71159886158972c Author: Anna Povzner Date: 2016-02-26T23:18:37Z Using one producer in upgrade test, because --prefixValue is only supported in verifiable producer in trunk > Add system test for KIP-31 and KIP-32 - Upgrade Test > > > Key: KAFKA-3201 > URL: https://issues.apache.org/jira/browse/KAFKA-3201 > Project: Kafka > Issue Type: Sub-task > Components: system tests >Reporter: Jiangjie Qin >Assignee: Anna Povzner > Fix For: 0.10.0.0 > > > This system test should test the procedure to upgrade a Kafka broker from > 0.8.x and 0.9.0 to 0.10.0 > The procedure is documented in KIP-32: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15160007#comment-15160007 ] Jiangjie Qin commented on KAFKA-3201: - [~apovzner] Yes, that was what I had in mind. Thanks for confirming. > Add system test for KIP-31 and KIP-32 - Upgrade Test > > > Key: KAFKA-3201 > URL: https://issues.apache.org/jira/browse/KAFKA-3201 > Project: Kafka > Issue Type: Sub-task > Components: system tests >Reporter: Jiangjie Qin >Assignee: Anna Povzner > Fix For: 0.10.0.0 > > > This system test should test the procedure to upgrade a Kafka broker from > 0.8.x and 0.9.0 to 0.10.0 > The procedure is documented in KIP-32: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159416#comment-15159416 ] Anna Povzner commented on KAFKA-3201: - Here is a set of tests: 1. Setup: Producer (0.8) → Kafka Cluster → Consumer (0.8) First rolling bounce: Set inter.broker.protocol.version = 0.8 and message.format.version = 0.8 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 2. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 3. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus: use inter.broker.protocol.version = 0.10 and message.format.version = 0.9 All three tests will be producing/consuming in the background; in the end the messages consumed will be validated; we will use a mix of producers: using compression and not using compression. [~becket_qin] I understand the test where message format changes on the fly is KAFKA-3202, so I will not do a third phase in setup 3 to move to 0.10 message format (because it will be covered in KAFKA-3202). Is my understanding correct? > Add system test for KIP-31 and KIP-32 - Upgrade Test > > > Key: KAFKA-3201 > URL: https://issues.apache.org/jira/browse/KAFKA-3201 > Project: Kafka > Issue Type: Sub-task > Components: system tests >Reporter: Jiangjie Qin >Assignee: Anna Povzner > Fix For: 0.10.0.0 > > > This system test should test the procedure to upgrade a Kafka broker from > 0.8.x and 0.9.0 to 0.10.0 > The procedure is documented in KIP-32: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3201) Add system test for KIP-31 and KIP-32 - Upgrade Test
[ https://issues.apache.org/jira/browse/KAFKA-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15153249#comment-15153249 ] Anna Povzner commented on KAFKA-3201: - Actually, it makes more sense to do this one before the compatibility tests (KAFKA-3188). I will assign to myself and start working on it. > Add system test for KIP-31 and KIP-32 - Upgrade Test > > > Key: KAFKA-3201 > URL: https://issues.apache.org/jira/browse/KAFKA-3201 > Project: Kafka > Issue Type: Sub-task > Components: system tests >Reporter: Jiangjie Qin > Fix For: 0.10.0.0 > > > This system test should test the procedure to upgrade a Kafka broker from > 0.8.x and 0.9.0 to 0.10.0 > The procedure is documented in KIP-32: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-32+-+Add+timestamps+to+Kafka+message -- This message was sent by Atlassian JIRA (v6.3.4#6332)