[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514882473 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] ning2008wisc commented on pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-10-29 Thread GitBox
ning2008wisc commented on pull request #9224: URL: https://github.com/apache/kafka/pull/9224#issuecomment-719201216 A kind reminder for additional feedback and comments, based on my response above This is an automated

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514872477 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514861991 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514861389 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514819326 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] dengziming opened a new pull request #9535: MINOR: remove redundant return statement

2020-10-29 Thread GitBox
dengziming opened a new pull request #9535: URL: https://github.com/apache/kafka/pull/9535 the result of `GroupMetadataManager.storeOffsets` is Unit, so remove the redundant return statement This is an automated message

[jira] [Updated] (KAFKA-6217) Allow consumers to read messages from LEO

2020-10-29 Thread Ashok Bala (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashok Bala updated KAFKA-6217: -- Description: we have a use case(real time customer facing application) to support high

[GitHub] [kafka] ableegoldman opened a new pull request #9534: KAFKA-10664: Delete existing checkpoint when writing empty offsets

2020-10-29 Thread GitBox
ableegoldman opened a new pull request #9534: URL: https://github.com/apache/kafka/pull/9534 ...otherwise we can get stuck in an endless loop of initializing corrupted offsets, hitting OffsetOutOfRangeException and closing the task, then reviving the task with those same corrupted

[jira] [Commented] (KAFKA-10635) Streams application fails with OutOfOrderSequenceException after rolling restarts of brokers

2020-10-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223324#comment-17223324 ] Matthias J. Sax commented on KAFKA-10635: - \cc [~hachikuji] [~bob-barrett] > Streams

[jira] [Created] (KAFKA-10664) Streams fails to overwrite corrupted offsets leading to infinite OffsetOutOfRangeException loop

2020-10-29 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-10664: -- Summary: Streams fails to overwrite corrupted offsets leading to infinite OffsetOutOfRangeException loop Key: KAFKA-10664 URL:

[jira] [Comment Edited] (KAFKA-10645) Forwarding a record from a punctuator sometimes it results in a NullPointerException

2020-10-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17222393#comment-17222393 ] Matthias J. Sax edited comment on KAFKA-10645 at 10/30/20, 1:31 AM:

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514636997 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514636692 ## File path: core/src/main/scala/kafka/network/RequestChannel.scala ## @@ -162,7 +161,7 @@ object RequestChannel extends Logging { } }

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514635784 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] mjsax merged pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax merged pull request #9513: URL: https://github.com/apache/kafka/pull/9513 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mjsax commented on pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax commented on pull request #9513: URL: https://github.com/apache/kafka/pull/9513#issuecomment-719085653 > However, it may be overkill since users don't create ConsumerRecord. Yeah, I tend to agree. Would not hurt to add it, but should not be strictly necessary. Will merge this

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719085324 Cherrypicked to 2.6 & 2.7 cc/ @bbejeck This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] mjsax commented on pull request #9518: KAFKA-10645: Add null check to the array/Iterable values in RecordHeaders constructor

2020-10-29 Thread GitBox
mjsax commented on pull request #9518: URL: https://github.com/apache/kafka/pull/9518#issuecomment-719084829 @showuon Seems we worked on this in parallel. Sorry for that... This is an automated message from the Apache Git

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514621561 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software

[GitHub] [kafka] ableegoldman merged pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman merged pull request #9530: URL: https://github.com/apache/kafka/pull/9530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719074084 One flaky test failed: `EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true]` Confirmed systems test passed. Merging to trunk

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514617626 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514614216 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514610499 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-719063925 Merged to 2.6 (cc @mimaison ) Thanks again for the contribution, @thake ! This is an automated message

[jira] [Updated] (KAFKA-10515) NPE: Foreign key join serde may not be initialized with default serde if application is distributed

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10515: - Fix Version/s: 2.6.1 > NPE: Foreign key join serde may not be initialized with default serde if

[GitHub] [kafka] vvcephei merged pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei merged pull request #9467: URL: https://github.com/apache/kafka/pull/9467 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514594596 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] gitlw opened a new pull request #9533: Show log end offset during truncation to help estimate data loss during ULE

2020-10-29 Thread GitBox
gitlw opened a new pull request #9533: URL: https://github.com/apache/kafka/pull/9533 During Unclean Leader Election, there could be data loss due to truncation at the resigned leader. This PR tries to add more logs to understand the scale of message loss during an unclean leader

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - *

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - *

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514584469 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] bbejeck commented on a change in pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on a change in pull request #9532: URL: https://github.com/apache/kafka/pull/9532#discussion_r514580970 ## File path: docs/upgrade.html ## @@ -21,6 +21,48 @@ Notable changes in 2.8.0 +Upgrading to 2.7.0 from any version 0.8.x through 2.6.x Review

[GitHub] [kafka] bbejeck commented on pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on pull request #9532: URL: https://github.com/apache/kafka/pull/9532#issuecomment-719035513 Note that most of this PR already exists in 2.6 and doesn't need a close look This is an automated message from

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577992 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577021 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514576105 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514575372 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514574814 ## File path: core/src/main/scala/kafka/network/RequestChannel.scala ## @@ -162,7 +161,7 @@ object RequestChannel extends Logging { } } -

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514566700 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514567028 ## File path: streams/src/main/java/org/apache/kafka/streams/errors/StreamsUncaughtExceptionHandler.java ## @@ -0,0 +1,45 @@ +/* + * Licensed to the

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514535737 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -282,6 +284,15 @@ public boolean isRunning() {

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514539737 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -329,8 +387,9 @@ private void appendLeaderChangeMessage(LeaderState

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514537694 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514536828 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514535761 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/ThresholdPurgatory.java ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514533712 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/RecordsBatchReader.java ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514527605 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final

[GitHub] [kafka] bbejeck opened a new pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9532: URL: https://github.com/apache/kafka/pull/9532 For the 2.7 release, we need to migrate some docs changes that went to `kafka-site` but didn't go into `kafka/docs` This PR covers the `documentation.html` and `uprade.html` changes. Once these

[GitHub] [kafka] hachikuji opened a new pull request #9531: KAFKA-10661; Add new resigned state for graceful shutdown/initialization

2020-10-29 Thread GitBox
hachikuji opened a new pull request #9531: URL: https://github.com/apache/kafka/pull/9531 When initializing the raft state machine after shutting down as a leader, we were previously entering the "unattached" state, which means we have no leader and no voted candidate. This was a bug

[GitHub] [kafka] abbccdda merged pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda merged pull request #9529: URL: https://github.com/apache/kafka/pull/9529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514518360 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Created] (KAFKA-10663) Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures

2020-10-29 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10663: --- Summary: Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures Key: KAFKA-10663 URL: https://issues.apache.org/jira/browse/KAFKA-10663 Project: Kafka

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514516570 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514515763 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514514051 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514512213 ## File path: clients/src/main/java/org/apache/kafka/common/metrics/stats/Percentiles.java ## @@ -113,11 +113,11 @@ protected HistogramSample

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514508755 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AddOffsetsToTxnRequest.java ## @@ -57,6 +57,10 @@ public

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718965450 System tests run: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4276/ This is an automated

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514504593 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718961222 This fix needs to be cherry-picked to 2.7 and 2.6 This is an automated message from the Apache Git Service. To

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718960604 Call for review: @lct45 @ableegoldman @vvcephei This is an automated message from the Apache Git Service. To

[GitHub] [kafka] cadonna opened a new pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna opened a new pull request #9530: URL: https://github.com/apache/kafka/pull/9530 The system test StreamsUpgradeTest.test_version_probing_upgrade tries to verify the wrong version for version probing. ### Committer Checklist (excluded from commit message) - [ ] Verify

[jira] [Created] (KAFKA-10662) Possible hanging test in 2.6 on JDK 11

2020-10-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10662: Summary: Possible hanging test in 2.6 on JDK 11 Key: KAFKA-10662 URL: https://issues.apache.org/jira/browse/KAFKA-10662 Project: Kafka Issue Type: Bug

[GitHub] [kafka] senthilm-ms commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
senthilm-ms commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718944686 > @rajinisivaram : What do you think? Thanks. @junrao @rajinisivaram - take a look at the stack trace mentioned in the JIRA bug where you don't see the extension/custom

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514482551 ## File path: core/src/main/scala/kafka/raft/TimingWheelExpirationService.scala ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] junrao commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
junrao commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718942968 @rajinisivaram : What do you think? Thanks. This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r513200010 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] junrao commented on pull request #6915: KAFKA-8334 Executor to retry delayed operations failed to obtain lock

2020-10-29 Thread GitBox
junrao commented on pull request #6915: URL: https://github.com/apache/kafka/pull/6915#issuecomment-718933493 @yinchuanwang : The short answer is that the retry logic could cause a watcher event not to be triggered. If there is no future event triggering the same watcher, the

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514467652 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[jira] [Created] (KAFKA-10661) Add resigned state to raft state machine to preserve leader/epoch information

2020-10-29 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-10661: --- Summary: Add resigned state to raft state machine to preserve leader/epoch information Key: KAFKA-10661 URL: https://issues.apache.org/jira/browse/KAFKA-10661

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223094#comment-17223094 ] Jun Rao commented on KAFKA-9892: [~kbrajesh176]: Thanks for your interest. Just added you to Kafka

[GitHub] [kafka] vvcephei merged pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei merged pull request #9426: URL: https://github.com/apache/kafka/pull/9426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223070#comment-17223070 ] Brajesh Kumar commented on KAFKA-9892: --- [~junrao] [~ijuma] Can I pick this up please? It seems

[GitHub] [kafka] vvcephei commented on pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei commented on pull request #9426: URL: https://github.com/apache/kafka/pull/9426#issuecomment-718900823 The only build failure was: `Build / JDK 11 /

[jira] [Resolved] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10638. -- Resolution: Fixed > QueryableStateIntegrationTest fails due to stricter store checking >

[jira] [Updated] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10638: - Fix Version/s: (was: 2.7.1) (was: 2.8.0) 2.7.0 >

[jira] [Commented] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223044#comment-17223044 ] Brajesh Kumar commented on KAFKA-9948: --- [~dulvinw] Which command you are using to build kafka?

[jira] [Comment Edited] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223044#comment-17223044 ] Brajesh Kumar edited comment on KAFKA-9948 at 10/29/20, 5:02 PM: --

[GitHub] [kafka] vvcephei commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-10-29 Thread GitBox
vvcephei commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-718889494 Hey @dongjinleekr , just a quick check-in: do you plan to take a look at the test failures? This is an

[jira] [Commented] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223041#comment-17223041 ] Brajesh Kumar commented on KAFKA-10373: Hi What is your Environment configurations(Linux/MAC

[jira] [Comment Edited] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223041#comment-17223041 ] Brajesh Kumar edited comment on KAFKA-10373 at 10/29/20, 4:59 PM: --- Hi

[GitHub] [kafka] vvcephei merged pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei merged pull request #9521: URL: https://github.com/apache/kafka/pull/9521 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei commented on pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on pull request #9521: URL: https://github.com/apache/kafka/pull/9521#issuecomment-718885978 The tests passed, and I answered the two review questions. I'll go ahead and merge. This is an automated

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-718882348 _Finally_ merged that Jenkinsfile PR (seeing a lot of timeouts, but decided just to go ahead and merge it). Merged in the 2.6 branch, and now the tests are running! We

[GitHub] [kafka] vvcephei merged pull request #9471: MINOR: Add Jenkinsfile to 2.6

2020-10-29 Thread GitBox
vvcephei merged pull request #9471: URL: https://github.com/apache/kafka/pull/9471 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Created] (KAFKA-10660) Poll time out logstash

2020-10-29 Thread David (Jira)
David created KAFKA-10660: - Summary: Poll time out logstash Key: KAFKA-10660 URL: https://issues.apache.org/jira/browse/KAFKA-10660 Project: Kafka Issue Type: Bug Components: KafkaConnect

[GitHub] [kafka] abbccdda commented on pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda commented on pull request #9529: URL: https://github.com/apache/kafka/pull/9529#issuecomment-718871156 @bbejeck @hachikuji for a review This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] abbccdda opened a new pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda opened a new pull request #9529: URL: https://github.com/apache/kafka/pull/9529 Need to make sure 2.7 doesn't bump the request header unexpectedly. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

[GitHub] [kafka] abbccdda commented on pull request #9522: MINOR: revert initial principal PR

2020-10-29 Thread GitBox
abbccdda commented on pull request #9522: URL: https://github.com/apache/kafka/pull/9522#issuecomment-718869803 Reopening this PR to merge to 2.7 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] bbejeck opened a new pull request #9528: MINOR: Add releaseTarGz to args for building docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9528: URL: https://github.com/apache/kafka/pull/9528 This PR adds the `releaseTarGz` command back into the `release.py::command_stage_docs` method when building docs for staging *Summary of testing strategy (including rationale) for the feature

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-718828042 @chia7712 One thing that would be useful is running the producer-performance test, just to make sure the the performance is inline. Might be worth checking flame graphs as

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514326022 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] vvcephei commented on pull request #9471: MINOR: Add Jenkinsfile to 2.6

2020-10-29 Thread GitBox
vvcephei commented on pull request #9471: URL: https://github.com/apache/kafka/pull/9471#issuecomment-718826291 Ok, I've run it a total of 5 times now, and it has passed two times and timed out three times. I did some analysis on the timed out logs, and found that every test that

[GitHub] [kafka] vvcephei commented on a change in pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on a change in pull request #9521: URL: https://github.com/apache/kafka/pull/9521#discussion_r514325710 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StoreQueryIntegrationTest.java ## @@ -153,51 +179,75 @@ public void

[GitHub] [kafka] vvcephei commented on a change in pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on a change in pull request #9521: URL: https://github.com/apache/kafka/pull/9521#discussion_r514323639 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StoreQueryIntegrationTest.java ## @@ -117,17 +126,34 @@ public void

  1   2   >