[GitHub] [kafka] cmccabe commented on a change in pull request #10184: MINOR: enable topic deletion in the KIP-500 controller

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10184: URL: https://github.com/apache/kafka/pull/10184#discussion_r580825461 ## File path: config/log4j.properties ## @@ -61,11 +61,11 @@ log4j.appender.authorizerAppender.layout.ConversionPattern=[%d] %p %m (%c)%n

[GitHub] [kafka] omkreddy opened a new pull request #10188: MINOR: Update HttpClient to "4.5.13"

2021-02-22 Thread GitBox
omkreddy opened a new pull request #10188: URL: https://github.com/apache/kafka/pull/10188 Update HttpClient to recent bug fix version 4.5.13. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[GitHub] [kafka] showuon commented on pull request #10172: MINOR: add toString to Subscription classes

2021-02-22 Thread GitBox
showuon commented on pull request #10172: URL: https://github.com/apache/kafka/pull/10172#issuecomment-783954552 @chia7712 , thanks for the comments. Please take a look again. Thanks. This is an automated message from the

[GitHub] [kafka] showuon commented on a change in pull request #10172: MINOR: add toString to Subscription classes

2021-02-22 Thread GitBox
showuon commented on a change in pull request #10172: URL: https://github.com/apache/kafka/pull/10172#discussion_r580802728 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerPartitionAssignor.java ## @@ -132,6 +132,16 @@ public void

[GitHub] [kafka] showuon commented on a change in pull request #10172: MINOR: add toString to Subscription classes

2021-02-22 Thread GitBox
showuon commented on a change in pull request #10172: URL: https://github.com/apache/kafka/pull/10172#discussion_r580802108 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerPartitionAssignor.java ## @@ -132,6 +132,16 @@ public void

[GitHub] [kafka] guozhangwang commented on a change in pull request #10170: KAFKA-12323: Set timestamp in record context when punctuate

2021-02-22 Thread GitBox
guozhangwang commented on a change in pull request #10170: URL: https://github.com/apache/kafka/pull/10170#discussion_r580793422 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -792,7 +790,16 @@ public void

[GitHub] [kafka] showuon commented on pull request #10167: KAFKA-12261: Mention about potential delivery loss on partition split when auto.offset.reset = latest

2021-02-22 Thread GitBox
showuon commented on pull request #10167: URL: https://github.com/apache/kafka/pull/10167#issuecomment-783951911 > What do you mean by "splitting partitions"? @ijuma , @ocadaruma is saying that the partition number is increased/decreased. You are right, the phrase **splitting

[GitHub] [kafka] ijuma commented on pull request #10167: KAFKA-12261: Mention about potential delivery loss on partition split when auto.offset.reset = latest

2021-02-22 Thread GitBox
ijuma commented on pull request #10167: URL: https://github.com/apache/kafka/pull/10167#issuecomment-783944686 What do you mean by "splitting partitions"? This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ijuma commented on pull request #10169: MINOR: Update Scala to 2.13.5

2021-02-22 Thread GitBox
ijuma commented on pull request #10169: URL: https://github.com/apache/kafka/pull/10169#issuecomment-783943805 Merged to trunk and 2.8 branches. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] modax commented on pull request #9904: KAFKA-12211: don't change perm for base/state dir when no persistent store

2021-02-22 Thread GitBox
modax commented on pull request #9904: URL: https://github.com/apache/kafka/pull/9904#issuecomment-783943018 Hi, Is there any reason this is not in 2.6? 2.6.1 is affected by this problem. Thanks! This is an

[GitHub] [kafka] showuon commented on pull request #10167: KAFKA-12261: Mention about potential delivery loss on partition split when auto.offset.reset = latest

2021-02-22 Thread GitBox
showuon commented on pull request #10167: URL: https://github.com/apache/kafka/pull/10167#issuecomment-783941027 @ijuma , could you take a look at this PR? Thank you. This is an automated message from the Apache Git Service.

[GitHub] [kafka] chia7712 commented on a change in pull request #10183: KAFKA-12267; Implement `DescribeTransactions` API

2021-02-22 Thread GitBox
chia7712 commented on a change in pull request #10183: URL: https://github.com/apache/kafka/pull/10183#discussion_r580788136 ## File path: clients/src/main/resources/common/message/DescribeProducersResponse.json ## @@ -35,7 +35,7 @@ { "name": "ErrorMessage", "type":

[GitHub] [kafka] ijuma merged pull request #10169: MINOR: Update Scala to 2.13.5

2021-02-22 Thread GitBox
ijuma merged pull request #10169: URL: https://github.com/apache/kafka/pull/10169 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] guozhangwang commented on a change in pull request #10137: KAFKA-12268: Implement task idling semantics via currentLag API

2021-02-22 Thread GitBox
guozhangwang commented on a change in pull request #10137: URL: https://github.com/apache/kafka/pull/10137#discussion_r580786789 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java ## @@ -243,6 +244,11 @@ */ Map endOffsets(Collection

[GitHub] [kafka] ijuma commented on pull request #10169: MINOR: Update Scala to 2.13.5

2021-02-22 Thread GitBox
ijuma commented on pull request #10169: URL: https://github.com/apache/kafka/pull/10169#issuecomment-783932706 Unrelated flaky failures: > Build / JDK 15 / kafka.server.ListOffsetsRequestTest.testResponseIncludesLeaderEpoch() > Build / JDK 11 /

[GitHub] [kafka] chia7712 commented on a change in pull request #10183: KAFKA-12267; Implement `DescribeTransactions` API

2021-02-22 Thread GitBox
chia7712 commented on a change in pull request #10183: URL: https://github.com/apache/kafka/pull/10183#discussion_r580779375 ## File path: clients/src/main/java/org/apache/kafka/common/requests/DescribeTransactionsRequest.java ## @@ -0,0 +1,83 @@ +/* + * Licensed to the

[GitHub] [kafka] guozhangwang commented on a change in pull request #10137: KAFKA-12268: Implement task idling semantics via currentLag API

2021-02-22 Thread GitBox
guozhangwang commented on a change in pull request #10137: URL: https://github.com/apache/kafka/pull/10137#discussion_r580782275 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/Consumer.java ## @@ -243,6 +244,11 @@ */ Map endOffsets(Collection

[GitHub] [kafka] ijuma commented on a change in pull request #10184: MINOR: enable topic deletion in the KIP-500 controller

2021-02-22 Thread GitBox
ijuma commented on a change in pull request #10184: URL: https://github.com/apache/kafka/pull/10184#discussion_r580781460 ## File path: config/log4j.properties ## @@ -61,11 +61,11 @@ log4j.appender.authorizerAppender.layout.ConversionPattern=[%d] %p %m (%c)%n

[GitHub] [kafka] ijuma commented on pull request #10179: MINOR: tune KIP-631 configurations

2021-02-22 Thread GitBox
ijuma commented on pull request #10179: URL: https://github.com/apache/kafka/pull/10179#issuecomment-783923893 @hachikuji can you please review these timeouts? This is an automated message from the Apache Git Service. To

[GitHub] [kafka] cmccabe commented on pull request #10179: MINOR: tune KIP-631 configurations

2021-02-22 Thread GitBox
cmccabe commented on pull request #10179: URL: https://github.com/apache/kafka/pull/10179#issuecomment-783922535 I added some JavaDoc to the RaftConfig class This is an automated message from the Apache Git Service. To

[GitHub] [kafka] ijuma commented on pull request #10174: KAFKA-12357: Do not inline methods from the scala package by default

2021-02-22 Thread GitBox
ijuma commented on pull request #10174: URL: https://github.com/apache/kafka/pull/10174#issuecomment-783892451 Merged to trunk and 2.8 branches. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ijuma merged pull request #10174: KAFKA-12357: Do not inline methods from the scala package by default

2021-02-22 Thread GitBox
ijuma merged pull request #10174: URL: https://github.com/apache/kafka/pull/10174 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] chia7712 commented on pull request #10174: KAFKA-12357: Do not inline methods from the scala package by default

2021-02-22 Thread GitBox
chia7712 commented on pull request #10174: URL: https://github.com/apache/kafka/pull/10174#issuecomment-783879649 > It seems better to aim for ease of use until there's a better way of handling this make sense. be a friendly libraries :)

[GitHub] [kafka] ijuma edited a comment on pull request #10174: KAFKA-12357: Do not inline methods from the scala package by default

2021-02-22 Thread GitBox
ijuma edited a comment on pull request #10174: URL: https://github.com/apache/kafka/pull/10174#issuecomment-783878266 @chia7712 For better or worse, libraries like https://github.com/embeddedkafka/embedded-kafka are often used. And there isn't a great alternative. Getting an exception

[GitHub] [kafka] ijuma commented on pull request #10174: KAFKA-12357: Do not inline methods from the scala package by default

2021-02-22 Thread GitBox
ijuma commented on pull request #10174: URL: https://github.com/apache/kafka/pull/10174#issuecomment-783878266 @chia7712 For better or worse, libraries like https://github.com/embeddedkafka/embedded-kafka are often used. And there isn't a great alternative. Getting an exception when

[GitHub] [kafka] chia7712 commented on a change in pull request #10172: MINOR: add toString to Subscription classes

2021-02-22 Thread GitBox
chia7712 commented on a change in pull request #10172: URL: https://github.com/apache/kafka/pull/10172#discussion_r580767910 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerPartitionAssignor.java ## @@ -132,6 +132,16 @@ public void

[GitHub] [kafka] chia7712 merged pull request #10166: MINOR: update the memberMetadata output

2021-02-22 Thread GitBox
chia7712 merged pull request #10166: URL: https://github.com/apache/kafka/pull/10166 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] hachikuji merged pull request #9912: MINOR: Move `RequestChannel.Response` creation logic into `RequestChannel`

2021-02-22 Thread GitBox
hachikuji merged pull request #9912: URL: https://github.com/apache/kafka/pull/9912 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji commented on pull request #10183: KAFKA-12267; Implement `DescribeTransactions` API

2021-02-22 Thread GitBox
hachikuji commented on pull request #10183: URL: https://github.com/apache/kafka/pull/10183#issuecomment-783868752 @dengziming Good call. I've added a few test cases. I think we were missing authorization of the topics included in the response.

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

2021-02-22 Thread GitBox
dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r580761072 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -92,4 +93,13 @@ public static Path createTempFile(Path logDir,

[GitHub] [kafka] chia7712 commented on pull request #10146: HOTFIX: fix Scala 2.12 build error caused by ControllerApisTest.scala

2021-02-22 Thread GitBox
chia7712 commented on pull request #10146: URL: https://github.com/apache/kafka/pull/10146#issuecomment-783863300 The 2.8 build with scala 2.12 is broken also so I back port this PR to 2.8 This is an automated message from

[GitHub] [kafka] ableegoldman commented on pull request #10187: MINOR: document restriction against running multiple Streams apps on same state.dir

2021-02-22 Thread GitBox
ableegoldman commented on pull request #10187: URL: https://github.com/apache/kafka/pull/10187#issuecomment-783862480 Call for review any of @cadonna @lct45 @wcarlson5 @rodesai @agavra This is an automated message from the

[GitHub] [kafka] ableegoldman commented on pull request #10187: MINOR: document restriction against running multiple Streams apps on same state.dir

2021-02-22 Thread GitBox
ableegoldman commented on pull request #10187: URL: https://github.com/apache/kafka/pull/10187#issuecomment-783862209 This should be cherrypicked back to the 2.8 branch (it's just docs) -- cc/ 2.8 release manager @vvcephei

[GitHub] [kafka] ableegoldman opened a new pull request #10187: MINOR: document restriction against running multiple Streams apps on same state.dir

2021-02-22 Thread GitBox
ableegoldman opened a new pull request #10187: URL: https://github.com/apache/kafka/pull/10187 Running more than one Streams instance on the same physical state directory has never been supported, but until now it's also not really been enforced. Since we fixed Streams to fail fast in the

[GitHub] [kafka] chia7712 commented on a change in pull request #10137: KAFKA-12268: Implement task idling semantics via currentLag API

2021-02-22 Thread GitBox
chia7712 commented on a change in pull request #10137: URL: https://github.com/apache/kafka/pull/10137#discussion_r580741128 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java ## @@ -2219,6 +2221,25 @@ public void resume(Collection

[GitHub] [kafka] mjsax closed pull request #9690: KAFKA-10017: fix flaky EOS-beta upgrade test

2021-02-22 Thread GitBox
mjsax closed pull request #9690: URL: https://github.com/apache/kafka/pull/9690 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mjsax commented on pull request #9690: KAFKA-10017: fix flaky EOS-beta upgrade test

2021-02-22 Thread GitBox
mjsax commented on pull request #9690: URL: https://github.com/apache/kafka/pull/9690#issuecomment-783848142 As we have a `2.8` release branch now, it seems not to be worth any longer to backport to fix to `2.6` branch. Closing this PR.

[GitHub] [kafka] showuon commented on pull request #10166: MINOR: update the memberMetadata output

2021-02-22 Thread GitBox
showuon commented on pull request #10166: URL: https://github.com/apache/kafka/pull/10166#issuecomment-783847072 @chia7712 , could you have a look at this one line change? Thanks. This is an automated message from the Apache

[jira] [Resolved] (KAFKA-12273) InterBrokerSendThread#pollOnce throws FatalExitError even though it is shutdown correctly

2021-02-22 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12273. Resolution: Fixed > InterBrokerSendThread#pollOnce throws FatalExitError even though it

[GitHub] [kafka] chia7712 commented on pull request #10024: KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even…

2021-02-22 Thread GitBox
chia7712 commented on pull request #10024: URL: https://github.com/apache/kafka/pull/10024#issuecomment-783842651 push to trunk and 2.8 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] chia7712 merged pull request #10024: KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even…

2021-02-22 Thread GitBox
chia7712 merged pull request #10024: URL: https://github.com/apache/kafka/pull/10024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mjsax opened a new pull request #10186: MINOR: bump release version to 3.0.0-SNAPSHOT

2021-02-22 Thread GitBox
mjsax opened a new pull request #10186: URL: https://github.com/apache/kafka/pull/10186 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[jira] [Resolved] (KAFKA-10665) Flaky Test StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumberOfPartitions[Optimization = all]

2021-02-22 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-10665. --- Resolution: Fixed > Flaky Test >

[GitHub] [kafka] showuon commented on pull request #10185: KAFKA-12284: wait for mm2 auto-created the topic

2021-02-22 Thread GitBox
showuon commented on pull request #10185: URL: https://github.com/apache/kafka/pull/10185#issuecomment-783839489 @mimaison , could you help review this PR? Thank you. This is an automated message from the Apache Git Service.

[GitHub] [kafka] showuon opened a new pull request #10185: KAFKA-12284: wait for mm2 auto-created the topic

2021-02-22 Thread GitBox
showuon opened a new pull request #10185: URL: https://github.com/apache/kafka/pull/10185 The reason why the test sometimes failed with: `TopicExistsException: Topic 'primary.test-topic-2' already exists.` is because we tried to create the topic that the MM2 already help us created. That

[GitHub] [kafka] mjsax commented on a change in pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

2021-02-22 Thread GitBox
mjsax commented on a change in pull request #10180: URL: https://github.com/apache/kafka/pull/10180#discussion_r580731514 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -91,6 +91,7 @@ // includes assigned &

[GitHub] [kafka] mjsax commented on a change in pull request #10180: KAFKA- 12347: expose offsets to streams client (WIP)

2021-02-22 Thread GitBox
mjsax commented on a change in pull request #10180: URL: https://github.com/apache/kafka/pull/10180#discussion_r580731029 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -307,6 +308,9 @@ public boolean isRunning() {

[GitHub] [kafka] dengziming commented on a change in pull request #10183: KAFKA-12267; Implement `DescribeTransactions` API

2021-02-22 Thread GitBox
dengziming commented on a change in pull request #10183: URL: https://github.com/apache/kafka/pull/10183#discussion_r580710886 ## File path: clients/src/main/resources/common/message/DescribeTransactionsResponse.json ## @@ -0,0 +1,42 @@ +// Licensed to the Apache Software

[GitHub] [kafka] showuon commented on pull request #9627: KAFKA-10746: Change to Warn logs when necessary to notify users

2021-02-22 Thread GitBox
showuon commented on pull request #9627: URL: https://github.com/apache/kafka/pull/9627#issuecomment-783805863 @abbccdda @guozhangwang @mjsax , please help review this PR. Thanks. This is an automated message from the

[GitHub] [kafka] cmccabe opened a new pull request #10184: MINOR: enable topic deletion in the KIP-500 controller

2021-02-22 Thread GitBox
cmccabe opened a new pull request #10184: URL: https://github.com/apache/kafka/pull/10184 Enable the new KIP-500 controller to delete topics. Fix a bug where feature level records were not correctly replayed. Fix a bug in TimelineHashMap#remove where the wrong type was being

[GitHub] [kafka] hachikuji commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

2021-02-22 Thread GitBox
hachikuji commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r580705481 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -92,4 +93,13 @@ public static Path createTempFile(Path logDir,

[jira] [Resolved] (KAFKA-12160) KafkaStreams configs are documented incorrectly

2021-02-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12160. - Fix Version/s: 2.8.0 Resolution: Fixed > KafkaStreams configs are documented

[GitHub] [kafka] mjsax commented on pull request #10182: KAFKA-12160: Remove max.poll.interval.ms from config docs (and KSTREAMS-4881)

2021-02-22 Thread GitBox
mjsax commented on pull request #10182: URL: https://github.com/apache/kafka/pull/10182#issuecomment-783792561 Merged to `trunk` and cherry-picked to `2.8` branch. This is an automated message from the Apache Git Service. To

[GitHub] [kafka] mjsax merged pull request #10182: KAFKA-12160: Remove max.poll.interval.ms from config docs (and KSTREAMS-4881)

2021-02-22 Thread GitBox
mjsax merged pull request #10182: URL: https://github.com/apache/kafka/pull/10182 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] hachikuji opened a new pull request #10183: KAFKA-12267; Implement `DescribeTransactions` API

2021-02-22 Thread GitBox
hachikuji opened a new pull request #10183: URL: https://github.com/apache/kafka/pull/10183 This patch implements the `DescribeTransactions` API as documented in KIP-664: https://cwiki.apache.org/confluence/display/KAFKA/KIP-664%3A+Provide+tooling+to+detect+and+abort+hanging+transactions.

[GitHub] [kafka] JimGalasyn opened a new pull request #10182: KAFKA-12160: Remove max.poll.interval.ms from config docs

2021-02-22 Thread GitBox
JimGalasyn opened a new pull request #10182: URL: https://github.com/apache/kafka/pull/10182 [KAFKA-12160](https://issues.apache.org/jira/browse/KAFKA-12160): Remove `max.poll.interval.ms` from the Kafka Streams docs. This

[GitHub] [kafka] hachikuji commented on a change in pull request #9958: MINOR: Clean up group instance id handling in `GroupCoordinator`

2021-02-22 Thread GitBox
hachikuji commented on a change in pull request #9958: URL: https://github.com/apache/kafka/pull/9958#discussion_r580635693 ## File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala ## @@ -725,50 +890,67 @@ class GroupCoordinator(val brokerId: Int,

[jira] [Updated] (KAFKA-12363) Simplify static group memberId update logic

2021-02-22 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12363: Description: In KAFKA-10284, we amended the JoinGroup logic to ensure that the memberId

[GitHub] [kafka] hachikuji commented on a change in pull request #9958: MINOR: Clean up group instance id handling in `GroupCoordinator`

2021-02-22 Thread GitBox
hachikuji commented on a change in pull request #9958: URL: https://github.com/apache/kafka/pull/9958#discussion_r580672171 ## File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala ## @@ -1055,11 +1227,15 @@ class GroupCoordinator(val brokerId: Int,

[jira] [Created] (KAFKA-12363) Simplify static group memberId update logic

2021-02-22 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12363: --- Summary: Simplify static group memberId update logic Key: KAFKA-12363 URL: https://issues.apache.org/jira/browse/KAFKA-12363 Project: Kafka Issue

[GitHub] [kafka] cmccabe merged pull request #10181: MINOR: fix a couple of failing system tests

2021-02-22 Thread GitBox
cmccabe merged pull request #10181: URL: https://github.com/apache/kafka/pull/10181 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] rondagostino commented on pull request #10181: MINOR: fix a couple of failing system tests

2021-02-22 Thread GitBox
rondagostino commented on pull request #10181: URL: https://github.com/apache/kafka/pull/10181#issuecomment-783734434 @cmccabe Can you take a look? I found these issues after the test run on the 2.8 PR branch completed.

[GitHub] [kafka] rondagostino opened a new pull request #10181: MINOR: fix a couple of failing system tests

2021-02-22 Thread GitBox
rondagostino opened a new pull request #10181: URL: https://github.com/apache/kafka/pull/10181 This patch fixes a couple of failing system tests due to https://github.com/apache/kafka/pull/10105/ and should be merged to both `trunk` and `2.8` ### Committer Checklist (excluded from

[jira] [Created] (KAFKA-12362) Determine if a Task is idling

2021-02-22 Thread Walker Carlson (Jira)
Walker Carlson created KAFKA-12362: -- Summary: Determine if a Task is idling Key: KAFKA-12362 URL: https://issues.apache.org/jira/browse/KAFKA-12362 Project: Kafka Issue Type: Improvement

[GitHub] [kafka] wcarlson5 opened a new pull request #10180: KAFKA- 12347: expose offsets to streams client

2021-02-22 Thread GitBox
wcarlson5 opened a new pull request #10180: URL: https://github.com/apache/kafka/pull/10180 collect the offsets after they are committed ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status

[GitHub] [kafka] ijuma edited a comment on pull request #10179: MINOR: tune KIP-631 configurations

2021-02-22 Thread GitBox
ijuma edited a comment on pull request #10179: URL: https://github.com/apache/kafka/pull/10179#issuecomment-783724833 Can we add a comment to the code explaining why these timeouts are lower than the ZK equivalent ones (similar to what's in the PR description). May even be worth including

[GitHub] [kafka] ijuma commented on pull request #10179: MINOR: tune KIP-631 configurations

2021-02-22 Thread GitBox
ijuma commented on pull request #10179: URL: https://github.com/apache/kafka/pull/10179#issuecomment-783724833 Can we add a comment to the code explaining why these timeouts are lower than the ZK equivalent ones (similar to what's in the PR description).

[GitHub] [kafka] cmccabe opened a new pull request #10179: MINOR: tune KIP-631 configurations

2021-02-22 Thread GitBox
cmccabe opened a new pull request #10179: URL: https://github.com/apache/kafka/pull/10179 Since we expect KIP-631 controller failovers to be fairly cheap, tune the default raft configuration parameters so that we detect node failures more quickly. Reduce the broker session

[GitHub] [kafka] hachikuji commented on a change in pull request #9958: MINOR: Clean up group instance id handling in `GroupCoordinator`

2021-02-22 Thread GitBox
hachikuji commented on a change in pull request #9958: URL: https://github.com/apache/kafka/pull/9958#discussion_r580635693 ## File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala ## @@ -725,50 +890,67 @@ class GroupCoordinator(val brokerId: Int,

[GitHub] [kafka] cmccabe closed pull request #10175: MINOR: V2.8 system tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe closed pull request #10175: URL: https://github.com/apache/kafka/pull/10175 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] cmccabe commented on pull request #10175: MINOR: V2.8 system tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on pull request #10175: URL: https://github.com/apache/kafka/pull/10175#issuecomment-783709474 thanks, @rondagostino! I backported this as part of the #10105 PR as we usually do (we don't usually have a separate PR for the branch). Thanks for showing me how to resolve

[GitHub] [kafka] cmccabe merged pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe merged pull request #10105: URL: https://github.com/apache/kafka/pull/10105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-12242) Decouple state store materialization enforcement from name/serde provider

2021-02-22 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288646#comment-17288646 ] John Roesler commented on KAFKA-12242: -- Thanks for raising this, [~guozhang] , I have seen several

[GitHub] [kafka] hachikuji commented on a change in pull request #9912: MINOR: Let TestRaftRequestHandler use RequestHandlerHelper

2021-02-22 Thread GitBox
hachikuji commented on a change in pull request #9912: URL: https://github.com/apache/kafka/pull/9912#discussion_r580606013 ## File path: core/src/test/scala/unit/kafka/server/KafkaApisTest.scala ## @@ -2127,21 +2118,15 @@ class KafkaApisTest { error = Errors.NONE

[jira] [Updated] (KAFKA-12347) Improve Kafka Streams ability to track progress

2021-02-22 Thread Walker Carlson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walker Carlson updated KAFKA-12347: --- Description: Add methods to track records being consumed fully and to tell if tasks are

[GitHub] [kafka] rhauch commented on a change in pull request #9950: KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string

2021-02-22 Thread GitBox
rhauch commented on a change in pull request #9950: URL: https://github.com/apache/kafka/pull/9950#discussion_r580584221 ## File path: connect/transforms/src/main/java/org/apache/kafka/connect/transforms/Cast.java ## @@ -364,11 +365,24 @@ private static String

[GitHub] [kafka] C0urante commented on pull request #10178: KAFKA-12361: Use default request.timeout.ms value for Connect producers

2021-02-22 Thread GitBox
C0urante commented on pull request #10178: URL: https://github.com/apache/kafka/pull/10178#issuecomment-783669441 @hachikuji Does this look alright to you? The changes are trivial but I'm also hoping my summary of the situation in the description is correct and won't mislead anyone.

[GitHub] [kafka] C0urante opened a new pull request #10178: KAFKA-12361: Use default request.timeout.ms value for Connect producers

2021-02-22 Thread GitBox
C0urante opened a new pull request #10178: URL: https://github.com/apache/kafka/pull/10178 [Jira](https://issues.apache.org/jira/browse/KAFKA-12361) The super-high request timeout makes it harder for the producer to gracefully handle unclean connection terminations, which might

[GitHub] [kafka] rhauch edited a comment on pull request #9950: KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string

2021-02-22 Thread GitBox
rhauch edited a comment on pull request #9950: URL: https://github.com/apache/kafka/pull/9950#issuecomment-783634110 Before this fix, the changes made in #4820 ([KAFKA-6684](https://issues.apache.org/jira/browse/KAFKA-6684)) resulted in `toString()` being called on `byte[]` and

[GitHub] [kafka] rhauch edited a comment on pull request #9950: KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string

2021-02-22 Thread GitBox
rhauch edited a comment on pull request #9950: URL: https://github.com/apache/kafka/pull/9950#issuecomment-783634110 Before this fix, the changes made in #4820 ([KAFKA-6684](https://issues.apache.org/jira/browse/KAFKA-6684)) resulted in `toString()` being called on `byte[]` and

[jira] [Assigned] (KAFKA-12361) Change default connect producer request timeout

2021-02-22 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton reassigned KAFKA-12361: - Assignee: Chris Egerton > Change default connect producer request timeout >

[GitHub] [kafka] rondagostino commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
rondagostino commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580575104 ## File path: tests/kafkatest/services/performance/end_to_end_latency.py ## @@ -74,12 +74,15 @@ def __init__(self, context, num_nodes, kafka, topic,

[GitHub] [kafka] cmccabe commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580571155 ## File path: tests/kafkatest/tests/core/round_trip_fault_test.py ## @@ -47,32 +55,48 @@ def __init__(self, test_context):

[GitHub] [kafka] cmccabe commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580570954 ## File path: tests/kafkatest/services/performance/end_to_end_latency.py ## @@ -74,12 +74,15 @@ def __init__(self, context, num_nodes, kafka, topic,

[GitHub] [kafka] rhauch commented on pull request #9950: KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string

2021-02-22 Thread GitBox
rhauch commented on pull request #9950: URL: https://github.com/apache/kafka/pull/9950#issuecomment-783634110 Before this fix, the changes made in #4820 ([KAFKA-6684](https://issues.apache.org/jira/browse/KAFKA-6684)) resulted in `toString()` being called on `byte[]` and `ByteBuffer`. As

[GitHub] [kafka] mjsax commented on a change in pull request #10170: KAFKA-12323: Set timestamp in record context when punctuate

2021-02-22 Thread GitBox
mjsax commented on a change in pull request #10170: URL: https://github.com/apache/kafka/pull/10170#discussion_r580546901 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1874,6 +1873,66 @@ public void close() {}

[GitHub] [kafka] mjsax commented on a change in pull request #10170: KAFKA-12323: Set timestamp in record context when punctuate

2021-02-22 Thread GitBox
mjsax commented on a change in pull request #10170: URL: https://github.com/apache/kafka/pull/10170#discussion_r580546373 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1874,6 +1873,66 @@ public void close() {}

[GitHub] [kafka] mjsax commented on a change in pull request #10170: KAFKA-12323: Set timestamp in record context when punctuate

2021-02-22 Thread GitBox
mjsax commented on a change in pull request #10170: URL: https://github.com/apache/kafka/pull/10170#discussion_r580544814 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -792,7 +790,16 @@ public void punctuate(final

[jira] [Assigned] (KAFKA-12323) Record timestamps not populated in event

2021-02-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-12323: --- Assignee: Guozhang Wang > Record timestamps not populated in event >

[jira] [Updated] (KAFKA-12360) Improve documentation of max.task.idle.ms (kafka-streams)

2021-02-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12360: Labels: beginner newbie trivial (was: kafka-streams) > Improve documentation of

[GitHub] [kafka] rondagostino commented on pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
rondagostino commented on pull request #10105: URL: https://github.com/apache/kafka/pull/10105#issuecomment-783624846 > I wonder if we should disallow tests without `[the @cluster]` annotation in the future. I think perhaps yes. Is there a need to allow it? A simple oversight

[GitHub] [kafka] rondagostino commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
rondagostino commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580532054 ## File path: tests/kafkatest/tests/core/round_trip_fault_test.py ## @@ -47,32 +55,48 @@ def __init__(self, test_context):

[GitHub] [kafka] rondagostino commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
rondagostino commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580532054 ## File path: tests/kafkatest/tests/core/round_trip_fault_test.py ## @@ -47,32 +55,48 @@ def __init__(self, test_context):

[GitHub] [kafka] rondagostino commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
rondagostino commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580527682 ## File path: tests/kafkatest/services/performance/end_to_end_latency.py ## @@ -74,12 +74,15 @@ def __init__(self, context, num_nodes, kafka, topic,

[jira] [Updated] (KAFKA-12361) Change default connect producer request timeout

2021-02-22 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12361: Description: Prior to KIP-91, which introduced delivery.timeout.ms, there was no easy way

[jira] [Created] (KAFKA-12361) Change default connect producer request timeout

2021-02-22 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12361: --- Summary: Change default connect producer request timeout Key: KAFKA-12361 URL: https://issues.apache.org/jira/browse/KAFKA-12361 Project: Kafka Issue

[GitHub] [kafka] cmccabe commented on pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on pull request #10105: URL: https://github.com/apache/kafka/pull/10105#issuecomment-783579134 Good work spotting the missing `@cluster` annotations! That is quite a performance win. I wonder if we should disallow tests without this annotation in the future.

[GitHub] [kafka] cmccabe commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580489004 ## File path: tests/kafkatest/tests/core/round_trip_fault_test.py ## @@ -47,32 +55,48 @@ def __init__(self, test_context):

[GitHub] [kafka] cmccabe commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580486218 ## File path: tests/kafkatest/tests/client/client_compatibility_produce_consume_test.py ## @@ -46,13 +47,16 @@ def __init__(self, test_context):

[GitHub] [kafka] cmccabe commented on a change in pull request #10105: MINOR: System tests for Raft-based metadata quorums

2021-02-22 Thread GitBox
cmccabe commented on a change in pull request #10105: URL: https://github.com/apache/kafka/pull/10105#discussion_r580485952 ## File path: tests/kafkatest/tests/client/client_compatibility_features_test.py ## @@ -120,10 +123,9 @@ def invoke_compatibility_program(self,

  1   2   >