[GitHub] [kafka] chia7712 merged pull request #10868: MINOR: make sure alterAclsPurgatory is closed when controller server …

2021-07-16 Thread GitBox
chia7712 merged pull request #10868: URL: https://github.com/apache/kafka/pull/10868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] satishd commented on pull request #11060: MINOR Refactored the existing CheckpointFile in core module, moved to server-common module and introduced it as SnapshotFile.

2021-07-16 Thread GitBox
satishd commented on pull request #11060: URL: https://github.com/apache/kafka/pull/11060#issuecomment-881843364 Thanks @junrao for the comment, replied inline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [kafka] satishd commented on a change in pull request #11060: MINOR Refactored the existing CheckpointFile in core module, moved to server-common module and introduced it as SnapshotFile.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #11060: URL: https://github.com/apache/kafka/pull/11060#discussion_r671627863 ## File path: core/src/main/scala/kafka/server/checkpoints/CheckpointFile.scala ## @@ -16,127 +16,41 @@ */ package kafka.server.checkpoints -import

[GitHub] [kafka] satishd commented on a change in pull request #11060: MINOR Refactored the existing CheckpointFile in core module, moved to server-common module and introduced it as SnapshotFile.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #11060: URL: https://github.com/apache/kafka/pull/11060#discussion_r671627863 ## File path: core/src/main/scala/kafka/server/checkpoints/CheckpointFile.scala ## @@ -16,127 +16,41 @@ */ package kafka.server.checkpoints -import

[GitHub] [kafka] dengziming commented on pull request #10753: KAFKA-12803: Support reassigning partitions when in KRaft mode

2021-07-16 Thread GitBox
dengziming commented on pull request #10753: URL: https://github.com/apache/kafka/pull/10753#issuecomment-881836203 @jacky1193610322 , The metadata will write to KafkaRaftClient, and the raft listener will handle this. see `QuorumMetaLogListener`. -- This is an automated message from the

[GitHub] [kafka] satishd edited a comment on pull request #11033: KAFKA-12988 Asynchronous API support for RemoteLogMetadataManager add/update methods.

2021-07-16 Thread GitBox
satishd edited a comment on pull request #11033: URL: https://github.com/apache/kafka/pull/11033#issuecomment-878780060 This change is built on top of https://github.com/apache/kafka/pull/10579. The latest commit https://github.com/apache/kafka/commit/9831fa9a4f65ed50b1f66c1c71018edfe48ea8

[GitHub] [kafka] chia7712 commented on pull request #10868: MINOR: make sure alterAclsPurgatory is closed when controller server …

2021-07-16 Thread GitBox
chia7712 commented on pull request #10868: URL: https://github.com/apache/kafka/pull/10868#issuecomment-881828511 > We should cherry-pick this to 3.0 please. will copy that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [kafka] ijuma commented on pull request #10868: MINOR: make sure alterAclsPurgatory is closed when controller server …

2021-07-16 Thread GitBox
ijuma commented on pull request #10868: URL: https://github.com/apache/kafka/pull/10868#issuecomment-881828358 We should cherry-pick this to 3.0 please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671609168 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -774,10 +774,10 @@ class KafkaApis(val requestChannel: RequestChannel, // whic

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671609095 ## File path: core/src/main/scala/kafka/log/LogConfig.scala ## @@ -479,4 +501,31 @@ object LogConfig { logProps.put(MessageDownConversionEnableProp, k

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671608959 ## File path: core/src/main/scala/kafka/log/LogConfig.scala ## @@ -479,4 +501,31 @@ object LogConfig { logProps.put(MessageDownConversionEnableProp, k

[GitHub] [kafka] chia7712 commented on pull request #10868: MINOR: make sure alterAclsPurgatory is closed when controller server …

2021-07-16 Thread GitBox
chia7712 commented on pull request #10868: URL: https://github.com/apache/kafka/pull/10868#issuecomment-881814393 `ConsumerBounceTest.testCloseDuringRebalance` is flaky on trunk so it is unrelated to this PR. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] satishd commented on pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on pull request #10579: URL: https://github.com/apache/kafka/pull/10579#issuecomment-881805037 Thanks @junrao for the comment. I addressed it with the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] showuon opened a new pull request #11072: [WIP] MINOR: stream stuck fix test

2021-07-16 Thread GitBox
showuon opened a new pull request #11072: URL: https://github.com/apache/kafka/pull/11072 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] tang7526 commented on a change in pull request #11062: KAFKA-13094: Session windows do not consider user-specified grace when computing retention time for changelog

2021-07-16 Thread GitBox
tang7526 commented on a change in pull request #11062: URL: https://github.com/apache/kafka/pull/11062#discussion_r671589951 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/SessionWindows.java ## @@ -187,7 +187,8 @@ public long inactivityGap() { */

[GitHub] [kafka] showuon commented on pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-16 Thread GitBox
showuon commented on pull request #11057: URL: https://github.com/apache/kafka/pull/11057#issuecomment-881784596 Also, @guozhangwang , if you just run this test in KAFKA-9295, it's really difficult to reproduce it. It needs some tweak for it. Anyway, I will test it and let you know. :) -

[GitHub] [kafka] showuon commented on pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-16 Thread GitBox
showuon commented on pull request #11057: URL: https://github.com/apache/kafka/pull/11057#issuecomment-881783736 Let me test it, and let you know. BTW, I like this simple client side fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] cmccabe merged pull request #10991: MINOR: system test fix for 3 co-located KRaft controllers

2021-07-16 Thread GitBox
cmccabe merged pull request #10991: URL: https://github.com/apache/kafka/pull/10991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] cmccabe commented on pull request #10991: MINOR: system test fix for 3 co-located KRaft controllers

2021-07-16 Thread GitBox
cmccabe commented on pull request #10991: URL: https://github.com/apache/kafka/pull/10991#issuecomment-881768683 Thanks, @rondagostino ! I like the concurrent-start solution a lot better than any of the previous ones we were discussing. LGTM. -- This is an automated message from the Apac

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671566299 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -774,10 +774,10 @@ class KafkaApis(val requestChannel: RequestChannel, // whic

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671566299 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -774,10 +774,10 @@ class KafkaApis(val requestChannel: RequestChannel, // whic

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671565901 ## File path: core/src/main/scala/kafka/admin/ConfigCommand.scala ## @@ -270,7 +271,8 @@ object ConfigCommand extends Config { } if (props.contain

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671565792 ## File path: clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java ## @@ -171,12 +171,24 @@ public static final String PREALLOCATE_DOC

[GitHub] [kafka] jolshan commented on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
jolshan commented on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881735255 Since I've confirmed that we don't use partition.metadata files for the metadata topic in 2.8, I've removed this version from the ticket. Thanks @jsancio for this patch. -- T

[jira] [Updated] (KAFKA-13098) No such file exception when recovering snapshots in metadata log dir

2021-07-16 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-13098: --- Affects Version/s: (was: 2.8.1) > No such file exception when recovering snapshots in me

[GitHub] [kafka] mjsax commented on pull request #11055: HOTFIX: Init stream-stream left/outer join emit interval correctly

2021-07-16 Thread GitBox
mjsax commented on pull request #11055: URL: https://github.com/apache/kafka/pull/11055#issuecomment-881729479 Merged to `trunk` and cherry-picked to `3.0` branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[jira] [Assigned] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-07-16 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe reassigned KAFKA-12777: Assignee: David Arthur > AutoTopicCreationManager does not handle response errors > -

[jira] [Resolved] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-07-16 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12777. -- Resolution: Fixed Committed #11069, thanks! > AutoTopicCreationManager does not handle respon

[GitHub] [kafka] cmccabe commented on pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-16 Thread GitBox
cmccabe commented on pull request #11054: URL: https://github.com/apache/kafka/pull/11054#issuecomment-881724553 Committed and cherry-picked to 3.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] cmccabe closed pull request #11054: KAFKA-13090: Improve kraft snapshot integration test

2021-07-16 Thread GitBox
cmccabe closed pull request #11054: URL: https://github.com/apache/kafka/pull/11054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] cmccabe commented on pull request #11066: MINOR: Test ReplicaManager Metric Names

2021-07-16 Thread GitBox
cmccabe commented on pull request #11066: URL: https://github.com/apache/kafka/pull/11066#issuecomment-881711772 > Maybe rename ReplicaManagerMetricNamesTest to ClusterMetricNamesTest so that it isn't specific to ReplicaManager? How about `BrokerMetricNamesTest`? -- This is an aut

[GitHub] [kafka] junrao commented on a change in pull request #11060: MINOR Refactored the existing CheckpointFile in core module, moved to server-common module and introduced it as SnapshotFile.

2021-07-16 Thread GitBox
junrao commented on a change in pull request #11060: URL: https://github.com/apache/kafka/pull/11060#discussion_r671517216 ## File path: core/src/main/scala/kafka/server/checkpoints/CheckpointFile.scala ## @@ -16,127 +16,41 @@ */ package kafka.server.checkpoints -import j

[GitHub] [kafka] junrao commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
junrao commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671516460 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerConfig.java ## @@ -0,0 +1,197 @@

[GitHub] [kafka] mjsax merged pull request #11055: HOTFIX: Init stream-stream left/outer join emit interval correctly

2021-07-16 Thread GitBox
mjsax merged pull request #11055: URL: https://github.com/apache/kafka/pull/11055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] guozhangwang commented on pull request #11057: KAFKA-13008: Try to refresh end offset when partitionLag returns empty

2021-07-16 Thread GitBox
guozhangwang commented on pull request #11057: URL: https://github.com/apache/kafka/pull/11057#issuecomment-881696896 Thanks @vvcephei , at the mean time I will try to just run https://issues.apache.org/jira/browse/KAFKA-9295 and see if it indeed help reducing the flakiness. Also cc @showu

[GitHub] [kafka] cmccabe merged pull request #11069: KAFKA-12777 auto topic creation manager npe

2021-07-16 Thread GitBox
cmccabe merged pull request #11069: URL: https://github.com/apache/kafka/pull/11069 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] jsancio edited a comment on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
jsancio edited a comment on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881693229 > I cherry-picked this to 3.0, but the commit does not apply to 2.8. Since we never created snapshot files in 2.8, are you sure it needs to be present there? @cmcc

[GitHub] [kafka] jsancio commented on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
jsancio commented on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881693229 > I cherry-picked this to 3.0, but the commit does not apply to 2.8. Since we never created snapshot files in 2.8, are you sure it needs to be present there? @cmccabe My

[GitHub] [kafka] cmccabe commented on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
cmccabe commented on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881686453 Possibly related: https://github.com/apache/kafka/pull/11056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] cmccabe commented on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
cmccabe commented on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881685696 I cherry-picked this to 3.0, but the commit does not apply to 2.8. Since we never created snapshot files in 2.8, are you sure it needs to be present there? -- This is an autom

[GitHub] [kafka] cmccabe merged pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
cmccabe merged pull request #11071: URL: https://github.com/apache/kafka/pull/11071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] jsancio commented on a change in pull request #11053: KAFKA-13015 Ducktape System Tests for Metadata Snapshots

2021-07-16 Thread GitBox
jsancio commented on a change in pull request #11053: URL: https://github.com/apache/kafka/pull/11053#discussion_r671454434 ## File path: tests/kafkatest/tests/core/snapshot_test.py ## @@ -0,0 +1,227 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# co

[GitHub] [kafka] niket-goel commented on pull request #11070: Validate the controllerListener config on startup

2021-07-16 Thread GitBox
niket-goel commented on pull request #11070: URL: https://github.com/apache/kafka/pull/11070#issuecomment-881665920 Looks like a lot of tests are failing with the exception I added. Will take a look. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Updated] (KAFKA-13098) No such file exception when recovering snapshots in metadata log dir

2021-07-16 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-13098: --- Affects Version/s: 2.8.1 3.0.0 > No such file

[GitHub] [kafka] jsancio commented on pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
jsancio commented on pull request #11071: URL: https://github.com/apache/kafka/pull/11071#issuecomment-881661296 This commit needs to get cherry picked to 2.8 and 3.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [kafka] guozhangwang commented on a change in pull request #11061: MINOR: Add test for verifying retention on changelog topics

2021-07-16 Thread GitBox
guozhangwang commented on a change in pull request #11061: URL: https://github.com/apache/kafka/pull/11061#discussion_r671450964 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/WindowedChangelogRetentionIntegrationTest.java ## @@ -0,0 +1,293 @@ +/* + *

[GitHub] [kafka] guozhangwang commented on pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-16 Thread GitBox
guozhangwang commented on pull request #10953: URL: https://github.com/apache/kafka/pull/10953#issuecomment-881635091 Cherry-picked to 3.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [kafka] C0urante commented on pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on pull request #10563: URL: https://github.com/apache/kafka/pull/10563#issuecomment-881634433 Thanks for taking a look @kkonstantine, this is ready for another round. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] guozhangwang merged pull request #10953: MINOR: Default GRACE with Old API should set as 24H minus window-size / inactivity-gap

2021-07-16 Thread GitBox
guozhangwang merged pull request #10953: URL: https://github.com/apache/kafka/pull/10953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsu

[jira] [Created] (KAFKA-13098) No such file exception when recovering snapshots in metadata log dir

2021-07-16 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-13098: -- Summary: No such file exception when recovering snapshots in metadata log dir Key: KAFKA-13098 URL: https://issues.apache.org/jira/browse/KAFKA-13098

[jira] [Created] (KAFKA-13099) Message too large error when expiring transactionalIds

2021-07-16 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-13099: --- Summary: Message too large error when expiring transactionalIds Key: KAFKA-13099 URL: https://issues.apache.org/jira/browse/KAFKA-13099 Project: Kafka

[GitHub] [kafka] jsancio opened a new pull request #11071: KAFKA-13098: Fix NoSuchFileException during snapshot recovery

2021-07-16 Thread GitBox
jsancio opened a new pull request #11071: URL: https://github.com/apache/kafka/pull/11071 Java's FileTreeIterator throws an NoSuchFileException when visting @metadata-0/partition.metadata.tmp. This is most like do to the fact that the Log type asynchronously creates and delete that fil

[GitHub] [kafka] niket-goel opened a new pull request #11070: Validate the controllerListener config on startup

2021-07-16 Thread GitBox
niket-goel opened a new pull request #11070: URL: https://github.com/apache/kafka/pull/11070 Also generate a better error if failing to startup controller due to an empty controllerListener config *More detailed description of your change, if necessary. The PR title and PR message

[GitHub] [kafka] guozhangwang commented on a change in pull request #11061: MINOR: Add test for verifying retention on changelog topics

2021-07-16 Thread GitBox
guozhangwang commented on a change in pull request #11061: URL: https://github.com/apache/kafka/pull/11061#discussion_r671431083 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/WindowedChangelogRetentionIntegrationTest.java ## @@ -0,0 +1,293 @@ +/* + *

[GitHub] [kafka] satishd commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671415369 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java ## @@ -0,0 +1,484 @@ +/*

[jira] [Created] (KAFKA-13097) Handle the requests gracefully to publish the events in TopicBasedRemoteLogMetadataManager when it is not yet initialized.

2021-07-16 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-13097: -- Summary: Handle the requests gracefully to publish the events in TopicBasedRemoteLogMetadataManager when it is not yet initialized. Key: KAFKA-13097 URL: https://issues.apache

[GitHub] [kafka] guozhangwang commented on a change in pull request #11062: KAFKA-13094: Session windows do not consider user-specified grace when computing retention time for changelog

2021-07-16 Thread GitBox
guozhangwang commented on a change in pull request #11062: URL: https://github.com/apache/kafka/pull/11062#discussion_r671425031 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/SessionWindows.java ## @@ -187,7 +187,8 @@ public long inactivityGap() {

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671422842 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -710,22 +758,35 @@ else if (!context.pausedPartit

[GitHub] [kafka] hachikuji commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
hachikuji commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671401329 ## File path: clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java ## @@ -171,12 +171,24 @@ public static final String PREALLOCATE

[GitHub] [kafka] satishd commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671419080 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerConfig.java ## @@ -0,0 +1,197 @

[GitHub] [kafka] mumrah opened a new pull request #11069: KAFKA-12777 auto topic creation manager npe

2021-07-16 Thread GitBox
mumrah opened a new pull request #11069: URL: https://github.com/apache/kafka/pull/11069 Split off from #10696, but only contains the bug fix. There are some cases where a response will include a `null` body, so we need to check for those cases before printing this log statement. -

[GitHub] [kafka] satishd commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671415369 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java ## @@ -0,0 +1,484 @@ +/*

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671409966 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -670,8 +707,7 @@ long getNextCommit() {

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671409811 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -367,42 +369,53 @@ private void doCommit(Map offs

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671407793 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -367,42 +369,53 @@ private void doCommit(Map offs

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671406780 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/errors/WorkerErrantRecordReporter.java ## @@ -49,7 +54,7 @@ private fi

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671405631 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -367,42 +369,53 @@ private void doCommit(Map offs

[GitHub] [kafka] C0urante commented on a change in pull request #10563: KAFKA-12487: Add support for cooperative consumer protocol with sink connectors

2021-07-16 Thread GitBox
C0urante commented on a change in pull request #10563: URL: https://github.com/apache/kafka/pull/10563#discussion_r671404000 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java ## @@ -412,32 +425,36 @@ private void commitOffsets(lon

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671389668 ## File path: core/src/main/scala/kafka/log/LogManager.scala ## @@ -403,27 +411,49 @@ class LogManager(logDirs: Seq[File], * Start the background threa

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671388968 ## File path: core/src/main/scala/kafka/log/LogConfig.scala ## @@ -479,4 +501,31 @@ object LogConfig { logProps.put(MessageDownConversionEnableProp, k

[GitHub] [kafka] ijuma commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
ijuma commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671387408 ## File path: clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java ## @@ -171,12 +171,24 @@ public static final String PREALLOCATE_DOC

[GitHub] [kafka] dajac commented on a change in pull request #11036: KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724)

2021-07-16 Thread GitBox
dajac commented on a change in pull request #11036: URL: https://github.com/apache/kafka/pull/11036#discussion_r671370260 ## File path: core/src/test/scala/unit/kafka/cluster/AbstractPartitionTest.scala ## @@ -103,4 +109,38 @@ class AbstractPartitionTest { TestUtils.clea

[GitHub] [kafka] junrao commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
junrao commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671383087 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerConfig.java ## @@ -0,0 +1,197 @@

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-07-16 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Priority: Critical (was: Major) > AutoTopicCreationManager does not handle response errors > --

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-07-16 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Fix Version/s: 3.0.0 > AutoTopicCreationManager does not handle response errors > --

[GitHub] [kafka] satishd commented on pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on pull request #10579: URL: https://github.com/apache/kafka/pull/10579#issuecomment-881556895 Thanks @junrao for your latest comments, replied to them inline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] satishd commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671367565 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerConfig.java ## @@ -0,0 +1,197 @

[jira] [Assigned] (KAFKA-4064) Add support for infinite endpoints for range queries in Kafka Streams KV stores

2021-07-16 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reassigned KAFKA-4064: --- Assignee: Patrick Stuedi > Add support for infinite endpoints for range queries in Kafka Str

[GitHub] [kafka] C0urante commented on pull request #10945: KAFKA-13017: Remove excessive logging for sink task deserialization errors

2021-07-16 Thread GitBox
C0urante commented on pull request #10945: URL: https://github.com/apache/kafka/pull/10945#issuecomment-881511869 Thanks @tombentley! It'd be great to cherry-pick if possible but only if that doesn't disrupt the release process (this isn't a new regression and has been around for several p

[GitHub] [kafka] tombentley commented on pull request #10945: KAFKA-13017: Remove excessive logging for sink task deserialization errors

2021-07-16 Thread GitBox
tombentley commented on pull request #10945: URL: https://github.com/apache/kafka/pull/10945#issuecomment-881501052 @C0urante did you need this cherry picking onto 3.0? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Resolved] (KAFKA-13017) Excessive logging on sink task deserialization errors

2021-07-16 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley resolved KAFKA-13017. - Fix Version/s: 3.1.0 Resolution: Fixed > Excessive logging on sink task deserialization e

[GitHub] [kafka] tombentley merged pull request #10945: KAFKA-13017: Remove excessive logging for sink task deserialization errors

2021-07-16 Thread GitBox
tombentley merged pull request #10945: URL: https://github.com/apache/kafka/pull/10945 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubs

[GitHub] [kafka] cadonna commented on pull request #11063: Add test to verify behavior of grace API

2021-07-16 Thread GitBox
cadonna commented on pull request #11063: URL: https://github.com/apache/kafka/pull/11063#issuecomment-881468501 @mjsax Thanks for the feedback! I acted on all of them. Regarding, not using an integration test, I would love to not use one, but I could not came up with an alternative solu

[jira] [Commented] (KAFKA-12929) KIP-750: Deprecate support for Java 8 in Kafka 3.0

2021-07-16 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382065#comment-17382065 ] Ismael Juma commented on KAFKA-12929: - Readme has been updated. The only remaining i

[jira] [Commented] (KAFKA-12930) KIP-751: Deprecate support for Scala 2.12 in Kafka 3.0

2021-07-16 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382064#comment-17382064 ] Ismael Juma commented on KAFKA-12930: - Readme has been updated. The only remaining i

[GitHub] [kafka] cadonna commented on a change in pull request #11063: Add test to verify behavior of grace API

2021-07-16 Thread GitBox
cadonna commented on a change in pull request #11063: URL: https://github.com/apache/kafka/pull/11063#discussion_r671139543 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/WindowedChangelogRetentionIntegrationTest.java ## @@ -0,0 +1,365 @@ +/* + * Lice

[GitHub] [kafka] cadonna commented on a change in pull request #11063: Add test to verify behavior of grace API

2021-07-16 Thread GitBox
cadonna commented on a change in pull request #11063: URL: https://github.com/apache/kafka/pull/11063#discussion_r671139543 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/WindowedChangelogRetentionIntegrationTest.java ## @@ -0,0 +1,365 @@ +/* + * Lice

[GitHub] [kafka] satishd commented on a change in pull request #10579: KAFKA-9555 Added default RLMM implementation based on internal topic storage.

2021-07-16 Thread GitBox
satishd commented on a change in pull request #10579: URL: https://github.com/apache/kafka/pull/10579#discussion_r671094510 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java ## @@ -0,0 +1,484 @@ +/*

[GitHub] [kafka] PhilHardwick commented on a change in pull request #10921: KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread

2021-07-16 Thread GitBox
PhilHardwick commented on a change in pull request #10921: URL: https://github.com/apache/kafka/pull/10921#discussion_r671069535 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1081,6 +1079,7 @@ private int getNumStreamThreads(final boolean

[GitHub] [kafka] PhilHardwick commented on pull request #10921: KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread

2021-07-16 Thread GitBox
PhilHardwick commented on pull request #10921: URL: https://github.com/apache/kafka/pull/10921#issuecomment-881274661 Closed by accident there^ Thanks @ableegoldman for creating a JIRA ticket for this. I've updated the PR with a test for querying the stores after a thread is removed.

[GitHub] [kafka] showuon commented on a change in pull request #10987: MINOR: improve the partitioner.class doc

2021-07-16 Thread GitBox
showuon commented on a change in pull request #10987: URL: https://github.com/apache/kafka/pull/10987#discussion_r671061691 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java ## @@ -234,7 +234,24 @@ /** partitioner.class */ pu

[GitHub] [kafka] showuon commented on pull request #10987: MINOR: improve the partitioner.class doc

2021-07-16 Thread GitBox
showuon commented on pull request #10987: URL: https://github.com/apache/kafka/pull/10987#issuecomment-881269844 @jlprat , thanks for the comments. I've updated the PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] PhilHardwick closed pull request #10921: KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread

2021-07-16 Thread GitBox
PhilHardwick closed pull request #10921: URL: https://github.com/apache/kafka/pull/10921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsu

[GitHub] [kafka] showuon commented on pull request #11026: KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error

2021-07-16 Thread GitBox
showuon commented on pull request #11026: URL: https://github.com/apache/kafka/pull/11026#issuecomment-881258733 @dajac , all checked and updated. Thank you very much for your patiently review all these PRs! After these update, we are more confident in these new handlers. :) -- This is

[jira] [Resolved] (KAFKA-13033) coordinator not available error should cause add into unmap list to do a new lookup

2021-07-16 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-13033. - Reviewer: David Jacot Resolution: Fixed > coordinator not available error should cause add

[GitHub] [kafka] dajac commented on pull request #11026: KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error

2021-07-16 Thread GitBox
dajac commented on pull request #11026: URL: https://github.com/apache/kafka/pull/11026#issuecomment-881255593 @showuon Thanks for the patches. Could you update the description of this PR and the others to ensure that the description reflects the changes? -- This is an automated message

[GitHub] [kafka] dajac commented on pull request #11026: KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error

2021-07-16 Thread GitBox
dajac commented on pull request #11026: URL: https://github.com/apache/kafka/pull/11026#issuecomment-881255176 Merged to trunk and 3.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [kafka] dajac merged pull request #11026: KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error

2021-07-16 Thread GitBox
dajac merged pull request #11026: URL: https://github.com/apache/kafka/pull/11026 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] dajac commented on pull request #11026: KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error

2021-07-16 Thread GitBox
dajac commented on pull request #11026: URL: https://github.com/apache/kafka/pull/11026#issuecomment-881254117 Failures are not related: ``` Build / JDK 16 and Scala 2.13 / testCommitTransactionTimeout() – kafka.api.TransactionsTest 9s Build / JDK 11 and Scala 2.13 / testCloseDu

[GitHub] [kafka] showuon opened a new pull request #11068: KAFKA-13081: detect doubly assigned parition

2021-07-16 Thread GitBox
showuon opened a new pull request #11068: URL: https://github.com/apache/kafka/pull/11068 This is the fix 1 and fix 2 in https://github.com/apache/kafka/pull/10985 for v2.8, including the tests. ### Committer Checklist (excluded from commit message) - [ ] Verify design and impleme

  1   2   >