[GitHub] [kafka] mjsax commented on a change in pull request #11447: KAFKA-13024: Use not-null filter only in optimizable repartitions

2021-11-02 Thread GitBox
mjsax commented on a change in pull request #11447: URL: https://github.com/apache/kafka/pull/11447#discussion_r741604615 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/GroupedTableOperationRepartitionNode.java ## @@ -119,5 +119,10 @@

[GitHub] [kafka] mjsax commented on pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
mjsax commented on pull request #11368: URL: https://github.com/apache/kafka/pull/11368#issuecomment-958635275 @dajac It was originally planned for `3.1` -- did not pay attention to the feature freeze deadline. The PR is low risk, and if it's ok with you, I am happy to cherry-pick to

[jira] [Commented] (KAFKA-13425) KafkaConsumer#pause() will lose its effect after groupRebalance occurs, which maybe cause data loss on the consumer side

2021-11-02 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437705#comment-17437705 ] RivenSun commented on KAFKA-13425: -- Hi [~showuon] Because the

[GitHub] [kafka] RivenSun2 opened a new pull request #11460: KAFKA-13425: Optimization of KafkaConsumer#pause semantics

2021-11-02 Thread GitBox
RivenSun2 opened a new pull request #11460: URL: https://github.com/apache/kafka/pull/11460 1. Enhance the annotation of KafkaConsumer#pause(...) method 2. Add log output when clearing the paused mark of topicPartitions Story JIRA: https://issues.apache.org/jira/browse/KAFKA-13425

[jira] [Commented] (KAFKA-13422) Even if the correct username and password are configured, when ClientBroker or KafkaClient tries to establish a SASL connection to ServerBroker, an exception is thrown

2021-11-02 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437699#comment-17437699 ] RivenSun commented on KAFKA-13422: -- Hi [~guozhang] , [~showuon] Can you promote the process of this

[jira] [Commented] (KAFKA-13428) server hang on shutdown

2021-11-02 Thread yuhuo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437697#comment-17437697 ] yuhuo commented on KAFKA-13428: --- Thanks [~dajac]. that’s the same question. > server hang on shutdown >

[GitHub] [kafka] jolshan opened a new pull request #11459: KAFKA-13394: Topic IDs should be removed from PartitionFetchState if they are no longer sent by the controller

2021-11-02 Thread GitBox
jolshan opened a new pull request #11459: URL: https://github.com/apache/kafka/pull/11459 With KAFKA-13102, we added topic IDs to the InitialFetchState and the PartitionFetchState in order to send fetch requests using topic IDs when ibp is 3.1.  However, there are some cases where

[jira] [Commented] (KAFKA-12257) Consumer mishandles topics deleted and recreated with the same name

2021-11-02 Thread Adem Efe Gencer (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437645#comment-17437645 ] Adem Efe Gencer commented on KAFKA-12257: - We were able to reproduce this issue in Kafka 2.4. >

[jira] [Commented] (KAFKA-13373) ValueTransformerWithKeySupplier doesn't work with store()

2021-11-02 Thread Anatoly Tsyganenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437620#comment-17437620 ] Anatoly Tsyganenko commented on KAFKA-13373: And as I said error popped in init() on this

[GitHub] [kafka] vamossagar12 commented on pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2021-11-02 Thread GitBox
vamossagar12 commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957932804 > Personally I'd suggest we enforce the rentention at the inner store, which means: > > 1. for built-in in-memory stores, enforce at the InMemoryXXStore level. >

[GitHub] [kafka] sglass520 commented on pull request #11443: KAFKA-13411: This code enforces the creation of the correct sasl client (OAuthBear…

2021-11-02 Thread GitBox
sglass520 commented on pull request #11443: URL: https://github.com/apache/kafka/pull/11443#issuecomment-957103498 @junrao I can confirm this issue running Kafka in WildFly. The code in createSaslClient() is calling Java Security API to get the factory for specified mechanism. However,

[GitHub] [kafka] cmccabe closed pull request #11383: KAFKA-13340: Change ZooKeeperTestHarness to QuorumTestHarness

2021-11-02 Thread GitBox
cmccabe closed pull request #11383: URL: https://github.com/apache/kafka/pull/11383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] ableegoldman commented on a change in pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2021-11-02 Thread GitBox
ableegoldman commented on a change in pull request #11424: URL: https://github.com/apache/kafka/pull/11424#discussion_r740700461 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -385,7 +385,7 @@ public

[GitHub] [kafka] cadonna commented on pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna commented on pull request #11455: URL: https://github.com/apache/kafka/pull/11455#issuecomment-957320922 @guozhangwang AFAICS not all `setState(DEAD)` are called during normal processing in `GlobalStreamThread`. The `finally` clause in the `run()` is also called when an exception

[GitHub] [kafka] ambroff closed pull request #3110: KAFKA-5297: Reduce time required for shutdown

2021-11-02 Thread GitBox
ambroff closed pull request #3110: URL: https://github.com/apache/kafka/pull/3110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] kkonstantine commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
kkonstantine commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r741409428 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SubmittedRecords.java ## @@ -0,0 +1,162 @@ +/* + * Licensed to the

[GitHub] [kafka] sglass520 edited a comment on pull request #11443: KAFKA-13411: This code enforces the creation of the correct sasl client (OAuthBear…

2021-11-02 Thread GitBox
sglass520 edited a comment on pull request #11443: URL: https://github.com/apache/kafka/pull/11443#issuecomment-957103498 @junrao I can confirm this issue running Kafka in WildFly. The code in createSaslClient() is calling Java Security API to get the factory for specified mechanism.

[jira] [Commented] (KAFKA-13373) ValueTransformerWithKeySupplier doesn't work with store()

2021-11-02 Thread Anatoly Tsyganenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437614#comment-17437614 ] Anatoly Tsyganenko commented on KAFKA-13373: [~Ge] Unfortunately I didn't save sources,

[GitHub] [kafka] dongjinleekr commented on a change in pull request #11430: KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config

2021-11-02 Thread GitBox
dongjinleekr commented on a change in pull request #11430: URL: https://github.com/apache/kafka/pull/11430#discussion_r741090237 ## File path: clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java ## @@ -50,12 +50,24 @@ private final List configEntries;

[GitHub] [kafka] ambroff closed pull request #4540: KAFKA-6469 Batch ISR change notifications

2021-11-02 Thread GitBox
ambroff closed pull request #4540: URL: https://github.com/apache/kafka/pull/4540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] guozhangwang commented on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-958155691 Hi @showuon I think we were referring to different things :) There are different places where we encode the assigned partitions as well as generation: 1) Inside the

[GitHub] [kafka] dajac commented on pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
dajac commented on pull request #11368: URL: https://github.com/apache/kafka/pull/11368#issuecomment-957503502 @mjsax Do we plan to get this PR in 3.1? It seems that the PR has been approved but not merge in time for the feature freeze. -- This is an automated message from the Apache

[GitHub] [kafka] showuon commented on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
showuon commented on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-957039415 @guozhangwang , thanks for your comment. Answer your question below. > Only resetting the generation at the coordinator layer would not change what user data would be

[GitHub] [kafka] dajac merged pull request #11458: MINOR: Bump trunk to 3.2.0-SNAPSHOT

2021-11-02 Thread GitBox
dajac merged pull request #11458: URL: https://github.com/apache/kafka/pull/11458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] C0urante commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
C0urante commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r741279732 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -474,9 +479,24 @@ public boolean

[GitHub] [kafka] showuon edited a comment on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
showuon edited a comment on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-957039415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] cadonna edited a comment on pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna edited a comment on pull request #11455: URL: https://github.com/apache/kafka/pull/11455#issuecomment-957320922 @guozhangwang AFAICS not all `setState(DEAD)` are called during normal processing in `GlobalStreamThread`. The `finally` clause in `run()` is also called when an

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r740921447 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread

[GitHub] [kafka] shirenchuang commented on a change in pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
shirenchuang commented on a change in pull request #11453: URL: https://github.com/apache/kafka/pull/11453#discussion_r740684255 ## File path: core/src/main/scala/kafka/server/ZkAdminManager.scala ## @@ -152,7 +152,7 @@ class ZkAdminManager(val config: KafkaConfig,

[GitHub] [kafka] shirenchuang commented on pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
shirenchuang commented on pull request #11453: URL: https://github.com/apache/kafka/pull/11453#issuecomment-957037691 Thank you for your reply. I plan to repair the logic in kRaft together. In addition, I will add an article to explain why sorting is needed and the solution, and of course

[GitHub] [kafka] guozhangwang commented on pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957002931 Personally I'd suggest we enforce the rentention at the inner store, which means: 1) for built-in in-memory stores, enforce at the InMemoryXXStore level. 2) for

[GitHub] [kafka] mimaison commented on a change in pull request #11450: KAFKA-13414: Replace Powermock/EasyMock by Mockito in connect.storage

2021-11-02 Thread GitBox
mimaison commented on a change in pull request #11450: URL: https://github.com/apache/kafka/pull/11450#discussion_r741054431 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/storage/FileOffsetBackingStoreTest.java ## @@ -77,21 +74,17 @@ public void

[GitHub] [kafka] cmccabe commented on pull request #11383: KAFKA-13340: Change ZooKeeperTestHarness to QuorumTestHarness

2021-11-02 Thread GitBox
cmccabe commented on pull request #11383: URL: https://github.com/apache/kafka/pull/11383#issuecomment-958015023 Duplicate of #11417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] RivenSun2 commented on pull request #11340: KAFKA-13310 : KafkaConsumer cannot jump out of the poll method, and the…

2021-11-02 Thread GitBox
RivenSun2 commented on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-957138929 Hi @showuon and @guozhangwang Any suggestions for my last reply? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vcrfxia commented on a change in pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
vcrfxia commented on a change in pull request #11368: URL: https://github.com/apache/kafka/pull/11368#discussion_r740646037 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/KTable.java ## @@ -2117,12 +2118,37 @@ * @param the key type

[GitHub] [kafka] jonmcewen commented on a change in pull request #8690: KAFKA-9965: Fix uneven distribution in RoundRobinPartitioner

2021-11-02 Thread GitBox
jonmcewen commented on a change in pull request #8690: URL: https://github.com/apache/kafka/pull/8690#discussion_r741175055 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/RoundRobinPartitioner.java ## @@ -65,12 +65,20 @@ public int partition(String

[GitHub] [kafka] kowshik edited a comment on pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2021-11-02 Thread GitBox
kowshik edited a comment on pull request #11390: URL: https://github.com/apache/kafka/pull/11390#issuecomment-956456320 @satishd It'd be helpful if you could please update the PR description explaining the scope of the draft PR (in its current form) and what's remaining to be done. --

[GitHub] [kafka] guozhangwang commented on pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11453: URL: https://github.com/apache/kafka/pull/11453#issuecomment-956859211 +1 to @junrao 's comment, could you fix the sorting in `ReplicationControlManager` too, and also add a unit test (similar to the one you used to illustrate this issue) to

[GitHub] [kafka] cmccabe commented on pull request #11326: KAFKA-9965/KAFKA-13303: RoundRobinPartitioner broken by KIP-480

2021-11-02 Thread GitBox
cmccabe commented on pull request #11326: URL: https://github.com/apache/kafka/pull/11326#issuecomment-956773105 Seems like this is the same issue as https://github.com/apache/kafka/pull/8690 ? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] mjsax commented on a change in pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
mjsax commented on a change in pull request #11368: URL: https://github.com/apache/kafka/pull/11368#discussion_r729526952 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/TableJoined.java ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #11353: KAFKA-13322: Reducing amount of garbage that gets generated during a poll operation

2021-11-02 Thread GitBox
cmccabe commented on a change in pull request #11353: URL: https://github.com/apache/kafka/pull/11353#discussion_r740554575 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java ## @@ -423,9 +423,9 @@ synchronized int

[GitHub] [kafka] junrao commented on a change in pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
junrao commented on a change in pull request #11453: URL: https://github.com/apache/kafka/pull/11453#discussion_r740573525 ## File path: core/src/main/scala/kafka/server/ZkAdminManager.scala ## @@ -152,7 +152,7 @@ class ZkAdminManager(val config: KafkaConfig,

[GitHub] [kafka] junrao commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-11-02 Thread GitBox
junrao commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-956575167 @ccding : Thanks for reporting the test failure. This brings up a good point. It's kind of weird to ever have recovery point > log end offset. I am thinking that another

[GitHub] [kafka] guozhangwang commented on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-958155691 Hi @showuon I think we were referring to different things :) There are different places where we encode the assigned partitions as well as generation: 1) Inside the

[GitHub] [kafka] kkonstantine commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
kkonstantine commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r741409428 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SubmittedRecords.java ## @@ -0,0 +1,162 @@ +/* + * Licensed to the

[GitHub] [kafka] cmccabe commented on pull request #11383: KAFKA-13340: Change ZooKeeperTestHarness to QuorumTestHarness

2021-11-02 Thread GitBox
cmccabe commented on pull request #11383: URL: https://github.com/apache/kafka/pull/11383#issuecomment-958015023 Duplicate of #11417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] cmccabe closed pull request #11383: KAFKA-13340: Change ZooKeeperTestHarness to QuorumTestHarness

2021-11-02 Thread GitBox
cmccabe closed pull request #11383: URL: https://github.com/apache/kafka/pull/11383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] vamossagar12 commented on pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2021-11-02 Thread GitBox
vamossagar12 commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957932804 > Personally I'd suggest we enforce the rentention at the inner store, which means: > > 1. for built-in in-memory stores, enforce at the InMemoryXXStore level. >

[GitHub] [kafka] sglass520 commented on pull request #11443: KAFKA-13411: This code enforces the creation of the correct sasl client (OAuthBear…

2021-11-02 Thread GitBox
sglass520 commented on pull request #11443: URL: https://github.com/apache/kafka/pull/11443#issuecomment-957103498 @junrao I can confirm this issue running Kafka in WildFly. The code in createSaslClient() is calling Java Security API to get the factory for specified mechanism. However,

[GitHub] [kafka] cadonna commented on pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna commented on pull request #11455: URL: https://github.com/apache/kafka/pull/11455#issuecomment-957320922 @guozhangwang AFAICS not all `setState(DEAD)` are called during normal processing in `GlobalStreamThread`. The `finally` clause in the `run()` is also called when an exception

[GitHub] [kafka] ambroff closed pull request #3110: KAFKA-5297: Reduce time required for shutdown

2021-11-02 Thread GitBox
ambroff closed pull request #3110: URL: https://github.com/apache/kafka/pull/3110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] dongjinleekr commented on a change in pull request #11430: KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config

2021-11-02 Thread GitBox
dongjinleekr commented on a change in pull request #11430: URL: https://github.com/apache/kafka/pull/11430#discussion_r741090237 ## File path: clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java ## @@ -50,12 +50,24 @@ private final List configEntries;

[GitHub] [kafka] cmccabe commented on pull request #11326: KAFKA-9965/KAFKA-13303: RoundRobinPartitioner broken by KIP-480

2021-11-02 Thread GitBox
cmccabe commented on pull request #11326: URL: https://github.com/apache/kafka/pull/11326#issuecomment-956773105 Seems like this is the same issue as https://github.com/apache/kafka/pull/8690 ? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] ambroff closed pull request #4540: KAFKA-6469 Batch ISR change notifications

2021-11-02 Thread GitBox
ambroff closed pull request #4540: URL: https://github.com/apache/kafka/pull/4540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] kowshik commented on pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2021-11-02 Thread GitBox
kowshik commented on pull request #11390: URL: https://github.com/apache/kafka/pull/11390#issuecomment-956456320 @satishd It'd be helpful if you could please update the PR description explaining the scope of the draft PR (in its current form) and what's remaining to be done. Feel free to

[jira] [Created] (KAFKA-13429) Update gitignore to include new modules

2021-11-02 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-13429: Summary: Update gitignore to include new modules Key: KAFKA-13429 URL: https://issues.apache.org/jira/browse/KAFKA-13429 Project:

[GitHub] [kafka] dajac commented on pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
dajac commented on pull request #11368: URL: https://github.com/apache/kafka/pull/11368#issuecomment-957503502 @mjsax Do we plan to get this PR in 3.1? It seems that the PR has been approved but not merge in time for the feature freeze. -- This is an automated message from the Apache

[GitHub] [kafka] guozhangwang commented on pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11453: URL: https://github.com/apache/kafka/pull/11453#issuecomment-956859211 +1 to @junrao 's comment, could you fix the sorting in `ReplicationControlManager` too, and also add a unit test (similar to the one you used to illustrate this issue) to

[GitHub] [kafka] showuon commented on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
showuon commented on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-957039415 @guozhangwang , thanks for your comment. Answer your question below. > Only resetting the generation at the coordinator layer would not change what user data would be

[GitHub] [kafka] dajac merged pull request #11458: MINOR: Bump trunk to 3.2.0-SNAPSHOT

2021-11-02 Thread GitBox
dajac merged pull request #11458: URL: https://github.com/apache/kafka/pull/11458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] showuon edited a comment on pull request #11451: [WIP] KAFKA-13419: resetStateAndRejoin when rebalanceInProgress in syncgroup

2021-11-02 Thread GitBox
showuon edited a comment on pull request #11451: URL: https://github.com/apache/kafka/pull/11451#issuecomment-957039415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] C0urante commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
C0urante commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r740365960 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SubmittedRecords.java ## @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cadonna edited a comment on pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna edited a comment on pull request #11455: URL: https://github.com/apache/kafka/pull/11455#issuecomment-957320922 @guozhangwang AFAICS not all `setState(DEAD)` are called during normal processing in `GlobalStreamThread`. The `finally` clause in `run()` is also called when an

[GitHub] [kafka] kowshik edited a comment on pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2021-11-02 Thread GitBox
kowshik edited a comment on pull request #11390: URL: https://github.com/apache/kafka/pull/11390#issuecomment-956456320 @satishd It'd be helpful if you could please update the PR description explaining the scope of the draft PR (in its current form) and what's remaining to be done. --

[GitHub] [kafka] ableegoldman commented on a change in pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2021-11-02 Thread GitBox
ableegoldman commented on a change in pull request #11424: URL: https://github.com/apache/kafka/pull/11424#discussion_r740700461 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -385,7 +385,7 @@ public

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-02 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r740921447 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread

[GitHub] [kafka] shirenchuang commented on a change in pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
shirenchuang commented on a change in pull request #11453: URL: https://github.com/apache/kafka/pull/11453#discussion_r740684255 ## File path: core/src/main/scala/kafka/server/ZkAdminManager.scala ## @@ -152,7 +152,7 @@ class ZkAdminManager(val config: KafkaConfig,

[GitHub] [kafka] junrao commented on a change in pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
junrao commented on a change in pull request #11453: URL: https://github.com/apache/kafka/pull/11453#discussion_r740573525 ## File path: core/src/main/scala/kafka/server/ZkAdminManager.scala ## @@ -152,7 +152,7 @@ class ZkAdminManager(val config: KafkaConfig,

[GitHub] [kafka] guozhangwang commented on pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2021-11-02 Thread GitBox
guozhangwang commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957002931 Personally I'd suggest we enforce the rentention at the inner store, which means: 1) for built-in in-memory stores, enforce at the InMemoryXXStore level. 2) for

[GitHub] [kafka] rhauch commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
rhauch commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r740478689 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -474,9 +479,24 @@ public boolean commitOffsets()

[GitHub] [kafka] cmccabe commented on a change in pull request #11353: KAFKA-13322: Reducing amount of garbage that gets generated during a poll operation

2021-11-02 Thread GitBox
cmccabe commented on a change in pull request #11353: URL: https://github.com/apache/kafka/pull/11353#discussion_r740554575 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java ## @@ -423,9 +423,9 @@ synchronized int

[GitHub] [kafka] shirenchuang commented on pull request #11453: createTopic need sort brokerMetadata

2021-11-02 Thread GitBox
shirenchuang commented on pull request #11453: URL: https://github.com/apache/kafka/pull/11453#issuecomment-957037691 Thank you for your reply. I plan to repair the logic in kRaft together. In addition, I will add an article to explain why sorting is needed and the solution, and of course

[GitHub] [kafka] mimaison commented on a change in pull request #11450: KAFKA-13414: Replace Powermock/EasyMock by Mockito in connect.storage

2021-11-02 Thread GitBox
mimaison commented on a change in pull request #11450: URL: https://github.com/apache/kafka/pull/11450#discussion_r741054431 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/storage/FileOffsetBackingStoreTest.java ## @@ -77,21 +74,17 @@ public void

[GitHub] [kafka] guozhangwang commented on a change in pull request #11449: MINOR: Log client disconnect events at INFO level

2021-11-02 Thread GitBox
guozhangwang commented on a change in pull request #11449: URL: https://github.com/apache/kafka/pull/11449#discussion_r740417424 ## File path: clients/src/main/java/org/apache/kafka/clients/NetworkClient.java ## @@ -319,23 +319,29 @@ public void disconnect(String nodeId) {

[GitHub] [kafka] junrao commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-11-02 Thread GitBox
junrao commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-956575167 @ccding : Thanks for reporting the test failure. This brings up a good point. It's kind of weird to ever have recovery point > log end offset. I am thinking that another

[GitHub] [kafka] sglass520 edited a comment on pull request #11443: KAFKA-13411: This code enforces the creation of the correct sasl client (OAuthBear…

2021-11-02 Thread GitBox
sglass520 edited a comment on pull request #11443: URL: https://github.com/apache/kafka/pull/11443#issuecomment-957103498 @junrao I can confirm this issue running Kafka in WildFly. The code in createSaslClient() is calling Java Security API to get the factory for specified mechanism.

[GitHub] [kafka] RivenSun2 commented on pull request #11340: KAFKA-13310 : KafkaConsumer cannot jump out of the poll method, and the…

2021-11-02 Thread GitBox
RivenSun2 commented on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-957138929 Hi @showuon and @guozhangwang Any suggestions for my last reply? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] vcrfxia commented on a change in pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
vcrfxia commented on a change in pull request #11368: URL: https://github.com/apache/kafka/pull/11368#discussion_r740646037 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/KTable.java ## @@ -2117,12 +2118,37 @@ * @param the key type

[GitHub] [kafka] jonmcewen commented on a change in pull request #8690: KAFKA-9965: Fix uneven distribution in RoundRobinPartitioner

2021-11-02 Thread GitBox
jonmcewen commented on a change in pull request #8690: URL: https://github.com/apache/kafka/pull/8690#discussion_r741175055 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/RoundRobinPartitioner.java ## @@ -65,12 +65,20 @@ public int partition(String

[GitHub] [kafka] mjsax commented on a change in pull request #11368: KAFKA-13261: Add support for custom partitioners in foreign key joins

2021-11-02 Thread GitBox
mjsax commented on a change in pull request #11368: URL: https://github.com/apache/kafka/pull/11368#discussion_r729526952 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/TableJoined.java ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-13426) Add recordMetadata to StateStoreContext

2021-11-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13426: Labels: kip (was: ) > Add recordMetadata to StateStoreContext >

[jira] [Updated] (KAFKA-13426) Add recordMetadata to StateStoreContext

2021-11-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13426: Description: KIP-791:

[GitHub] [kafka] vamossagar12 commented on pull request #11211: KAFKA-12960: Enforcing strict retention time for WindowStore and Sess…

2021-11-02 Thread GitBox
vamossagar12 commented on pull request #11211: URL: https://github.com/apache/kafka/pull/11211#issuecomment-957932804 > Personally I'd suggest we enforce the rentention at the inner store, which means: > > 1. for built-in in-memory stores, enforce at the InMemoryXXStore level. >

[GitHub] [kafka] C0urante commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
C0urante commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r741279732 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -474,9 +479,24 @@ public boolean

[GitHub] [kafka] C0urante commented on a change in pull request #11323: KAFKA-12226: Commit source task offsets without blocking on batch delivery

2021-11-02 Thread GitBox
C0urante commented on a change in pull request #11323: URL: https://github.com/apache/kafka/pull/11323#discussion_r741279732 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -474,9 +479,24 @@ public boolean

[jira] [Commented] (KAFKA-13428) server hang on shutdown

2021-11-02 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437424#comment-17437424 ] David Jacot commented on KAFKA-13428: - [~yuhuo] Thanks for reporting this. It seems similar to

[GitHub] [kafka] jonmcewen commented on a change in pull request #8690: KAFKA-9965: Fix uneven distribution in RoundRobinPartitioner

2021-11-02 Thread GitBox
jonmcewen commented on a change in pull request #8690: URL: https://github.com/apache/kafka/pull/8690#discussion_r741175055 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/RoundRobinPartitioner.java ## @@ -65,12 +65,20 @@ public int partition(String

[jira] [Commented] (KAFKA-6080) Transactional EoS for source connectors

2021-11-02 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437419#comment-17437419 ] Tom Bentley commented on KAFKA-6080: [~sliebau],

[jira] [Commented] (KAFKA-13425) KafkaConsumer#pause() will lose its effect after groupRebalance occurs, which maybe cause data loss on the consumer side

2021-11-02 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437413#comment-17437413 ] RivenSun commented on KAFKA-13425: -- [~showuon] In fact, the purpose of the modified code I added is to

[jira] [Commented] (KAFKA-13247) Adding functionality for loading private key entry by alias from the keystore

2021-11-02 Thread Dongjin Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437398#comment-17437398 ] Dongjin Lee commented on KAFKA-13247: - Hi [~tmargaryan]  Thank you for reporting this issue. I was

[jira] [Assigned] (KAFKA-13183) Dropping nul key/value records upstream to repartiton topic not tracked via metrics

2021-11-02 Thread Aleksandr Sorokoumov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Sorokoumov reassigned KAFKA-13183: Assignee: Aleksandr Sorokoumov > Dropping nul key/value records upstream

[GitHub] [kafka] dongjinleekr commented on a change in pull request #11430: KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config

2021-11-02 Thread GitBox
dongjinleekr commented on a change in pull request #11430: URL: https://github.com/apache/kafka/pull/11430#discussion_r741104074 ## File path: clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java ## @@ -50,12 +50,24 @@ private final List configEntries;

[GitHub] [kafka] dongjinleekr commented on a change in pull request #11430: KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config

2021-11-02 Thread GitBox
dongjinleekr commented on a change in pull request #11430: URL: https://github.com/apache/kafka/pull/11430#discussion_r741090237 ## File path: clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java ## @@ -50,12 +50,24 @@ private final List configEntries;

[jira] [Commented] (KAFKA-6080) Transactional EoS for source connectors

2021-11-02 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-6080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437377#comment-17437377 ] Sönke Liebau commented on KAFKA-6080: - Are there any developments around this? I recently came across

[GitHub] [kafka] dongjinleekr commented on a change in pull request #11430: KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config

2021-11-02 Thread GitBox
dongjinleekr commented on a change in pull request #11430: URL: https://github.com/apache/kafka/pull/11430#discussion_r741090237 ## File path: clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java ## @@ -50,12 +50,24 @@ private final List configEntries;

[jira] [Updated] (KAFKA-13348) Allow Source Tasks to Handle Producer Exceptions

2021-11-02 Thread Knowles Atchison Jr (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knowles Atchison Jr updated KAFKA-13348: Description: KAFKA-8586 added capture of Producer Exceptions which will kill the

[jira] [Updated] (KAFKA-13348) Allow Source Tasks to Handle Producer Exceptions

2021-11-02 Thread Knowles Atchison Jr (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knowles Atchison Jr updated KAFKA-13348: Description: KAFKA-8586 added capture of Producer Exceptions which will kill the

[jira] [Updated] (KAFKA-13348) Allow Source Tasks to Handle Producer Exceptions

2021-11-02 Thread Knowles Atchison Jr (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knowles Atchison Jr updated KAFKA-13348: Description: KAFKA-8586 added capture of Producer Exceptions which will kill the

[jira] [Commented] (KAFKA-13373) ValueTransformerWithKeySupplier doesn't work with store()

2021-11-02 Thread Aleksandr Sorokoumov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437367#comment-17437367 ] Aleksandr Sorokoumov commented on KAFKA-13373: -- [~htol] Can you please provide an example

[jira] [Commented] (KAFKA-13425) KafkaConsumer#pause() will lose its effect after groupRebalance occurs, which maybe cause data loss on the consumer side

2021-11-02 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437366#comment-17437366 ] RivenSun commented on KAFKA-13425: -- [~showuon] So we have reached a consensus:When the rebalance is

  1   2   >