[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r970342727 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/WorkerErrantRecordReporterTest.java: ## @@ -78,4 +68,41 @@ public void testGetFutures() {

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r970342561 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperatorTest.java: ## @@ -250,97 +258,90 @@ public void

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r970342294 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperatorTest.java: ## @@ -222,25 +224,31 @@ public void

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970291640 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1129,31 +1129,34 @@ public void

[GitHub] [kafka] junrao commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
junrao commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970263402 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1129,31 +1129,34 @@ public void onCompletion(RecordMetadata

[GitHub] [kafka] showuon commented on pull request #12589: KAFKA-14197: use raft error RaftBatchTooLargeException instead of exi…

2022-09-13 Thread GitBox
showuon commented on PR #12589: URL: https://github.com/apache/kafka/pull/12589#issuecomment-1246131692 Agree. Closed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] showuon closed pull request #12589: KAFKA-14197: use raft error RaftBatchTooLargeException instead of exi…

2022-09-13 Thread GitBox
showuon closed pull request #12589: KAFKA-14197: use raft error RaftBatchTooLargeException instead of exi… URL: https://github.com/apache/kafka/pull/12589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] dengziming commented on pull request #12589: KAFKA-14197: use raft error RaftBatchTooLargeException instead of exi…

2022-09-13 Thread GitBox
dengziming commented on PR #12589: URL: https://github.com/apache/kafka/pull/12589#issuecomment-1246128861 This is already fixed, we can close it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] showuon commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
showuon commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1246111859 Make sense to me. Thank you all. I'll close this PR then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] showuon closed pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
showuon closed pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1 URL: https://github.com/apache/kafka/pull/12620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (KAFKA-14224) Consumer should auto-commit prior to cooperative partition revocation

2022-09-13 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-14224: Description: With the old "eager" reassignment logic, we always revoked all partitions

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
guozhangwang commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r970140299 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -305,6 +305,32 @@ public void buildAndOptimizeTopology(final

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970145425 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970145425 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void

[jira] [Assigned] (KAFKA-13656) Connect Transforms support for nested structures

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya reassigned KAFKA-13656: Assignee: Jorge Esteban Quilcate Otoya > Connect

[GitHub] [kafka] jeqo opened a new pull request, #12637: KAFKA-14226: [connect:transform] Introduce support for nested structures

2022-09-13 Thread GitBox
jeqo opened a new pull request, #12637: URL: https://github.com/apache/kafka/pull/12637 KIP-821 initial implementation. *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and

[jira] [Created] (KAFKA-14231) Support for nested structures: ReplaceField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14231: Summary: Support for nested structures: ReplaceField Key: KAFKA-14231 URL: https://issues.apache.org/jira/browse/KAFKA-14231 Project:

[jira] [Created] (KAFKA-14230) Support for nested structures: Cast

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14230: Summary: Support for nested structures: Cast Key: KAFKA-14230 URL: https://issues.apache.org/jira/browse/KAFKA-14230 Project: Kafka

[jira] [Created] (KAFKA-14232) Support for nested structures: InsertField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14232: Summary: Support for nested structures: InsertField Key: KAFKA-14232 URL: https://issues.apache.org/jira/browse/KAFKA-14232 Project:

[jira] [Created] (KAFKA-14228) Support for nested structures: ValueToKey

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14228: Summary: Support for nested structures: ValueToKey Key: KAFKA-14228 URL: https://issues.apache.org/jira/browse/KAFKA-14228 Project:

[jira] [Created] (KAFKA-14229) Support for nested structures: HoistValue

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14229: Summary: Support for nested structures: HoistValue Key: KAFKA-14229 URL: https://issues.apache.org/jira/browse/KAFKA-14229 Project:

[jira] [Created] (KAFKA-14227) Support for nested structures: MaskField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14227: Summary: Support for nested structures: MaskField Key: KAFKA-14227 URL: https://issues.apache.org/jira/browse/KAFKA-14227 Project:

[jira] [Created] (KAFKA-14226) Introduce support for nested structures

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14226: Summary: Introduce support for nested structures Key: KAFKA-14226 URL: https://issues.apache.org/jira/browse/KAFKA-14226 Project:

[GitHub] [kafka] cmccabe opened a new pull request, #12636: KAFKA-14214: Convert StandardAuthorizer to copy-on-write

2022-09-13 Thread GitBox
cmccabe opened a new pull request, #12636: URL: https://github.com/apache/kafka/pull/12636 Convert StandardAuthorizer to use a copy-on-write sorted array rather than a ConcurrentSkipList. The issue with the skiplist was that because it was modified while in use by

[jira] [Commented] (KAFKA-10635) Streams application fails with OutOfOrderSequenceException after rolling restarts of brokers

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603789#comment-17603789 ] Guozhang Wang commented on KAFKA-10635: --- Hi [~nicktelford], regarding the broker's behavior: >

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970110815 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970110217 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] C0urante commented on pull request #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
C0urante commented on PR #12632: URL: https://github.com/apache/kafka/pull/12632#issuecomment-1245985132 Should we also update the system test [here](https://github.com/apache/kafka/blob/7b49f175b964c468d5ca7ac9bd70dfd72e6a2e4e/tests/kafkatest/services/streams.py#L547)? -- This is an

[GitHub] [kafka] C0urante commented on a diff in pull request #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
C0urante commented on code in PR #12632: URL: https://github.com/apache/kafka/pull/12632#discussion_r970105520 ## core/src/main/scala/kafka/tools/StreamsResetter.java: ## @@ -213,8 +217,12 @@ private void parseArguments(final String[] args) { .ofType(String.class)

[GitHub] [kafka] ijuma commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
ijuma commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970107576 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970105135 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r970065964 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperatorTest.java: ## @@ -222,25 +224,31 @@ public void

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-13 Thread GitBox
guozhangwang commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r970078065 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/Tasks.java: ## @@ -103,42 +101,75 @@ public void addPendingStandbyTasksToCreate(final Map

[GitHub] [kafka] junrao commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
junrao commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970071437 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void onCompletion(RecordMetadata

[jira] [Commented] (KAFKA-14224) Consumer should auto-commit prior to cooperative partition revocation

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603743#comment-17603743 ] Guozhang Wang commented on KAFKA-14224: --- cc [~pnee] I think we can address this by always pairing

[jira] [Updated] (KAFKA-14224) Consumer should auto-commit prior to cooperative partition revocation

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-14224: -- Labels: new-consumer-threading-should-fix (was: ) > Consumer should auto-commit prior to

[GitHub] [kafka] ijuma opened a new pull request, #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
ijuma opened a new pull request, #12635: URL: https://github.com/apache/kafka/pull/12635 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes)

[GitHub] [kafka] ijuma merged pull request #12630: MINOR; Explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma merged PR #12630: URL: https://github.com/apache/kafka/pull/12630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12630: MINOR; Explain how to set old default partitioner

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969977220 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky

[GitHub] [kafka] hshi2022 opened a new pull request, #12634: KAFKA-14225: Fix deadlock caused by lazy val exemptSensor

2022-09-13 Thread GitBox
hshi2022 opened a new pull request, #12634: URL: https://github.com/apache/kafka/pull/12634 …ntRequestQuotaManager. In the version Scala 2.12, the lazy val initialization is under the object lock. The deadlock could happen in the following condition: In thread a, when

[jira] [Updated] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huilin Shi updated KAFKA-14225: --- Description: There is a chance to cause deadlock when multiple threads access

[jira] [Updated] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huilin Shi updated KAFKA-14225: --- Description: There is a chance to cause deadlock when multiple threads access

[GitHub] [kafka] hachikuji merged pull request #12629: MINOR: Clean up FetcherTest

2022-09-13 Thread GitBox
hachikuji merged PR #12629: URL: https://github.com/apache/kafka/pull/12629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
Huilin Shi created KAFKA-14225: -- Summary: lazy val exemptSensor Could Cause Deadlock Key: KAFKA-14225 URL: https://issues.apache.org/jira/browse/KAFKA-14225 Project: Kafka Issue Type: Bug

[GitHub] [kafka] ijuma commented on a diff in pull request #12392: KAFKA-14053: Transactional producer should bump the epoch and skip ab…

2022-09-13 Thread GitBox
ijuma commented on code in PR #12392: URL: https://github.com/apache/kafka/pull/12392#discussion_r969939389 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -2594,27 +2596,20 @@ public void testDropCommitOnBatchExpiry()

[GitHub] [kafka] philipnee commented on pull request #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee commented on PR #12633: URL: https://github.com/apache/kafka/pull/12633#issuecomment-1245767412 @showuon - Seems like you are quite familiar with the kafka consumer. So, cc you here. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] philipnee commented on pull request #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee commented on PR #12633: URL: https://github.com/apache/kafka/pull/12633#issuecomment-1245764396 @hachikuji @guozhangwang - This is the skeleton of the background thread. Please feel free to pull in reviewers. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] philipnee opened a new pull request, #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee opened a new pull request, #12633: URL: https://github.com/apache/kafka/pull/12633 * This is part of the effort for the consumer refactor project. This PR is the skeleton of the background thread that runs in the background. *Testing strategies Tests will be add once

[GitHub] [kafka] nizhikov opened a new pull request, #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
nizhikov opened a new pull request, #12632: URL: https://github.com/apache/kafka/pull/12632 Implementation of [KIP-865](https://cwiki.apache.org/confluence/display/KAFKA/KIP-865%3A+Support+--bootstrap-server+in+kafka-streams-application-reset) Support of `--bootstrap-server` option

[GitHub] [kafka] littlehorse-eng commented on a diff in pull request #12392: KAFKA-14053: Transactional producer should bump the epoch and skip ab…

2022-09-13 Thread GitBox
littlehorse-eng commented on code in PR #12392: URL: https://github.com/apache/kafka/pull/12392#discussion_r969894336 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -2594,27 +2596,20 @@ public void

[jira] [Updated] (KAFKA-14206) Upgrade zookeeper to 3.7.1 to address security vulnerabilities

2022-09-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-14206: --- Fix Version/s: 3.4.0 > Upgrade zookeeper to 3.7.1 to address

[jira] [Resolved] (KAFKA-14073) Logging the reason for creating a snapshot

2022-09-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-14073. Resolution: Fixed > Logging the reason for creating a snapshot >

[GitHub] [kafka] jsancio merged pull request #12414: KAFKA-14073; Log the reason for snapshot

2022-09-13 Thread GitBox
jsancio merged PR #12414: URL: https://github.com/apache/kafka/pull/12414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jsancio commented on pull request #12610: MINOR; Update documentation for printing dependencies

2022-09-13 Thread GitBox
jsancio commented on PR #12610: URL: https://github.com/apache/kafka/pull/12610#issuecomment-1245683056 @ijuma What do you think? Can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante closed pull request #12023: Create AbstractLogContext class with static and dynamic subclasses, remove DynamicPrefixLogger

2022-09-13 Thread GitBox
C0urante closed pull request #12023: Create AbstractLogContext class with static and dynamic subclasses, remove DynamicPrefixLogger URL: https://github.com/apache/kafka/pull/12023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-14220: -- Reviewer: Chris Egerton (was: Konstantine Karantasis) > "partition-count" not getting

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-14220: -- Fix Version/s: (was: 3.0.1) > "partition-count" not getting updated after revocation of

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969850848 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969848670 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969819802 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void

[GitHub] [kafka] mumrah commented on pull request #12627: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-13 Thread GitBox
mumrah commented on PR #12627: URL: https://github.com/apache/kafka/pull/12627#issuecomment-1245603207 This was closed in favor of #12628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] ijuma commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
ijuma commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245590870 One more thing: ZK is going away in AK 4.0. We probably need to do one more upgrade before that. It may make sense to wait a bit and go straight to 3.8 so that we benefit from longer

[GitHub] [kafka] ijuma commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
ijuma commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245588332 Exactly right @divijvaidya, we have generally fixed transitive dependency issues directly. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] jsancio merged pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-13 Thread GitBox
jsancio merged PR #12625: URL: https://github.com/apache/kafka/pull/12625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jsancio commented on pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-13 Thread GitBox
jsancio commented on PR #12625: URL: https://github.com/apache/kafka/pull/12625#issuecomment-1245546902 Thanks @hachikuji . Merging -- the failures look unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969737990 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,82 +78,41 @@ public void tearDown() { public void

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969668198 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,82 +78,41 @@ public void tearDown() { public void

[GitHub] [kafka] vamossagar12 commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

2022-09-13 Thread GitBox
vamossagar12 commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1245454894 > @vamossagar12 there are failing unit tests, can you check those out and ping us after you've taken a look and fixed any issues? Oops.. Sorry about that. I didn't check the

[GitHub] [kafka] C0urante commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

2022-09-13 Thread GitBox
C0urante commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1245436111 @vamossagar12 there are failing unit tests, can you check those out and ping us after you've taken a look and fixed any issues? -- This is an automated message from the Apache Git

[GitHub] [kafka] C0urante commented on pull request #12622: Update WorkerSinkTask.java

2022-09-13 Thread GitBox
C0urante commented on PR #12622: URL: https://github.com/apache/kafka/pull/12622#issuecomment-1245423635 Thanks @kumarpritam863, this is an important fix. I did some research on the history behind incremental consumer rebalancing and how it relates to the rebalance listener API, and found

[GitHub] [kafka] C0urante commented on a diff in pull request #12602: KAFKA-13985: Skip committing MirrorSourceTask records without metadata

2022-09-13 Thread GitBox
C0urante commented on code in PR #12602: URL: https://github.com/apache/kafka/pull/12602#discussion_r969617914 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorSourceTaskTest.java: ## @@ -160,6 +160,33 @@ public void testPoll() { } } +

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner,

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner,

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner,

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner,

[GitHub] [kafka] divijvaidya commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
divijvaidya commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245356187 I did some analysis on what has changed and here is my summary: ZK 3.7.1 [contains CVE fixes](https://zookeeper.apache.org/doc/r3.7.1/releasenotes.html) for: 1.

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969358545 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -1644,6 +1668,29 @@ private Map getClientCustomProps() { return props; } +

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969520183 ## streams/src/test/java/org/apache/kafka/streams/integration/SelfJoinUpgradeIntegrationTest.java: ## @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Pritam Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritam Kumar updated KAFKA-14220: - Description: Issue: 
In case of the revocation of partitions, the updation of "partition count"

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969484570 ## streams/src/test/java/org/apache/kafka/streams/StreamsConfigTest.java: ## @@ -1263,6 +1263,37 @@ public void testInvalidSecurityProtocol() {

[GitHub] [kafka] cadonna commented on a diff in pull request #12449: KAFKA-12947: Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest

2022-09-13 Thread GitBox
cadonna commented on code in PR #12449: URL: https://github.com/apache/kafka/pull/12449#discussion_r969376097 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/metrics/StreamsMetricsImplTest.java: ## @@ -287,7 +279,7 @@ public void

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969391230 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -663,6 +663,12 @@ public class StreamsConfig extends AbstractConfig { public static

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969371482 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java: ## @@ -181,13 +181,22 @@ public KStream join(final KStream lhs,

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969371482 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java: ## @@ -181,13 +181,22 @@ public KStream join(final KStream lhs,

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969358545 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -1644,6 +1668,29 @@ private Map getClientCustomProps() { return props; } +

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969353780 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamSelfJoin.java: ## @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969353364 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -305,6 +307,36 @@ public void buildAndOptimizeTopology(final

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969351766 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,17 +277,12 @@ private void

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969349691 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969341737 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969340204 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969339721 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[jira] [Updated] (KAFKA-14196) Duplicated consumption during rebalance, causing OffsetValidationTest to act flaky

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-14196: -- Fix Version/s: 3.2.2 (was: 3.2.3) > Duplicated consumption during rebalance,

[jira] [Resolved] (KAFKA-14196) Duplicated consumption during rebalance, causing OffsetValidationTest to act flaky

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-14196. --- Resolution: Fixed > Duplicated consumption during rebalance, causing OffsetValidationTest to act >

[jira] [Resolved] (KAFKA-14208) KafkaConsumer#commitAsync throws unexpected WakeupException

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-14208. --- Resolution: Fixed > KafkaConsumer#commitAsync throws unexpected WakeupException >

[jira] [Updated] (KAFKA-14208) KafkaConsumer#commitAsync throws unexpected WakeupException

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-14208: -- Fix Version/s: 3.2.2 > KafkaConsumer#commitAsync throws unexpected WakeupException >

  1   2   >