Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1410274394 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryProvider.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1410274394 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryProvider.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1410273486 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryProvider.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-15831: KIP-1000 protocol and admin client [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield closed pull request #14811: KAFKA-15831: KIP-1000 protocol and admin client URL: https://github.com/apache/kafka/pull/14811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1410260721 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java: ## @@ -454,4 +480,19 @@ public void shutdown() { public Map

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1410258175 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1791,6 +1794,52 @@ protected int processStreamThread(final Consumer consumer) {

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
mjsax commented on PR #14864: URL: https://github.com/apache/kafka/pull/14864#issuecomment-1833242672 @AndrewJSchofield -- updated this PR to cover more cases. Still not complete, but more review input is welcome. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15817: Avoid reconnecting to the same IP address if multiple ad… [kafka]

2023-11-29 Thread via GitHub
ijuma commented on PR #14813: URL: https://github.com/apache/kafka/pull/14813#issuecomment-1833215345 @lucasbru Did you cherry-pick to 3.6? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on PR #14843: URL: https://github.com/apache/kafka/pull/14843#issuecomment-1833203010 Hi @kirktrue, thanks for taking time out to review and leaving feedback. I have addressed the feedback, can you please re-review. -- This is an automated message from the

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1410224871 ## clients/src/main/java/org/apache/kafka/clients/NetworkClient.java: ## @@ -361,6 +370,9 @@ private void cancelInFlightRequests(String nodeId,

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1410198452 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -4387,7 +4398,52 @@ public FenceProducersResult fenceProducers(Collection

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1410158203 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3697,16 +3698,60 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1410157251 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3747,16 +3747,56 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
junrao commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1410140939 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3697,16 +3698,60 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] KAFKA-15704: Update clientId and clientHost in MemberMetadata when static member is replaced. [kafka]

2023-11-29 Thread via GitHub
Lucent-Wong commented on PR #14666: URL: https://github.com/apache/kafka/pull/14666#issuecomment-1833071663 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1410123451 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryUtils.java: ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1410120896 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3697,16 +3698,60 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] Source task stop call was added to force stopping execution. [kafka]

2023-11-29 Thread via GitHub
github-actions[bot] commented on PR #14101: URL: https://github.com/apache/kafka/pull/14101#issuecomment-1833053201 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on PR #14724: URL: https://github.com/apache/kafka/pull/14724#issuecomment-1833016137 @mjsax I have also moved ClientTelemetryUtils to `internals` package in this PR as well, the class is new and have changes in both PRs. -- This is an automated message from the

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410046365 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410043469 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410043469 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-13403 Fix KafkaServer crashes when deleting topics due to the race in log deletion [kafka]

2023-11-29 Thread via GitHub
arunmathew88 commented on PR #11438: URL: https://github.com/apache/kafka/pull/11438#issuecomment-1832945607 @divijvaidya A gentle nudge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] MINOR: Fix flaky BrokerLifecycleManagerTest [kafka]

2023-11-29 Thread via GitHub
soarez commented on code in PR #14836: URL: https://github.com/apache/kafka/pull/14836#discussion_r1410038816 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -201,7 +201,7 @@ class BrokerLifecycleManagerTest { while (!future.isDone ||

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
hachikuji commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410034374 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

[PR] MINOR: Fix flaky `MetadataLoaderTest.testNoPublishEmptyImage` [kafka]

2023-11-29 Thread via GitHub
hachikuji opened a new pull request, #14875: URL: https://github.com/apache/kafka/pull/14875 There is a race in the assertion on `capturedImages`. Since the future is signaled first, it is still possible to see an empty list. ### Committer Checklist (excluded from commit message)

Re: [PR] KAFKA-15910: New group coordinator needs to generate snapshots while loading [kafka]

2023-11-29 Thread via GitHub
jeffkbkim commented on code in PR #14849: URL: https://github.com/apache/kafka/pull/14849#discussion_r1410029648 ## core/src/main/scala/kafka/coordinator/group/CoordinatorLoaderImpl.scala: ## @@ -148,6 +156,16 @@ class CoordinatorLoaderImpl[T]( }

[PR] MINOR: Comment explaining test approach [kafka]

2023-11-29 Thread via GitHub
soarez opened a new pull request, #14874: URL: https://github.com/apache/kafka/pull/14874 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

Re: [PR] MINOR: No need for response callback when applying controller mutation throttle [kafka]

2023-11-29 Thread via GitHub
hachikuji merged PR #14861: URL: https://github.com/apache/kafka/pull/14861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: No need for response callback when applying controller mutation throttle [kafka]

2023-11-29 Thread via GitHub
hachikuji commented on PR #14861: URL: https://github.com/apache/kafka/pull/14861#issuecomment-1832915608 All failures seem to be flaky. I tested them locally. I will merge to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410014399 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
hachikuji commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1410013739 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

[jira] [Commented] (KAFKA-15950) CommunicationEvent should be scheduled with EarliestDeadlineFunction

2023-11-29 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791367#comment-17791367 ] Jun Rao commented on KAFKA-15950: - cc [~soarez]  > CommunicationEvent should be scheduled with

[jira] [Created] (KAFKA-15950) CommunicationEvent should be scheduled with EarliestDeadlineFunction

2023-11-29 Thread Jun Rao (Jira)
Jun Rao created KAFKA-15950: --- Summary: CommunicationEvent should be scheduled with EarliestDeadlineFunction Key: KAFKA-15950 URL: https://issues.apache.org/jira/browse/KAFKA-15950 Project: Kafka

[jira] [Updated] (KAFKA-15941) Flaky test: shouldRestoreNullRecord() – org.apache.kafka.streams.integration.RestoreIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15941: Labels: flaky-test (was: ) > Flaky test: shouldRestoreNullRecord() – >

[jira] [Updated] (KAFKA-15941) Flaky test: shouldRestoreNullRecord() – org.apache.kafka.streams.integration.RestoreIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15941: Component/s: streams unit tests > Flaky test: shouldRestoreNullRecord()

[jira] [Updated] (KAFKA-15944) Flaky test - verifyStore[cache=false, log=true, supplier=ROCKS_KV, kind=DSL] – org.apache.kafka.streams.integration.PositionRestartIntegrationTest

2023-11-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15944: Component/s: unit tests > Flaky test - verifyStore[cache=false, log=true,

[jira] [Resolved] (KAFKA-15311) Fix docs about reverting to ZooKeeper mode during KRaft migration

2023-11-29 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15311. -- Fix Version/s: 3.7.0 Resolution: Fixed > Fix docs about reverting to ZooKeeper mode

Re: [PR] MINOR: Zk to KRaft migration is now production ready [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on PR #14546: URL: https://github.com/apache/kafka/pull/14546#issuecomment-1832839662 I'm going to close this for now since it is superseded by #14160. If you find more stuff that should be fixed, please do open a new one! :) -- This is an automated message from the

Re: [PR] MINOR: Zk to KRaft migration is now production ready [kafka]

2023-11-29 Thread via GitHub
cmccabe closed pull request #14546: MINOR: Zk to KRaft migration is now production ready URL: https://github.com/apache/kafka/pull/14546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15886: Always specify directories for new partition registrations [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14820: URL: https://github.com/apache/kafka/pull/14820#discussion_r1409963570 ## metadata/src/main/java/org/apache/kafka/metadata/placement/DefaultDirProvider.java: ## @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409942673 ## server-common/src/test/java/org/apache/kafka/server/common/MetadataVersionTest.java: ## @@ -386,4 +418,21 @@ public void

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409942437 ## server-common/src/test/java/org/apache/kafka/server/common/MetadataVersionTest.java: ## @@ -329,19 +336,44 @@ public void

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409941745 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -207,6 +213,22 @@ public enum MetadataVersion { */ public static

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409941250 ## metadata/src/main/resources/common/metadata/PartitionRecord.json: ## @@ -42,13 +42,13 @@ "about": "The epoch of the partition leader." }, { "name":

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409938934 ## core/src/test/scala/unit/kafka/cluster/PartitionTest.scala: ## @@ -2191,7 +2191,7 @@ class PartitionTest extends AbstractPartitionTest { val partition = new

[jira] [Created] (KAFKA-15949) Improve the KRaft metadata version related messages

2023-11-29 Thread Jakub Scholz (Jira)
Jakub Scholz created KAFKA-15949: Summary: Improve the KRaft metadata version related messages Key: KAFKA-15949 URL: https://issues.apache.org/jira/browse/KAFKA-15949 Project: Kafka Issue

[jira] [Commented] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2023-11-29 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791354#comment-17791354 ] Haruki Okada commented on KAFKA-9693: - [~paolomoriello] [~novosibman] Hi, I believe the latency spike

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
junrao commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1409930152 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3697,16 +3698,60 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] MINOR: Change test logging capture to per-test, reducing jenkins truncation [kafka]

2023-11-29 Thread via GitHub
gharris1727 commented on PR #14795: URL: https://github.com/apache/kafka/pull/14795#issuecomment-1832781742 Also worth a follow-up is maybe tackling the outliers that do experience truncation, and try to reduce their log volume to a more reasonable level. The biggest offenders above 10MB

Re: [PR] KAFKA-15645 ReplicationQuotasTestRig rewritten in java [kafka]

2023-11-29 Thread via GitHub
nizhikov commented on PR #14588: URL: https://github.com/apache/kafka/pull/14588#issuecomment-1832765723 Hello @jolshan are you ready to merge this? Can I improve this PR somehow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] MINOR: Change test logging capture to per-test, reducing jenkins truncation [kafka]

2023-11-29 Thread via GitHub
gharris1727 commented on PR #14795: URL: https://github.com/apache/kafka/pull/14795#issuecomment-1832757011 I did some statistics on the current state of log truncation in CI. I learned that: * A full run of `./gradlew test` writes 1.73GB of logs * 814 of 1532 test suites (53%)

[jira] [Resolved] (KAFKA-15947) Null pointer on LZ4 compression since Kafka 3.6

2023-11-29 Thread Ludo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ludo resolved KAFKA-15947. -- Fix Version/s: 3.6.1 Resolution: Duplicate > Null pointer on LZ4 compression since Kafka 3.6 >

[jira] [Commented] (KAFKA-15947) Null pointer on LZ4 compression since Kafka 3.6

2023-11-29 Thread Ludo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791343#comment-17791343 ] Ludo commented on KAFKA-15947: -- Yes seems the same, and yes I use transaction (with Kafka Streams). 

[jira] [Comment Edited] (KAFKA-15947) Null pointer on LZ4 compression since Kafka 3.6

2023-11-29 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791332#comment-17791332 ] Justine Olshan edited comment on KAFKA-15947 at 11/29/23 9:41 PM: -- Hey

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-11-29 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1832745572 In the current implementation, when poll timer expires, the consumer sends a leavegroup request w/o revoking the partitions. This is obviously a bit different to the current state

[jira] [Commented] (KAFKA-15947) Null pointer on LZ4 compression since Kafka 3.6

2023-11-29 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791332#comment-17791332 ] Justine Olshan commented on KAFKA-15947: Hey – is this the same as

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409900140 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -59,10 +59,14 @@ object StorageTool extends Logging { case "format" => val directories

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-11-29 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1409899121 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -169,6 +174,19 @@ public NetworkClientDelegate.PollResult

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1409884417 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-11-29 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1409862992 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -169,6 +174,19 @@ public NetworkClientDelegate.PollResult

Re: [PR] MINOR: DirectoryId.MIGRATING should be all zeros [kafka]

2023-11-29 Thread via GitHub
cmccabe merged PR #14858: URL: https://github.com/apache/kafka/pull/14858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15886: Always specify directories for new partition registrations [kafka]

2023-11-29 Thread via GitHub
soarez commented on code in PR #14820: URL: https://github.com/apache/kafka/pull/14820#discussion_r1409866239 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -405,7 +415,7 @@ private void completeReassignmentIfNeeded() {

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on PR #14860: URL: https://github.com/apache/kafka/pull/14860#issuecomment-1832698639 > Left a few comments, nothing major directly on the changes. > > I do have a high-level thought to put out there. This currently prevents the storage tool from formatting with a

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-11-29 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1409862992 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -169,6 +174,19 @@ public NetworkClientDelegate.PollResult

Re: [PR] MINOR: DirectoryId.MIGRATING should be all zeros [kafka]

2023-11-29 Thread via GitHub
cmccabe commented on PR #14858: URL: https://github.com/apache/kafka/pull/14858#issuecomment-1832693965 > The directory fields in BrokerRegistrationRequest, RegisterBrokerRecord, BrokerRegistrationChangeRecord, PartitionRecord and PartitionChangeRecord are all of type Uuid[], which

Re: [PR] KAFKA-15886: Always specify directories for new partition registrations [kafka]

2023-11-29 Thread via GitHub
soarez commented on code in PR #14820: URL: https://github.com/apache/kafka/pull/14820#discussion_r1409860115 ## metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java: ## @@ -377,7 +386,7 @@ public ApiMessageAndVersion toRecord(Uuid topicId, int

[PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-11-29 Thread via GitHub
philipnee opened a new pull request, #14873: URL: https://github.com/apache/kafka/pull/14873 Currently, poll interval is not being respected during consumer#poll. We need to make sure the consumer can leave the group actively when the user doesn't poll frequently enough. This PR added 1.

Re: [PR] KAFKA-15910: New group coordinator needs to generate snapshots while loading [kafka]

2023-11-29 Thread via GitHub
jeffkbkim commented on code in PR #14849: URL: https://github.com/apache/kafka/pull/14849#discussion_r1409851471 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorShard.java: ## @@ -47,4 +47,11 @@ default void

Re: [PR] KAFKA-14438: Throw if async consumer configured with invalid group ID [kafka]

2023-11-29 Thread via GitHub
cadonna commented on code in PR #14872: URL: https://github.com/apache/kafka/pull/14872#discussion_r1409850375 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -752,6 +757,70 @@ public void testSubscriptionOnEmptyTopic() {

Re: [PR] KAFKA-14438: Throw if async consumer configured with invalid group ID [kafka]

2023-11-29 Thread via GitHub
cadonna commented on code in PR #14872: URL: https://github.com/apache/kafka/pull/14872#discussion_r1409848052 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -398,6 +387,38 @@ public class AsyncKafkaConsumer implements

Re: [PR] KAFKA-14438: Throw if async consumer configured with invalid group ID [kafka]

2023-11-29 Thread via GitHub
cadonna commented on PR #14872: URL: https://github.com/apache/kafka/pull/14872#issuecomment-1832675480 @kirktrue please review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id for AsyncKafkaConsumer [kafka]

2023-11-29 Thread via GitHub
cadonna commented on PR #14768: URL: https://github.com/apache/kafka/pull/14768#issuecomment-1832672849 @kirktrue My PR regarding [KAFKA-15281](https://issues.apache.org/jira/browse/KAFKA-15281) overlaps a lot with this. So I broke out the relevant parts for the group ID verification from

[PR] KAFKA-14438: Throw if async consumer configured with invalid group ID [kafka]

2023-11-29 Thread via GitHub
cadonna opened a new pull request, #14872: URL: https://github.com/apache/kafka/pull/14872 Verifies that the group ID passed into the async consumer is valid. That is, if the group ID is not null, it is not empty or it does not consist of only whitespaces. This change stores the

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
soarez commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409826262 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -207,6 +213,22 @@ public enum MetadataVersion { */ public static

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409822776 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java: ## @@ -454,4 +480,19 @@ public void shutdown() { public

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409821677 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java: ## @@ -310,6 +317,25 @@ public void run() {

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409821009 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java: ## @@ -310,6 +317,25 @@ public void run() {

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409819732 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1791,6 +1794,52 @@ protected int processStreamThread(final Consumer consumer) {

Re: [PR] MINOR: DirectoryId.MIGRATING should be all zeros [kafka]

2023-11-29 Thread via GitHub
soarez commented on PR #14858: URL: https://github.com/apache/kafka/pull/14858#issuecomment-1832632146 The directory fields in `BrokerRegistrationRequest`, `RegisterBrokerRecord`, `BrokerRegistrationChangeRecord`, `PartitionRecord` and `PartitionChangeRecord` are all of type `Uuid[]`,

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-29 Thread via GitHub
gharris1727 commented on PR #14729: URL: https://github.com/apache/kafka/pull/14729#issuecomment-1832624782 Hey @cmccabe @hachikuji Could you take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
hachikuji commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1409800961 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] Kafka-14748: Relax non-null FK left-join requirement [kafka]

2023-11-29 Thread via GitHub
wcarlson5 commented on PR #14107: URL: https://github.com/apache/kafka/pull/14107#issuecomment-1832612612 I'm going to start my review of this today. Hopefully can be actionable sometime tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409794850 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java: ## @@ -310,6 +317,25 @@ public void run() {

[jira] [Updated] (KAFKA-15816) Typos in tests leak network sockets

2023-11-29 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15816: Description: There are a few tests which leak network sockets due to small typos in the tests

Re: [PR] KAFKA-15816: Fix leaked sockets in streams tests [kafka]

2023-11-29 Thread via GitHub
gharris1727 merged PR #14769: URL: https://github.com/apache/kafka/pull/14769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15816: Fix leaked sockets in streams tests [kafka]

2023-11-29 Thread via GitHub
gharris1727 commented on PR #14769: URL: https://github.com/apache/kafka/pull/14769#issuecomment-1832605577 The streams test failures appear unrelated, as they are not subclasses of the changed tests, and all streams tests pass locally. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

2023-11-29 Thread via GitHub
AndrewJSchofield commented on code in PR #14767: URL: https://github.com/apache/kafka/pull/14767#discussion_r1409788411 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3697,16 +3698,60 @@ class KafkaApis(val requestChannel: RequestChannel,

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-11-29 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1409030872 ## streams/src/main/java/org/apache/kafka/streams/ClientInstanceIds.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
kirktrue commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1409772670 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -4387,7 +4398,52 @@ public FenceProducersResult fenceProducers(Collection

Re: [PR] KAFKA-15022: add config for balance subtopology in rack aware task assignment [kafka]

2023-11-29 Thread via GitHub
mjsax merged PR #14711: URL: https://github.com/apache/kafka/pull/14711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (KAFKA-15948) Refactor AsyncKafkaConsumer shutdown

2023-11-29 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15948: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Bug) > Refactor AsyncKafkaConsumer shutdown

[jira] [Created] (KAFKA-15948) Refactor AsyncKafkaConsumer shutdown

2023-11-29 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15948: -- Summary: Refactor AsyncKafkaConsumer shutdown Key: KAFKA-15948 URL: https://issues.apache.org/jira/browse/KAFKA-15948 Project: Kafka Issue Type: Bug

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-11-29 Thread via GitHub
jolshan commented on code in PR #14702: URL: https://github.com/apache/kafka/pull/14702#discussion_r1409765577 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -820,6 +823,37 @@ class TransactionsTest extends IntegrationTestHarness {

[jira] [Created] (KAFKA-15947) Null pointer on LZ4 compression since Kafka 3.6

2023-11-29 Thread Ludo (Jira)
Ludo created KAFKA-15947: Summary: Null pointer on LZ4 compression since Kafka 3.6 Key: KAFKA-15947 URL: https://issues.apache.org/jira/browse/KAFKA-15947 Project: Kafka Issue Type: Bug

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

2023-11-29 Thread via GitHub
rondagostino commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1409696104 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -59,10 +59,14 @@ object StorageTool extends Logging { case "format" => val

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on PR #14843: URL: https://github.com/apache/kafka/pull/14843#issuecomment-1832520113 Thanks @mjsax for reviewing, yes the code is more of wiring. @AndrewJSchofield @philipnee @kirktrue can you please help with you feedback for defining the class/method in KIP, and

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

2023-11-29 Thread via GitHub
apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1409739909 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -160,6 +164,16 @@ public

Re: [PR] MINOR: Fix flaky BrokerLifecycleManagerTest [kafka]

2023-11-29 Thread via GitHub
junrao commented on code in PR #14836: URL: https://github.com/apache/kafka/pull/14836#discussion_r1409739363 ## core/src/test/scala/unit/kafka/server/BrokerLifecycleManagerTest.scala: ## @@ -201,7 +201,7 @@ class BrokerLifecycleManagerTest { while (!future.isDone ||

[jira] [Updated] (KAFKA-15946) AsyncKafkaConsumer should retry commits on the application thread instead of autoretry

2023-11-29 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15946: --- Labels: kip-848-client-support (was: ) > AsyncKafkaConsumer should retry commits on the

  1   2   3   >