Re: [PR] MINOR: Minor update patch versions [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14641: URL: https://github.com/apache/kafka/pull/14641#issuecomment-1780451891 @satishd or @showuon -- did we notice these versions causing issues for the system tests for the 3.6 release? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-25 Thread via GitHub
philipnee commented on PR #14642: URL: https://github.com/apache/kafka/pull/14642#issuecomment-1780387305 Apparently, it is not very easy to validate if the server-side assignor is used with the local assignor option instantiating the config. We should validate this when starting up the

[PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-25 Thread via GitHub
philipnee opened a new pull request, #14642: URL: https://github.com/apache/kafka/pull/14642 Protocol Name | Type | Default | Description -- | -- | -- | -- group.protocol | enum | generic | A flag which indicates if the new protocol should be used or not. It could be: generic or

[jira] [Commented] (KAFKA-14419) Same message consumed again by the same stream task after partition is lost and reassigned

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779731#comment-17779731 ] A. Sophie Blee-Goldman commented on KAFKA-14419: Hey, sorry for the long delay – I'm

Re: [PR] KAFKA-15602: revert KAFKA-4852 [kafka]

2023-10-25 Thread via GitHub
mjsax commented on PR #14617: URL: https://github.com/apache/kafka/pull/14617#issuecomment-1780343175 @showuon Considering this for 3.5.2 bug-fix release. (Given that we wait for RocksDB testing to complete -- we are on it -- we could squeeze this one in, too?). Also \cc

Re: [PR] KAFKA-15602: revert KAFKA-4852 [kafka]

2023-10-25 Thread via GitHub
mjsax commented on PR #14617: URL: https://github.com/apache/kafka/pull/14617#issuecomment-1780342224 I would focusing on reverting -- if we want to update tests, we can do it in a separate PR? -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Resolved] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-15116. Resolution: Not A Problem > Kafka Streams processing blocked during

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779725#comment-17779725 ] A. Sophie Blee-Goldman commented on KAFKA-15116: I tend to agree with Matthias here –

[jira] [Commented] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-25 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779721#comment-17779721 ] Philip Nee commented on KAFKA-15562: Hi [~grigorii] - I haven't had a chance to find a ticket for

Re: [PR] MINOR: Fix NPE during fetchSnapshot [kafka]

2023-10-25 Thread via GitHub
hudeqi commented on code in PR #14615: URL: https://github.com/apache/kafka/pull/14615#discussion_r1372501784 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -595,7 +595,7 @@ public void deleteSnapshotsBefore(long offset) throws

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-25 Thread via GitHub
pprovenzano commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1372498997 ## metadata/src/main/java/org/apache/kafka/metadata/properties/PropertiesUtils.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Resolved] (KAFKA-12550) Introduce RESTORING state to the KafkaStreams FSM

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12550. Resolution: Won't Fix Closing this out since it's usefulness is preempted

[jira] [Updated] (KAFKA-12550) Introduce RESTORING state to the KafkaStreams FSM

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12550: --- Fix Version/s: (was: 4.0.0) > Introduce RESTORING state to the

[jira] [Resolved] (KAFKA-15463) StreamsException: Accessing from an unknown node

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-15463. Resolution: Not A Problem > StreamsException: Accessing from an unknown

[jira] [Commented] (KAFKA-15463) StreamsException: Accessing from an unknown node

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779715#comment-17779715 ] A. Sophie Blee-Goldman commented on KAFKA-15463: [~yevsh]  Well the intended pattern is

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372479451 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372479451 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15563: Provide informative error messages when Connect REST requests time out [kafka]

2023-10-25 Thread via GitHub
C0urante commented on PR #14562: URL: https://github.com/apache/kafka/pull/14562#issuecomment-1780284540 Okay, I've pushed a couple new commits that: - Introduce the notion of a completion time for a callback stage - Add granularity to the callback stages for the distributed herder's

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372471968 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372471968 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on PR #14621: URL: https://github.com/apache/kafka/pull/14621#issuecomment-1780279268 Hi @apoorvmittal10 Thanks for taking my suggestions. I left a few more, mostly about the aesthetic aspect as scala is a pretty idiomatic language... Terribly sorry that I don't

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372471968 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372467166 ## core/src/test/scala/kafka/metrics/ClientMetricsTestUtils.scala: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
ocadaruma commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1372463913 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1617,10 +1617,16 @@ class UnifiedLog(@volatile var logStartOffset: Long, // may actually be ahead of

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-25 Thread via GitHub
dongnuo123 commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1372463509 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -268,7 +269,9 @@ public CompletableFuture

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372460806 ## core/src/main/scala/kafka/metrics/ClientMetricsConfig.scala: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372457923 ## core/src/main/scala/kafka/metrics/ClientMetricsConfig.scala: ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372452303 ## core/src/main/scala/kafka/server/ClientMetricsManager.scala: ## @@ -0,0 +1,38 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372451618 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,119 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[PR] MINOR: Minor update patch versions [kafka]

2023-10-25 Thread via GitHub
jolshan opened a new pull request, #14641: URL: https://github.com/apache/kafka/pull/14641 We were missing the versions for these patch releases. The versions are in versions.py and Dockerfile. ### Committer Checklist (excluded from commit message) - [ ] Verify design and

Re: [PR] KAFKA-15468: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2023-10-25 Thread via GitHub
jsancio commented on PR #14489: URL: https://github.com/apache/kafka/pull/14489#issuecomment-1780207833 @jolshan I started a new build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372414818 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372414239 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

[jira] [Commented] (KAFKA-13152) Replace "buffered.records.per.partition" & "cache.max.bytes.buffering" with "{statestore.cache}/{input.buffer}.max.bytes"

2023-10-25 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779695#comment-17779695 ] A. Sophie Blee-Goldman commented on KAFKA-13152: Hey [~sagarrao] – I'm happy to try and

Re: [PR] KAFKA-14484: [1/N] Move PartitionMetadataFile to storage module [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14607: URL: https://github.com/apache/kafka/pull/14607#discussion_r1372393891 ## storage/src/main/java/org/apache/kafka/storage/internals/log/PartitionMetadataReadBuffer.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1372394072 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -360,7 +360,9 @@ public short partitionRecordVersion() { } public

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1372393724 ## clients/src/main/resources/common/message/FetchRequest.json: ## @@ -53,7 +53,9 @@ // // Version 15 adds the ReplicaState which includes new field ReplicaEpoch

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1372392217 ## clients/src/main/resources/common/message/FetchResponse.json: ## @@ -102,6 +104,15 @@ "about": "The preferred read replica for the consumer to use on its

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1372391952 ## clients/src/main/resources/common/message/FetchResponse.json: ## @@ -45,7 +45,9 @@ // Version 14 is the same as version 13 but it also receives a new error

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1780155389 Hey there. If this is KAFKA-15661, then the title of the PR should be `KAFKA-15661: KIP-951: Server side and protocol changes` -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372384881 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -361,11 +410,16 @@ public Optional build() {

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372384540 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -253,17 +269,30 @@ private ElectionResult electAnyLeader() {

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372381836 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372379141 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on PR #14575: URL: https://github.com/apache/kafka/pull/14575#issuecomment-1780137826 @junrao @mjsax @wcarlson5 The following tests are failing which are independent of changes in the PR. ``` testTrustStoreAlter(String).quorum=kraft –

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1372379598 ## clients/src/main/java/org/apache/kafka/common/telemetry/package-info.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372377851 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372376795 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This

Re: [PR] MINOR: Push down logic from TransactionManager to TxnPartitionEntry [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14591: URL: https://github.com/apache/kafka/pull/14591#issuecomment-1780126154 Sorry I lost track of this. I will take a look soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1372371671 ## clients/src/main/java/org/apache/kafka/common/telemetry/package-info.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-25 Thread via GitHub
kirktrue opened a new pull request, #14640: URL: https://github.com/apache/kafka/pull/14640 Added logic for reconciling partition assignment between the target assignment provided by the group coordinator and the current assignment as specified in the SubscriptionState. This

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372366141 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
mumrah commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372350163 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -253,17 +269,30 @@ private ElectionResult electAnyLeader() { return

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372329928 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372322049 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372308747 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-25 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1372301334 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -776,64 +779,55 @@ public void verifyStore() {

[jira] [Commented] (KAFKA-15474) AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky

2023-10-25 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779667#comment-17779667 ] Philip Nee commented on KAFKA-15474: {code:java} [2023-10-25 13:27:51,171] ERROR JoinGroup failed:

[PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-25 Thread via GitHub
philipnee opened a new pull request, #14639: URL: https://github.com/apache/kafka/pull/14639 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372270435 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] Kafka 12317: Relax non-null key requirement in Kafka Streams [kafka]

2023-10-25 Thread via GitHub
florin-akermann commented on code in PR #14174: URL: https://github.com/apache/kafka/pull/14174#discussion_r1372233874 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -351,8 +353,23 @@ private void optimizeTopology(final

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372169588 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372169588 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-10-25 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779623#comment-17779623 ] Arpit Goyal commented on KAFKA-15388: - Thanks [~divijvaidya] for providing the initial steps

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779818679 Thanks -- just catching up with the discussion. Just to clarify when we say: > This is ok for server restart, because on restart, we will rebuild the snapshot by scanning last few

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] Add pre-requisite check in release.py [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14636: URL: https://github.com/apache/kafka/pull/14636#discussion_r1372147994 ## release.py: ## @@ -491,6 +491,25 @@ def command_release_announcement_email(): Do you have all of of these setup? (y/n): """ % (PREFS_FILE, json.dumps(prefs,

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372147456 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1364689270 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1617,10 +1617,16 @@ class UnifiedLog(@volatile var logStartOffset: Long, // may actually be ahead of the

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1372120378 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/metrics/StreamsMetricsImplTest.java: ## @@ -1293,43 +1204,33 @@ private void

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372124366 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on PR #14638: URL: https://github.com/apache/kafka/pull/14638#issuecomment-1779768168 Hi @kirktrue - I made a pass and left some comments. Let me know if anything is unclear! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372118635 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -162,13 +162,13 @@ else if (this.subscriptionType != type)

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372117884 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -285,10 +285,8 @@ public synchronized void

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372115691 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -162,13 +162,13 @@ else if (this.subscriptionType != type)

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372114475 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372112998 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -1081,4 +1026,74 @@ boolean

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372104074 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372064089 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on PR #14638: URL: https://github.com/apache/kafka/pull/14638#issuecomment-1779709409 @dajac @philipnee Can you add the `ctr` and `KIP-848` tags, please? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue opened a new pull request, #14638: URL: https://github.com/apache/kafka/pull/14638 Straightforward refactoring to extract an inner class and methods related to `ConsumerRebalanceListener` for reuse in the KIP-848 implementation of the consumer group protocol. Also using `Optional`

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on PR #14565: URL: https://github.com/apache/kafka/pull/14565#issuecomment-1779699978 This is now outdated. Please see #14638. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (KAFKA-15680) Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing

2023-10-25 Thread Pritam Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritam Kumar updated KAFKA-15680: - Fix Version/s: 3.6.1 > Partition-Count is not getting updated Correctly in the Incremental >

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372064203 ## core/src/main/scala/kafka/server/BrokerLifecycleManager.scala: ## @@ -190,7 +192,7 @@ class BrokerLifecycleManager( /** * The broker epoch from the

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372063969 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -85,17 +87,16 @@ void write(long brokerEpoch, int

Re: [PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-25 Thread via GitHub
gnarula commented on code in PR #14625: URL: https://github.com/apache/kafka/pull/14625#discussion_r1372058951 ## clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java: ## @@ -63,15 +61,9 @@ import static org.mockito.Mockito.when; public class

Re: [PR] Add explicit exit call to each shell command [kafka]

2023-10-25 Thread via GitHub
gharris1727 commented on PR #14633: URL: https://github.com/apache/kafka/pull/14633#issuecomment-1779677563 Hi @tkornai thanks for the PR! I think it's a relatively common expectation that returning from `main` will exit the program, and this problem is likely to regress in the

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue closed pull request #14565: KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse URL: https://github.com/apache/kafka/pull/14565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-25 Thread via GitHub
kirktrue closed pull request #14357: KAFKA-15276: Implement partition assignment reconciliation URL: https://github.com/apache/kafka/pull/14357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372047782 ## core/src/test/scala/kafka/metrics/ClientMetricsTestUtils.scala: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372046737 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372045729 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

  1   2   3   >