[PR] Add explicit exit call to each shell command [kafka]

2023-10-25 Thread via GitHub
tkornai opened a new pull request, #14633: URL: https://github.com/apache/kafka/pull/14633 Using a custom security.provider may keep shell commands running indefinitely if System.exit() is not called explicitly. This behaviour is due to the reason that a security.provider implementation

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
ppatierno commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1371736949 ## docs/ops.html: ## @@ -3822,6 +3822,9 @@ Provisioning the KRaft controller quorum # The inter broker listener in brokers to allow KRaft controller send RPCs to

[PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock opened a new pull request, #14635: URL: https://github.com/apache/kafka/pull/14635 KIP-848 introduced this check, but we have seen since upgrading to broker 3.4 with both Sarama and Librdkafka clients when issuing LeaveGroup and getting an error, the single member gets lost

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779425711 > @wolfchimneyrock Thanks for the PR. I am not to fully understand the issue yet. Could you please elaborate a bit more about the condition leading to it? there is more

[jira] [Created] (KAFKA-15684) Add support to describe all subscriptions through utility

2023-10-25 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15684: - Summary: Add support to describe all subscriptions through utility Key: KAFKA-15684 URL: https://issues.apache.org/jira/browse/KAFKA-15684 Project: Kafka

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on PR #14564: URL: https://github.com/apache/kafka/pull/14564#issuecomment-1779551983 @AndrewJSchofield & @kirktrue addressed the feedback so far! thanks for the reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue opened a new pull request, #14638: URL: https://github.com/apache/kafka/pull/14638 Straightforward refactoring to extract an inner class and methods related to `ConsumerRebalanceListener` for reuse in the KIP-848 implementation of the consumer group protocol. Also using `Optional`

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] KAFKA-15632: Drop the invalid remote log metadata events [kafka]

2023-10-25 Thread via GitHub
kamalcph commented on code in PR #14576: URL: https://github.com/apache/kafka/pull/14576#discussion_r1371713733 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataCache.java: ## @@ -157,13 +157,20 @@ public void

Re: [PR] KAFKA-15598 & KAFKA-15461: Add integration tests for DescribeGroups API, DeleteGroups API, OffsetDelete API and ListGroups API [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1371838869 ## core/src/test/scala/unit/kafka/server/DescribeGroupsRequestTest.scala: ## @@ -0,0 +1,118 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779404361 > If we ignore producer-state-flush failure here, recovery-point might be incremented even with stale on-disk producer state snapshot. So, in case of restart after power failure, the

[jira] [Commented] (KAFKA-15200) verify pre-requisite at start of release.py

2023-10-25 Thread Owen C.H. Leung (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779537#comment-17779537 ] Owen C.H. Leung commented on KAFKA-15200: - [~divijvaidya] I've created PR for this issue. Can I

Re: [PR] Kafka 15680 [kafka]

2023-10-25 Thread via GitHub
C0urante commented on PR #14630: URL: https://github.com/apache/kafka/pull/14630#issuecomment-1779644800 @kumarpritam863 is this the same as https://github.com/apache/kafka/pull/12622 and https://issues.apache.org/jira/browse/KAFKA-14220? -- This is an automated message from the Apache

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372046033 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on PR #14565: URL: https://github.com/apache/kafka/pull/14565#issuecomment-1779699978 This is now outdated. Please see #14638. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Created] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15682: Summary: Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments Key: KAFKA-15682 URL:

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1779271622 Build depends on PR - https://github.com/apache/kafka/pull/14621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] Add pre-requisite check in release.py [kafka]

2023-10-25 Thread via GitHub
Owen-CH-Leung opened a new pull request, #14636: URL: https://github.com/apache/kafka/pull/14636 As per the request in [KAFKA-15200](https://issues.apache.org/jira/browse/KAFKA-15200), This PR adds pre-requitsite-check in the `release.py` before the release process begins ###

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372045729 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372047546 ## core/src/main/scala/kafka/server/ZkConfigManager.scala: ## @@ -38,7 +38,8 @@ object ConfigType { val User = "users" val Broker = "brokers" val Ip =

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372064203 ## core/src/main/scala/kafka/server/BrokerLifecycleManager.scala: ## @@ -190,7 +192,7 @@ class BrokerLifecycleManager( /** * The broker epoch from the

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372063969 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -85,17 +87,16 @@ void write(long brokerEpoch, int

[PR] MINOR: Fix UT stubbing [kafka]

2023-10-25 Thread via GitHub
iamlockon opened a new pull request, #14637: URL: https://github.com/apache/kafka/pull/14637 *More detailed description of your change, Originally the assert and stub statements are not verifying correctly, so `assertEquals` will always be true. I have checked the whole file and fixed

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372044581 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -2619,6 +2619,9 @@ private ConfigEntry.ConfigSource

[jira] [Commented] (KAFKA-15564) Kafka 3.5.1- Mirror Maker 2 replicating the wrong consumer group offsets in destination

2023-10-25 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779568#comment-17779568 ] Greg Harris commented on KAFKA-15564: - [~hemanthsavasere] Were you able to work around this problem

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372045362 ## core/src/main/scala/kafka/metrics/ClientMetricsConfig.scala: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on PR #14638: URL: https://github.com/apache/kafka/pull/14638#issuecomment-1779709409 @dajac @philipnee Can you add the `ctr` and `KIP-848` tags, please? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
rondagostino commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371865719 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -54,6 +54,7 @@ public static boolean

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14410: URL: https://github.com/apache/kafka/pull/14410#issuecomment-1779407230 > @divijvaidya Do you have cycles to review this? Sure, I can get to this before the end this week. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
dajac commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779417281 @wolfchimneyrock Thanks for the PR. I am not to fully understand the issue yet. Could you please elaborate a bit more about the condition leading to it? -- This is an automated message

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372017036 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -73,10 +77,8 @@ void write(long brokerEpoch, int version)

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372046737 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372047782 ## core/src/test/scala/kafka/metrics/ClientMetricsTestUtils.scala: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on code in PR #14410: URL: https://github.com/apache/kafka/pull/14410#discussion_r1371770193 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsProducerTest.java: ## @@ -197,8 +203,7 @@ public void before() { );

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on PR #14410: URL: https://github.com/apache/kafka/pull/14410#issuecomment-1779286800 @divijvaidya Do you have cycles to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779515287 Here is a broker trace: ``` [2023-10-24 01:17:17,214] ERROR [KafkaApi-28598] Unexpected error handling request RequestHeader(apiKey=LEAVE_GROUP, apiVersion=1,

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779439685 I'm not sure I can elaborate on what the underlying error response that would have been sent with the LeaveGroup is, since the current code raises that exception without

Re: [PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-25 Thread via GitHub
gnarula commented on code in PR #14625: URL: https://github.com/apache/kafka/pull/14625#discussion_r1372058951 ## clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java: ## @@ -63,15 +61,9 @@ import static org.mockito.Mockito.when; public class

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1371818152 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -310,16 +313,22 @@ public CompletableFuture joinGroup(

[jira] [Created] (KAFKA-15683) Delete subscription from metadata when all configs are deleted

2023-10-25 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15683: - Summary: Delete subscription from metadata when all configs are deleted Key: KAFKA-15683 URL: https://issues.apache.org/jira/browse/KAFKA-15683 Project: Kafka

Re: [PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14625: URL: https://github.com/apache/kafka/pull/14625#discussion_r1372035225 ## clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java: ## @@ -63,15 +61,9 @@ import static org.mockito.Mockito.when; public class

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-25 Thread via GitHub
kirktrue closed pull request #14357: KAFKA-15276: Implement partition assignment reconciliation URL: https://github.com/apache/kafka/pull/14357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue closed pull request #14565: KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse URL: https://github.com/apache/kafka/pull/14565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (KAFKA-15680) Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing

2023-10-25 Thread Pritam Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritam Kumar updated KAFKA-15680: - Fix Version/s: 3.6.1 > Partition-Count is not getting updated Correctly in the Incremental >

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1371720108 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
showuon commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1371720987 ## docs/ops.html: ## @@ -3819,6 +3819,9 @@ Provisioning the KRaft controller quorum # ZooKeeper client configuration zookeeper.connect=localhost:2181 +# The inter

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on PR #12728: URL: https://github.com/apache/kafka/pull/12728#issuecomment-1779282256 @mimaison Do you have the cycles to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-15348: add support for 'range IQs with versioned state stores (KIP-969) [kafka]

2023-10-25 Thread via GitHub
aliehsaeedii opened a new pull request, #14634: URL: https://github.com/apache/kafka/pull/14634 This is the PoC PR for KIP-969. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
rondagostino commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371825053 ## metadata/src/main/resources/common/metadata/PartitionRecord.json: ## @@ -47,6 +47,13 @@ "about": "The eligible leader replicas of this partition." },

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1371749943 ## metadata/src/main/java/org/apache/kafka/metadata/properties/MetaProperties.java: ## @@ -0,0 +1,266 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
pprovenzano commented on PR #14578: URL: https://github.com/apache/kafka/pull/14578#issuecomment-1779628602 Closing this PR in favor of https://github.com/apache/kafka/pull/14628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
pprovenzano closed pull request #14578: KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. URL: https://github.com/apache/kafka/pull/14578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add explicit exit call to each shell command [kafka]

2023-10-25 Thread via GitHub
gharris1727 commented on PR #14633: URL: https://github.com/apache/kafka/pull/14633#issuecomment-1779677563 Hi @tkornai thanks for the PR! I think it's a relatively common expectation that returning from `main` will exit the program, and this problem is likely to regress in the

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372064089 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372115691 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -162,13 +162,13 @@ else if (this.subscriptionType != type)

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372124366 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-10-25 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779623#comment-17779623 ] Arpit Goyal commented on KAFKA-15388: - Thanks [~divijvaidya] for providing the initial steps

Re: [PR] Kafka 12317: Relax non-null key requirement in Kafka Streams [kafka]

2023-10-25 Thread via GitHub
florin-akermann commented on code in PR #14174: URL: https://github.com/apache/kafka/pull/14174#discussion_r1372233874 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -351,8 +353,23 @@ private void optimizeTopology(final

[PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-25 Thread via GitHub
philipnee opened a new pull request, #14639: URL: https://github.com/apache/kafka/pull/14639 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-25 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1372301334 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -776,64 +779,55 @@ public void verifyStore() {

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372308747 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1372371671 ## clients/src/main/java/org/apache/kafka/common/telemetry/package-info.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] MINOR: Push down logic from TransactionManager to TxnPartitionEntry [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14591: URL: https://github.com/apache/kafka/pull/14591#issuecomment-1780126154 Sorry I lost track of this. I will take a look soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372377851 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779818679 Thanks -- just catching up with the discussion. Just to clarify when we say: > This is ok for server restart, because on restart, we will rebuild the snapshot by scanning last few

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372169588 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

[PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-25 Thread via GitHub
kirktrue opened a new pull request, #14640: URL: https://github.com/apache/kafka/pull/14640 Added logic for reconciling partition assignment between the target assignment provided by the group coordinator and the current assignment as specified in the SubscriptionState. This

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372384540 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -253,17 +269,30 @@ private ElectionResult electAnyLeader() {

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
kirktrue commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372381836 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372379141 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372117884 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -285,10 +285,8 @@ public synchronized void

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372118635 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -162,13 +162,13 @@ else if (this.subscriptionType != type)

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1364689270 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1617,10 +1617,16 @@ class UnifiedLog(@volatile var logStartOffset: Long, // may actually be ahead of the

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

[jira] [Commented] (KAFKA-15474) AbstractCoordinator.testWakeupAfterSyncGroupReceivedExternalCompletion seems flaky

2023-10-25 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779667#comment-17779667 ] Philip Nee commented on KAFKA-15474: {code:java} [2023-10-25 13:27:51,171] ERROR JoinGroup failed:

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
mumrah commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372350163 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -253,17 +269,30 @@ private ElectionResult electAnyLeader() { return

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on PR #14575: URL: https://github.com/apache/kafka/pull/14575#issuecomment-1780137826 @junrao @mjsax @wcarlson5 The following tests are failing which are independent of changes in the PR. ``` testTrustStoreAlter(String).quorum=kraft –

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-25 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1372384881 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -361,11 +410,16 @@ public Optional build() {

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
jolshan commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1780155389 Hey there. If this is KAFKA-15661, then the title of the PR should be `KAFKA-15661: KIP-951: Server side and protocol changes` -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372104074 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372114475 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1372379598 ## clients/src/main/java/org/apache/kafka/common/telemetry/package-info.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1372120378 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/metrics/StreamsMetricsImplTest.java: ## @@ -1293,43 +1204,33 @@ private void

Re: [PR] Add pre-requisite check in release.py [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14636: URL: https://github.com/apache/kafka/pull/14636#discussion_r1372147994 ## release.py: ## @@ -491,6 +491,25 @@ def command_release_announcement_email(): Do you have all of of these setup? (y/n): """ % (PREFS_FILE, json.dumps(prefs,

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372148618 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on code in PR #14638: URL: https://github.com/apache/kafka/pull/14638#discussion_r1372112998 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -1081,4 +1026,74 @@ boolean

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372156380 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372270435 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -846,7 +759,8 @@ class ReplicaManager(val config: KafkaConfig, producerId =

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-25 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1372329928 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-25 Thread via GitHub
philipnee commented on PR #14638: URL: https://github.com/apache/kafka/pull/14638#issuecomment-1779768168 Hi @kirktrue - I made a pass and left some comments. Let me know if anything is unclear! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372146226 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1372147456 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

  1   2   3   >