Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395236215 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395228455 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.jo

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395220263 ## core/src/main/java/kafka/metrics/ClientMetricsInstance.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395198255 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813871310 @lucasbru - I think we probably need to refactor KafkaConsumerTest again because most of the tests don't really apply to the async KafkaConsumer due to different protocol use etc. -

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395212845 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395204633 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.jo

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395204343 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -30,17 +68,366 @@ public class ClientMetricsManager implements Closeable { private sta

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395198255 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813848500 I think the test failures was probably caused by heartbeat bubbled up to the consumer subsequently throwing an exception -- This is an automated message from the Apache Git Service. T

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
kamalcph commented on PR #14755: URL: https://github.com/apache/kafka/pull/14755#issuecomment-1813827068 @satishd Addressed your review comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
kamalcph commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1395176822 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -477,20 +492,22 @@ private NewTopic createR

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395171995 ## core/src/test/java/kafka/server/ClientMetricsManagerTest.java: ## @@ -0,0 +1,921 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395166691 ## core/src/test/java/kafka/server/ClientMetricsManagerTest.java: ## @@ -0,0 +1,921 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395163631 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.jo

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395163631 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.jo

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395156877 ## core/src/test/java/kafka/metrics/ClientMetricsInstanceTest.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

[jira] [Updated] (KAFKA-13447) Consumer should not reuse committed offset after topic recreation

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-13447: --- Labels: consumer needs-kip (was: needs-kip) > Consumer should not reuse committed offset after topi

[jira] [Commented] (KAFKA-15325) Integrate topicId in OffsetFetch and OffsetCommit async consumer calls

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786609#comment-17786609 ] Philip Nee commented on KAFKA-15325: [~lianetm] a relevant ticket https://issues.apa

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395143611 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

[jira] [Assigned] (KAFKA-15839) Review topic recreation handling in client reconciliation

2023-11-15 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15839: -- Assignee: Lianet Magrans > Review topic recreation handling in client reconciliation

[jira] [Created] (KAFKA-15839) Review topic recreation handling in client reconciliation

2023-11-15 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15839: -- Summary: Review topic recreation handling in client reconciliation Key: KAFKA-15839 URL: https://issues.apache.org/jira/browse/KAFKA-15839 Project: Kafka

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395138981 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MemberState.java: ## @@ -17,57 +17,113 @@ package org.apache.kafka.clients.consumer.internals;

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395138233 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MemberState.java: ## @@ -17,57 +17,113 @@ package org.apache.kafka.clients.consumer.internals;

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395132794 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -137,39 +135,104 @@ public HeartbeatRequestManager( }

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395130865 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -137,39 +135,104 @@ public HeartbeatRequestManager( }

[jira] [Updated] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pangiawan updated KAFKA-15838: --- Description: ExtractField: Line 116-119 [https://github.com/a0x8o/kafka/blob/master/connect

[jira] [Updated] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pangiawan updated KAFKA-15838: --- Description: ExtractField: Line 116-119 [https://github.com/a0x8o/kafka/blob/master/connect

[jira] [Created] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
Eric Pangiawan created KAFKA-15838: -- Summary: [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields Key: KAFKA-15838 URL: https://issues.apache.org/jira/browse/K

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
satishd commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1395121310 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -477,20 +492,22 @@ private NewTopic createRe

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395121316 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395108554 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

[PR] [MINOR] Removing unused variables from StreamThreadTest [kafka]

2023-11-15 Thread via GitHub
vamossagar12 opened a new pull request, #14777: URL: https://github.com/apache/kafka/pull/14777 While working on https://github.com/apache/kafka/pull/13283 I found a few variables which aren't being used anymore but still exist. I believe this is happening due to some recent refactoring tha

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395108554 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

Re: [PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-15 Thread via GitHub
satishd merged PR #14759: URL: https://github.com/apache/kafka/pull/14759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-15 Thread via GitHub
satishd commented on PR #14759: URL: https://github.com/apache/kafka/pull/14759#issuecomment-1813729407 There are a few unrelated test failures, merging it to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2023-11-15 Thread via GitHub
vamossagar12 commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-1813728802 hey @ableegoldman , quick question. I am noticing that tests like `shouldRecoverFromInvalidOffsetExceptionOnRestoreAndFinishRestore` are failing with an error like ``` Exce

[jira] [Commented] (KAFKA-15821) Active topics for deleted connectors are not reset in standalone mode

2023-11-15 Thread David Jameson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786575#comment-17786575 ] David Jameson commented on KAFKA-15821: --- Had a go at this one because it looked li

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-15 Thread via GitHub
jeffkbkim commented on PR #14387: URL: https://github.com/apache/kafka/pull/14387#issuecomment-1813664706 @jolshan the other test failures look unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-15460: Add group type filter to the List Groups API [kafka]

2023-11-15 Thread via GitHub
rreddy-22 opened a new pull request, #14775: URL: https://github.com/apache/kafka/pull/14775 KIP-848 introduces the concept of "group types" in the context of the ListGroups API. The ListGroups API has been extended to support both filtering on the group types and returning the group types

Re: [PR] PROPOSAL: support async event execution in group coordinator [kafka]

2023-11-15 Thread via GitHub
artemlivshits commented on code in PR #14705: URL: https://github.com/apache/kafka/pull/14705#discussion_r1395062167 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -659,9 +660,21 @@ public TopicPartition key() {

Re: [PR] WIP KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-11-15 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1395060812 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -469,19 +469,35 @@ class GroupMetadataManager(brokerId: Int, responseCallb

[PR] WIP KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-11-15 Thread via GitHub
jolshan opened a new pull request, #14774: URL: https://github.com/apache/kafka/pull/14774 Rewrote the verification flow to have 3 stages 1. collect the partitions that need verification 2. with the callback result separate results into errors and verified partitions 3. Perform the action w

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394884307 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config: KafkaConfig,

[jira] [Resolved] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-11-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15346. - Fix Version/s: 3.7.0 Resolution: Fixed > Single-Key_single-timestamp IQs with ver

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-15 Thread via GitHub
mjsax commented on PR #14596: URL: https://github.com/apache/kafka/pull/14596#issuecomment-1813605452 Merged to `trunk`. Congrats to your first PR and KIP! 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-15 Thread via GitHub
mjsax merged PR #14596: URL: https://github.com/apache/kafka/pull/14596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Commented] (KAFKA-15834) Subscribing to non-existent topic blocks StreamThread from stopping

2023-11-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786564#comment-17786564 ] Matthias J. Sax commented on KAFKA-15834: - Thank for reporting and such a detail

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15836: Fix Version/s: 3.6.1 > KafkaConsumer subscribes to multiple topics does not respect max.poll.recor

[PR] KAFKA-15837: Throw error on use of Consumer.poll(long timeout) [kafka]

2023-11-15 Thread via GitHub
kirktrue opened a new pull request, #14773: URL: https://github.com/apache/kafka/pull/14773 Per KIP-266, the `Consumer.poll(long timeout)` method was deprecated in 2.0.0. The default implementation of `Consumer.poll(long timeout)` now throws a `KafkaException`, and the known implementations

Re: [PR] PROPOSAL: support async event execution in group coordinator [kafka]

2023-11-15 Thread via GitHub
junrao commented on code in PR #14705: URL: https://github.com/apache/kafka/pull/14705#discussion_r1395031179 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -659,9 +660,21 @@ public TopicPartition key() { */

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1395024903 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

[jira] [Created] (KAFKA-15837) Throw error on use of Consumer.poll(long timeout)

2023-11-15 Thread Kirk True (Jira)
Kirk True created KAFKA-15837: - Summary: Throw error on use of Consumer.poll(long timeout) Key: KAFKA-15837 URL: https://issues.apache.org/jira/browse/KAFKA-15837 Project: Kafka Issue Type: Impro

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue commented on code in PR #14768: URL: https://github.com/apache/kafka/pull/14768#discussion_r1395013498 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -150,11 +150,9 @@ public class LegacyKafkaConsumer implements Con

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15836: --- Labels: consumer (was: ) > KafkaConsumer subscribes to multiple topics does not respect max.poll.re

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15836: --- Summary: KafkaConsumer subscribes to multiple topics does not respect max.poll.records (was: KafkaC

[jira] [Created] (KAFKA-15836) KafkaConsumer subscribe to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15836: -- Summary: KafkaConsumer subscribe to multiple topics does not respect max.poll.records Key: KAFKA-15836 URL: https://issues.apache.org/jira/browse/KAFKA-15836 Project: Kaf

[PR] KafkaConsumer not respecting max.poll.record when subscribing to more than 1 topics. [kafka]

2023-11-15 Thread via GitHub
philipnee opened a new pull request, #14772: URL: https://github.com/apache/kafka/pull/14772 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testi

[PR] KAFKA-15816: Fix leaked sockets in trogdor tests [kafka]

2023-11-15 Thread via GitHub
gharris1727 opened a new pull request, #14771: URL: https://github.com/apache/kafka/pull/14771 The AgentTest leaks the agent under test due to typos. Follow the existing style in the test to stop it. ### Committer Checklist (excluded from commit message) - [ ] Verify design and imp

[jira] [Comment Edited] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786537#comment-17786537 ] Phuc Hong Tran edited comment on KAFKA-15341 at 11/15/23 11:26 PM: ---

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-15 Thread via GitHub
scholzj commented on code in PR #14756: URL: https://github.com/apache/kafka/pull/14756#discussion_r1394966608 ## clients/src/main/java/org/apache/kafka/common/security/ssl/SslFactory.java: ## @@ -185,6 +188,18 @@ private SslEngineFactory createNewSslEngineFactory(Map newConfig

[jira] [Comment Edited] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786537#comment-17786537 ] Phuc Hong Tran edited comment on KAFKA-15341 at 11/15/23 11:09 PM: ---

Re: [PR] KAFKA-15826: Close consumer when sink task is cancelled [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on PR #14762: URL: https://github.com/apache/kafka/pull/14762#issuecomment-1813403690 The only problem I have with this approach is that the [official documentation](https://kafka.apache.org/36/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#multithreaded)

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-15 Thread via GitHub
soarez commented on PR #14770: URL: https://github.com/apache/kafka/pull/14770#issuecomment-1813400599 @pprovenzano @cmccabe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-15 Thread via GitHub
soarez opened a new pull request, #14770: URL: https://github.com/apache/kafka/pull/14770 Instead of only sending failed log directory UUIDs in the heartbeat request until a successful response is received, the broker sends the full cumulative set of failed directories since startup time.

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-15 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1394939779 ## streams/src/main/java/org/apache/kafka/streams/TopologyConfig.java: ## @@ -216,12 +232,38 @@ public TopologyConfig(final String topologyName, final StreamsConfig glo

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813394707 I rebased the trunk and was unable to reproduce the failure locally. I wonder if the tests is flaky and was caused by the delegate PR. -- This is an automated message from the Apache

[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786537#comment-17786537 ] Phuc Hong Tran commented on KAFKA-15341: thanks [~divijvaidya] [~showuon] [~sati

[PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee opened a new pull request, #14680: URL: https://github.com/apache/kafka/pull/14680 The PR covers a few important points: 1. Exception handling: We should be thrown RetriableCommitException when the commit exception is retriable. We should throw FencedIdException on commit and

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee closed pull request #14680: KAFKA-15174: Ensure CommitAsync propagate the exception to the user URL: https://github.com/apache/kafka/pull/14680 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] KAFKA-15816: Fix leaked sockets in streams tests [kafka]

2023-11-15 Thread via GitHub
gharris1727 opened a new pull request, #14769: URL: https://github.com/apache/kafka/pull/14769 These tests leak sockets via Kafka clients, either directly or via KafkaStreams instances. The tests should close these resources when appropriate. ### Committer Checklist (excluded from com

Re: [PR] [Draft] KIP-994 (Part 1) Minor Enhancements to ListTransactionsRequest [kafka]

2023-11-15 Thread via GitHub
artemlivshits commented on code in PR #14731: URL: https://github.com/apache/kafka/pull/14731#discussion_r1394891315 ## core/src/main/scala/kafka/coordinator/transaction/TransactionStateManager.scala: ## @@ -327,6 +328,7 @@ class TransactionStateManager(brokerId: Int, }

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394886188 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted TopicPartitions

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394884307 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config: KafkaConfig,

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813338012 hm - why is it sending heartbeat? looking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
lucasbru commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813330936 @philipnee there is a suspicious test failure for `org.apache.kafka.clients.consumer.KafkaConsumerTest`, could you have a look? -- This is an automated message from the Apache Git Serv

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394832061 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted TopicPartitions

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394828896 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted TopicPartitions

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394827643 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14768: URL: https://github.com/apache/kafka/pull/14768#discussion_r1394821601 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -150,11 +150,9 @@ public class LegacyKafkaConsumer impleme

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394820962 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394818560 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config: KafkaConfig,

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue commented on PR #14768: URL: https://github.com/apache/kafka/pull/14768#issuecomment-1813279513 @philipnee—can you add the `ctr` and `KIP-848` labels? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue opened a new pull request, #14768: URL: https://github.com/apache/kafka/pull/14768 Per [KIP-289 (Improve the default group id behavior in `KafkaConsumer`)](https://cwiki.apache.org/confluence/display/KAFKA/KIP-289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer), the `gro

[jira] [Updated] (KAFKA-14438) Throw error when consumer configured with empty/whitespace-only group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Throw error when consumer configured with empty/whitespace-only group.id (was: Throw Invalid

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394789418 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394786770 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

[jira] [Assigned] (KAFKA-15835) Group commit/callbacks triggering logic

2023-11-15 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15835: -- Assignee: Lianet Magrans > Group commit/callbacks triggering logic >

[jira] [Created] (KAFKA-15835) Group commit/callbacks triggering logic

2023-11-15 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15835: -- Summary: Group commit/callbacks triggering logic Key: KAFKA-15835 URL: https://issues.apache.org/jira/browse/KAFKA-15835 Project: Kafka Issue Type: Sub-t

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394774045 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394759427 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -168,10 +308,37 @@ public void updateState(ConsumerGroupHeartbea

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394753399 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -77,32 +138,111 @@ public class MembershipManagerImpl implements

[jira] [Updated] (KAFKA-14438) Throw InvalidGroupIdException when consumer configured with empty/whitespace-only group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Throw InvalidGroupIdException when consumer configured with empty/whitespace-only group.id (

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Labels: kip-848-client-support kip-848-e2e kip-848-preview (was: ) > Stop supporting empty consumer g

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Description: Currently, a warning message is logged upon using an empty consumer groupId. In the next

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394746271 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Stop supporting empty consumer group.id (was: Stop supporting empty consumer groupId) > Sto

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1394744738 ## core/src/main/java/kafka/metrics/ClientMetricsConfigs.java: ## @@ -80,6 +82,11 @@ public class ClientMetricsConfigs { public static final String CLIENT

[jira] [Assigned] (KAFKA-14438) Stop supporting empty consumer groupId

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-14438: - Assignee: Kirk True > Stop supporting empty consumer groupId >

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394742401 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void updateState(ConsumerGroupHeartbe

  1   2   >