Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1357250930 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1994,6 +2014,24 @@ private void listReassigningTopic(ListPartition

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1357251569 ## metadata/src/main/resources/common/metadata/PartitionChangeRecord.json: ## @@ -40,6 +42,15 @@ "versions": "0+", "nullableVersions": "0+", "taggedVersi

[jira] [Updated] (KAFKA-15562) CommitRequestManager needs to test different error handling

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15562: -- Component/s: clients > CommitRequestManager needs to test different error handling > -

[jira] [Updated] (KAFKA-15548) Handling close() properly

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15548: -- Component/s: clients > Handling close() properly > - > > Key:

[jira] [Updated] (KAFKA-15475) Timeout request might retry forever even if the user API times out in PrototypeAsyncConsumer

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15475: -- Component/s: clients > Timeout request might retry forever even if the user API times out in > Protot

[jira] [Updated] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15592: -- Component/s: clients > Member does not need to always try to join a group when a groupId is > configu

[jira] [Updated] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15531: -- Component/s: clients > Ensure coordinator node is removed upon disconnection exception > -

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1357248280 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1994,6 +2014,24 @@ private void listReassigningTopic(ListPartition

[jira] [Updated] (KAFKA-15321) Document consumer group member state machine

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15321: -- Component/s: clients consumer > Document consumer group member state machine > --

[jira] [Updated] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15533: -- Component/s: clients consumer > Ensure HeartbeatRequestManager only send out some fie

[jira] [Updated] (KAFKA-15555) Ensure wakeups are handled correctly in PrototypeAsyncConsumer.poll()

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-1: -- Component/s: clients consumer > Ensure wakeups are handled correctly in PrototypeAsyn

[jira] [Updated] (KAFKA-15534) Propagate client response time when timeout to the request handler

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15534: -- Component/s: clients consumer > Propagate client response time when timeout to the re

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1357247099 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -79,29 +84,39 @@ public enum Election { private List targetReplica

[jira] [Updated] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15533: --- Labels: kip-848 kip-848-client-support kip-848-e2e kip-848-preview (was: kip-848-e2e kip-848-previe

[jira] [Updated] (KAFKA-14761) Integration Tests for the New Consumer Implementation

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14761: -- Priority: Critical (was: Major) > Integration Tests for the New Consumer Implementation > ---

[jira] [Updated] (KAFKA-15538) Client support for java regex based subscription

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15538: -- Priority: Critical (was: Major) > Client support for java regex based subscription > ---

[jira] [Updated] (KAFKA-15544) Enable existing client integration tests for new protocol

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15544: --- Labels: kip-848 kip-848-client-support kip-848-e2e kip-848-preview (was: kip-848 kip-848-client-sup

[jira] [Updated] (KAFKA-15561) Client support for new SubscriptionPattern based subscription

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15561: -- Priority: Critical (was: Major) > Client support for new SubscriptionPattern based subscription > --

[jira] [Updated] (KAFKA-15281) Implement the groupMetadata Consumer API

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15281: -- Priority: Blocker (was: Major) > Implement the groupMetadata Consumer API > -

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1357244420 ## metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java: ## @@ -434,6 +434,15 @@ Map getConfigs(ConfigResource configResource) {

[jira] [Updated] (KAFKA-15188) Implement more of the remaining PrototypeAsyncConsumer APIs

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15188: -- Priority: Blocker (was: Major) > Implement more of the remaining PrototypeAsyncConsumer APIs > --

[jira] [Updated] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15508: -- Priority: Blocker (was: Minor) > Method always return the same value ApplicationEventProcessor.java >

[jira] [Updated] (KAFKA-15276) Implement partition assignment reconciliation

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15276: -- Priority: Blocker (was: Major) > Implement partition assignment reconciliation >

[jira] [Updated] (KAFKA-14274) Introduce FetchRequestManager to integrate fetch into new consumer threading refactor

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14274: -- Priority: Blocker (was: Major) > Introduce FetchRequestManager to integrate fetch into new consumer t

[jira] [Updated] (KAFKA-15184) New consumer internals refactoring and clean up

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15184: -- Priority: Blocker (was: Major) > New consumer internals refactoring and clean up > --

[jira] [Updated] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced/failed

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15588: --- Labels: kip-848 kip-848-client-support (was: kip-848 kip-848-client-support kip-848-e2e) > Purge t

[jira] [Assigned] (KAFKA-15574) Integrate partition assignment reconciliation with heartbeat request manager

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-15574: - Assignee: (was: Kirk True) > Integrate partition assignment reconciliation with heartbeat r

[jira] [Assigned] (KAFKA-15573) Implement auto-commit on partition assignment revocation

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-15573: - Assignee: (was: Kirk True) > Implement auto-commit on partition assignment revocation > ---

[jira] [Updated] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15508: -- Labels: consumer-threading-refactor kip-848-e2e kip-848-preview (was: kip-848-e2e kip-848-preview) >

[jira] [Updated] (KAFKA-15184) New consumer internals refactoring and clean up

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15184: -- Labels: consumer-threading-refactor kip-848-e2e kip-848-preview (was: consumer-threading-refactor kip

Re: [PR] KAFKA-15597: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis [kafka]

2023-10-12 Thread via GitHub
mimaison commented on PR #14536: URL: https://github.com/apache/kafka/pull/14536#issuecomment-1760157903 Yes that's an option too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[jira] [Updated] (KAFKA-14724) Port tests in FetcherTest to FetchRequestManagerTest

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14724: -- Labels: consumer-threading-refactor kip-848-e2e kip-848-preview (was: consumer-threading-refactor kip

[jira] [Commented] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774649#comment-17774649 ] Philip Nee commented on KAFKA-15588: We might need to introduce specific logic becau

[jira] [Updated] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15508: -- Labels: kip-848-e2e kip-848-preview (was: ) > Method always return the same value ApplicationEventPro

[jira] [Updated] (KAFKA-14274) Introduce FetchRequestManager to integrate fetch into new consumer threading refactor

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14274: -- Labels: consumer-threading-refactor kip-848-e2e kip-848-preview (was: consumer-threading-refactor kip

[jira] [Updated] (KAFKA-15588) Purge the unsent offset commits/fetches when the member is fenced/failed

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15588: --- Summary: Purge the unsent offset commits/fetches when the member is fenced/failed (was: Purge the u

[jira] [Updated] (KAFKA-15188) Implement more of the remaining PrototypeAsyncConsumer APIs

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15188: -- Labels: consumer-threading-refactor kip-848-e2e kip-848-preview (was: consumer-threading-refactor kip

[jira] [Updated] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15592: --- Labels: kip-848 kip-848-client-support kip-848-e2e kip-848-preview (was: ) > Member does not need t

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-12 Thread via GitHub
hanyuzheng7 commented on code in PR #14523: URL: https://github.com/apache/kafka/pull/14523#discussion_r1357233277 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -863,6 +863,14 @@ private void shouldHandleWindowKeyQueries(fina

[jira] [Updated] (KAFKA-15562) CommitRequestManager needs to test different error handling

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15562: --- Labels: consumer-threading-refactor kip-848-preview (was: ) > CommitRequestManager needs to test di

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357229686 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { p

[jira] [Updated] (KAFKA-15555) Ensure wakeups are handled correctly in PrototypeAsyncConsumer.poll()

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-1: --- Labels: consumer-threading-refactor kip-848-preview (was: ) > Ensure wakeups are handled correctly

[jira] [Updated] (KAFKA-15550) OffsetsForTimes validation for negative timestamps in new consumer

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15550: --- Labels: kip-848-client-support kip-848-preview (was: kip-848-client-support) > OffsetsForTimes val

[jira] [Updated] (KAFKA-15277) Design & implement support for internal Consumer delegates

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15277: --- Labels: consumer-threading-refactor kip-848 kip-848-client-support kip-848-preview (was: consumer-t

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
splett2 commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357227824 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { p

[jira] [Updated] (KAFKA-14761) Integration Tests for the New Consumer Implementation

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-14761: --- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Integrati

[jira] [Updated] (KAFKA-15284) Implement ConsumerGroupProtocolVersionResolver to determine consumer group protocol

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15284: --- Labels: consumer-threading-refactor kip-848 kip-848-client-support kip-848-preview (was: consumer-t

[jira] [Updated] (KAFKA-14274) Introduce FetchRequestManager to integrate fetch into new consumer threading refactor

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14274: -- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Introduce F

[jira] [Updated] (KAFKA-14724) Port tests in FetcherTest to FetchRequestManagerTest

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14724: -- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Port tests

[jira] [Updated] (KAFKA-15184) New consumer internals refactoring and clean up

2023-10-12 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15184: -- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > New consume

[jira] [Updated] (KAFKA-15281) Implement the groupMetadata Consumer API

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15281: --- Labels: consumer-threading-refactor kip-848 kip-848-client-support kip-848-preview (was: consumer-t

[jira] [Updated] (KAFKA-15317) Fix for async consumer access to committed offsets with multiple consumers

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15317: --- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Fix for a

[jira] [Updated] (KAFKA-15188) Implement more of the remaining PrototypeAsyncConsumer APIs

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15188: --- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Implement

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357223235 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation (A

[jira] [Updated] (KAFKA-15475) Timeout request might retry forever even if the user API times out in PrototypeAsyncConsumer

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15475: --- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > Timeout r

[jira] [Updated] (KAFKA-15316) CommitRequestManager not calling RequestState callbacks

2023-10-12 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15316: --- Labels: consumer-threading-refactor kip-848-preview (was: consumer-threading-refactor) > CommitReq

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
divijvaidya commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1357210089 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LazyIndex.java: ## @@ -41,7 +41,7 @@ * Methods of this class are thread safe. Make sure to check `

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357208153 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { p

Re: [PR] KAFKA-15598: Add integration tests for DescribeGroups API, DeleteGroups API and OffsetDelete API [kafka]

2023-10-12 Thread via GitHub
rreddy-22 commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1357206875 ## core/src/test/scala/unit/kafka/server/OffsetDeleteRequestTest.scala: ## @@ -0,0 +1,157 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or mo

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
chb2ab commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357205210 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { pu

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-12 Thread via GitHub
mjsax commented on code in PR #14523: URL: https://github.com/apache/kafka/pull/14523#discussion_r1357200716 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -880,25 +888,113 @@ private void shouldHandleWindowKeyQueries(final F

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-12 Thread via GitHub
mjsax commented on code in PR #14523: URL: https://github.com/apache/kafka/pull/14523#discussion_r1357200115 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -880,25 +888,113 @@ private void shouldHandleWindowKeyQueries(final F

[jira] [Comment Edited] (KAFKA-14482) Move LogLoader to storage module

2023-10-12 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774641#comment-17774641 ] Alexandre Dupriez edited comment on KAFKA-14482 at 10/12/23 5:56 PM: -

[jira] [Commented] (KAFKA-14482) Move LogLoader to storage module

2023-10-12 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774641#comment-17774641 ] Alexandre Dupriez commented on KAFKA-14482: --- [~ijuma] I am really sorry - I wi

[jira] [Assigned] (KAFKA-14482) Move LogLoader to storage module

2023-10-12 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez reassigned KAFKA-14482: - Assignee: (was: Alexandre Dupriez) > Move LogLoader to storage module > ---

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-12 Thread via GitHub
mjsax commented on code in PR #14523: URL: https://github.com/apache/kafka/pull/14523#discussion_r1357193113 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -863,6 +863,14 @@ private void shouldHandleWindowKeyQueries(final Fun

[jira] [Resolved] (KAFKA-15540) Handle heartbeat and revocation when consumer leaves group

2023-10-12 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-15540. Resolution: Duplicate Duplicate of KAKFA-15548 > Handle heartbeat and revocation when con

[jira] [Updated] (KAFKA-15548) Handling close() properly

2023-10-12 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-15548: --- Description: Upon closing of the {{Consumer}} we need to: # Complete pending commits # Rev

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-12 Thread via GitHub
mjsax commented on code in PR #14523: URL: https://github.com/apache/kafka/pull/14523#discussion_r1357188427 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -716,7 +716,7 @@ public void process(final Record record) {

[jira] [Updated] (KAFKA-15548) Handling close() properly

2023-10-12 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-15548: --- Description: Upon closing of the {{Consumer}} we need to: # Complete pending commits # Aut

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357182431 ## clients/src/main/resources/common/message/ProduceResponse.json: ## @@ -32,7 +32,9 @@ // records that cause the whole batch to be dropped. See KIP-467 for detail

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357181404 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { p

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357176285 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { p

[jira] [Commented] (KAFKA-13639) NotEnoughReplicasException for __consumer_offsets topic due to out of order offset

2023-10-12 Thread shilin Lu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774633#comment-17774633 ] shilin Lu commented on KAFKA-13639: --- i found this problem in kafka 0.11.0.3 > NotEnou

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
imcdo commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1357170513 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -68,12 +67,21 @@ def produce_messages(self, topics, max_messages=1): self.logger.debug("Produce wo

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1357170133 ## tests/kafkatest/services/kafka/config_property.py: ## @@ -199,6 +200,9 @@ val SSLEndpointIdentificationAlgorithmProp = SSLConfigs.SSL_ENDPOINT_IDENTIFICATION_A

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
imcdo commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1357169852 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -68,12 +67,21 @@ def produce_messages(self, topics, max_messages=1): self.logger.debug("Produce wo

[jira] [Commented] (KAFKA-13639) NotEnoughReplicasException for __consumer_offsets topic due to out of order offset

2023-10-12 Thread shilin Lu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774631#comment-17774631 ] shilin Lu commented on KAFKA-13639: --- is there any progress for this issue?  [~ijuma] 

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-12 Thread via GitHub
chb2ab commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1357169045 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -338,7 +338,7 @@ public boolean isControllerRegistrationSupported() { pu

[jira] [Updated] (KAFKA-15599) Move classes in kafka.raft from core module to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15599: Description: Moving `UnifiedLog` and `LogManager` to the storage layer allows us to fix some comp

[jira] [Updated] (KAFKA-15599) Move classes in kafka.raft from core module to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15599: Description: Moving `UnifiedLog` and `LogManager` to the storage layer allows us to fix some comp

[jira] [Updated] (KAFKA-15599) Move classes in kafka.raft from core module to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15599: Description: This includes KafkaMetadataLog, KafkaNetworkChannel and RaftManager. As part of the

[jira] (KAFKA-15599) Move classes in kafka.raft from core module to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599 ] Ismael Juma deleted comment on KAFKA-15599: - was (Author: ijuma): Open question: where should `kafka.raft.KafkaNetworkChannel` move to? `metadata`, `raft` or somewhere else? > Move classes

[jira] [Updated] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15599: Summary: Move KafkaMetadataLog and RaftManager to raft module (was: Move KafkaMetadataLog and Raf

[jira] [Updated] (KAFKA-15599) Move classes in kafka.raft from core module to raft module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15599: Summary: Move classes in kafka.raft from core module to raft module (was: Move KafkaMetadataLog a

[jira] [Updated] (KAFKA-15594) Add 3.6.0 to streams upgrade/compatibility tests

2023-10-12 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15594: Component/s: streams system tests > Add 3.6.0 to streams upgrade/compatib

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357145438 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357145438 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Comment Edited] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma edited comment on KAFKA-15599 at 10/12/23 5:06 PM: ---

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
kirktrue commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357130345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
kirktrue commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357130345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15597: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis [kafka]

2023-10-12 Thread via GitHub
Schm1tz1 commented on PR #14536: URL: https://github.com/apache/kafka/pull/14536#issuecomment-1760013569 Hi @mimaison , sure, will do that. Fair point, I was also thinking whether it makes sense to add a second config for header pattern to allow both options. -- This is an automated m

[jira] [Comment Edited] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma edited comment on KAFKA-15599 at 10/12/23 4:58 PM: ---

[jira] [Commented] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma commented on KAFKA-15599: - Open question: should `KafkaNetworkChannel` m

[jira] [Created] (KAFKA-15600) KIP-990: Capability to SUSPEND Tasks on DeserializationException

2023-10-12 Thread Nicholas Telford (Jira)
Nicholas Telford created KAFKA-15600: Summary: KIP-990: Capability to SUSPEND Tasks on DeserializationException Key: KAFKA-15600 URL: https://issues.apache.org/jira/browse/KAFKA-15600 Project: Kaf

Re: [PR] KAFKA-15428: Cluster-wide dynamic log adjustments for Connect [kafka]

2023-10-12 Thread via GitHub
C0urante commented on PR #14538: URL: https://github.com/apache/kafka/pull/14538#issuecomment-1760004960 @gharris1727, @yashmayya, @mimaison would any of you be able to take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[jira] [Created] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-15599: --- Summary: Move KafkaMetadataLog and RaftManager to metadata module Key: KAFKA-15599 URL: https://issues.apache.org/jira/browse/KAFKA-15599 Project: Kafka Issue

[PR] KAFKA-15428: Cluster-wide dynamic log adjustments for Connect [kafka]

2023-10-12 Thread via GitHub
C0urante opened a new pull request, #14538: URL: https://github.com/apache/kafka/pull/14538 [Jira](https://issues.apache.org/jira/browse/KAFKA-15428), [KIP-976](https://cwiki.apache.org/confluence/display/KAFKA/KIP-976%3A+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect) Adds a n

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-10-12 Thread via GitHub
nikramakrishnan commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1357082765 ## core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java: ## @@ -68,14 +68,16 @@ */ public class ReplicaFetcherTierStateMachine implements Tier

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-10-12 Thread via GitHub
nikramakrishnan commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1357082765 ## core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java: ## @@ -68,14 +68,16 @@ */ public class ReplicaFetcherTierStateMachine implements Tier

<    1   2   3   >