[jira] [Updated] (KAFKA-15635) Investigate FetcherTest's/FetchRequestManager's testFetcherLeadMetric

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15635: -- Labels: consumer-threading-refactor kip-848-client-support kip-848-preview (was: consumer-threading-r

[jira] [Updated] (KAFKA-15652) Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUtils.getOffsetResetTimestamp()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Labels: consumer-threading-refactor kip-848-client-support kip-848-preview (was: consumer-threading-r

[jira] [Commented] (KAFKA-14402) Transactions Server Side Defense

2023-10-19 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1512#comment-1512 ] Travis Bischel commented on KAFKA-14402: Thanks, that is clearer. I note that cl

[jira] [Updated] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15556: -- Labels: consumer-threading-refactor kip-848-client-support kip-848-preview (was: consumer-threading-r

[jira] [Updated] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15556: -- Labels: consumer-threading-refactor (was: ) > Remove NetworkClientDelegate methods isUnavailable, may

[jira] [Updated] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15651: -- Labels: consumer-threading-refactor (was: ) > Investigate auto commit guarantees during Consumer.assi

[jira] [Updated] (KAFKA-15550) OffsetsForTimes validation for negative timestamps in new consumer

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15550: -- Labels: consumer-threading-refactor kip-848 kip-848-client-support kip-848-preview (was: kip-848 kip-

[jira] [Updated] (KAFKA-15558) Determine if Timer should be used elsewhere in PrototypeAsyncConsumer.updateFetchPositions()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15558: -- Labels: consumer-threading-refactor (was: ) > Determine if Timer should be used elsewhere in > Proto

[jira] [Updated] (KAFKA-15320) Document event queueing patterns

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15320: -- Labels: consumer-threading-refactor (was: ) > Document event queueing patterns >

[jira] [Updated] (KAFKA-15652) Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUtils.getOffsetResetTimestamp()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Description: In the {{updateFetchPositions()}} method implementation, both {{KafkaConsumer}} and {{Pr

[jira] [Updated] (KAFKA-15652) Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUtils.getOffsetResetTimestamp()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Labels: consumer-threading-refactor (was: ) > Add unit/integration tests to verify OffsetOutOfRangeEx

[jira] [Updated] (KAFKA-15652) Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUtils.getOffsetResetTimestamp()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Priority: Minor (was: Major) > Add unit/integration tests to verify OffsetOutOfRangeException is thro

[jira] [Comment Edited] (KAFKA-14402) Transactions Server Side Defense

2023-10-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1511#comment-1511 ] Justine Olshan edited comment on KAFKA-14402 at 10/20/23 12:13 AM: ---

[jira] [Commented] (KAFKA-14402) Transactions Server Side Defense

2023-10-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1511#comment-1511 ] Justine Olshan commented on KAFKA-14402: I've added a note to the KIP to make th

[jira] [Updated] (KAFKA-15652) Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUtils.getOffsetResetTimestamp()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Summary: Add unit/integration tests to verify OffsetOutOfRangeException is thrown for OffsetFetcherUti

[jira] [Updated] (KAFKA-15652) Investigate resetting offsets in SubscriptionState.resetInitializingPositions()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15652: -- Description: In the {{updateFetchPositions()}} method implementation, both {{KafkaConsumer}} and {{Pr

[jira] [Commented] (KAFKA-14402) Transactions Server Side Defense

2023-10-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1507#comment-1507 ] Justine Olshan commented on KAFKA-14402: HI [~twmb] . Sorry if this was unclear.

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366267515 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366266506 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

[jira] [Created] (KAFKA-15655) Consider making transactional apis more compatible with topic IDs

2023-10-19 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-15655: -- Summary: Consider making transactional apis more compatible with topic IDs Key: KAFKA-15655 URL: https://issues.apache.org/jira/browse/KAFKA-15655 Project: Kafka

[jira] [Created] (KAFKA-15654) Address Transactions Errors

2023-10-19 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-15654: -- Summary: Address Transactions Errors Key: KAFKA-15654 URL: https://issues.apache.org/jira/browse/KAFKA-15654 Project: Kafka Issue Type: Sub-task

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366262621 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -0,0 +1,241 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366237944 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-19 Thread via GitHub
junrao commented on PR #14465: URL: https://github.com/apache/kafka/pull/14465#issuecomment-1771843763 @ijuma : Thanks for the comment. Yes, agree that CleanShutdownFileHandler should be in the storage module. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366237460 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

[jira] [Created] (KAFKA-15653) NPE in ChunkedByteStream.

2023-10-19 Thread Travis Bischel (Jira)
Travis Bischel created KAFKA-15653: -- Summary: NPE in ChunkedByteStream. Key: KAFKA-15653 URL: https://issues.apache.org/jira/browse/KAFKA-15653 Project: Kafka Issue Type: Bug Compo

[jira] [Commented] (KAFKA-14402) Transactions Server Side Defense

2023-10-19 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1489#comment-1489 ] Travis Bischel commented on KAFKA-14402: Can the KIP be updated to refer that v5

[jira] [Created] (KAFKA-15652) Investigate resetting offsets in SubscriptionState.resetInitializingPositions()

2023-10-19 Thread Kirk True (Jira)
Kirk True created KAFKA-15652: - Summary: Investigate resetting offsets in SubscriptionState.resetInitializingPositions() Key: KAFKA-15652 URL: https://issues.apache.org/jira/browse/KAFKA-15652 Project: Ka

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-19 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1366182334 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

[jira] [Updated] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15651: -- Description: In the {{assign()}} method implementation, both {{KafkaConsumer}} and {{PrototypeAsyncCo

[jira] [Commented] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

2023-10-19 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1482#comment-1482 ] Kirk True commented on KAFKA-15651: --- cc [~pnee]  > Investigate auto commit guarantees

[jira] [Created] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

2023-10-19 Thread Kirk True (Jira)
Kirk True created KAFKA-15651: - Summary: Investigate auto commit guarantees during Consumer.assign() Key: KAFKA-15651 URL: https://issues.apache.org/jira/browse/KAFKA-15651 Project: Kafka Issue

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-19 Thread via GitHub
jolshan commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1771759687 Thanks @nizhikov for the changes. I'm going let the build run. I will also wait a day or so to see if @mimaison has any more comments. -- This is an automated message from the Apache Gi

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-19 Thread via GitHub
nizhikov commented on code in PR #13247: URL: https://github.com/apache/kafka/pull/13247#discussion_r136618 ## tools/src/test/java/org/apache/kafka/tools/reassign/ReassignPartitionsIntegrationTest.java: ## @@ -857,68 +864,13 @@ public void close() { } } -

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-19 Thread via GitHub
CalvinConfluent commented on PR #14465: URL: https://github.com/apache/kafka/pull/14465#issuecomment-1771750393 @ijuma I will make a PR to move it under the storage. cc @junrao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] KAFKA-15607:Possible NPE is thrown in MirrorCheckpointTask [kafka]

2023-10-19 Thread via GitHub
C0urante commented on PR #14587: URL: https://github.com/apache/kafka/pull/14587#issuecomment-1771748040 Thanks @hudeqi, I think this is reasonable. Do you know why Sarama sets offsets to -1? If it's for normal operations and not indicative of something wrong, we may not even need to log a

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #13247: URL: https://github.com/apache/kafka/pull/13247#discussion_r1366159733 ## tools/src/test/java/org/apache/kafka/tools/reassign/ReassignPartitionsIntegrationTest.java: ## @@ -857,68 +864,13 @@ public void close() { } } -p

Re: [PR] KAFKA-15607:Possible NPE is thrown in MirrorCheckpointTask [kafka]

2023-10-19 Thread via GitHub
C0urante commented on code in PR #14587: URL: https://github.com/apache/kafka/pull/14587#discussion_r1366159693 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorCheckpointTaskTest.java: ## @@ -169,6 +169,33 @@ public void testSyncOffset() {

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-19 Thread via GitHub
nizhikov commented on code in PR #13247: URL: https://github.com/apache/kafka/pull/13247#discussion_r1366140798 ## tools/src/test/java/org/apache/kafka/tools/reassign/ReassignPartitionsUnitTest.java: ## @@ -769,22 +762,12 @@ public void testPropagateInvalidJsonError() { }

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-19 Thread via GitHub
nizhikov commented on code in PR #13247: URL: https://github.com/apache/kafka/pull/13247#discussion_r1366130656 ## tools/src/test/java/org/apache/kafka/tools/reassign/ReassignPartitionsUnitTest.java: ## @@ -769,22 +762,12 @@ public void testPropagateInvalidJsonError() { }

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
philipnee commented on PR #14575: URL: https://github.com/apache/kafka/pull/14575#issuecomment-1771707605 @AndrewJSchofield @apoorvmittal10 - Do you guys want to use a KIP-714 flag for tracking? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] KAFKA-15563: Provide informative error messages when Connect REST requests time out [kafka]

2023-10-19 Thread via GitHub
C0urante commented on PR #14562: URL: https://github.com/apache/kafka/pull/14562#issuecomment-1771701339 Thanks Greg. To be clear, this isn't really a first pass--it's the first published one :) With regards to giving each request a single scope--this would fail to capture blocking o

[jira] [Created] (KAFKA-15650) Data-loss on leader shutdown right after partition creation?

2023-10-19 Thread Igor Soarez (Jira)
Igor Soarez created KAFKA-15650: --- Summary: Data-loss on leader shutdown right after partition creation? Key: KAFKA-15650 URL: https://issues.apache.org/jira/browse/KAFKA-15650 Project: Kafka I

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
philipnee commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1366056376 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/Emitter.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] Kafka 12317: Relax non-null key requirement in Kafka Streams [kafka]

2023-10-19 Thread via GitHub
wcarlson5 commented on code in PR #14174: URL: https://github.com/apache/kafka/pull/14174#discussion_r1366083940 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamJoinWindow.java: ## @@ -51,8 +51,8 @@ public void init(final ProcessorContext context) {

[jira] [Created] (KAFKA-15649) Handle directory failure timeout

2023-10-19 Thread Igor Soarez (Jira)
Igor Soarez created KAFKA-15649: --- Summary: Handle directory failure timeout Key: KAFKA-15649 URL: https://issues.apache.org/jira/browse/KAFKA-15649 Project: Kafka Issue Type: Sub-task

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on PR #14545: URL: https://github.com/apache/kafka/pull/14545#issuecomment-1771582104 Thanks for the PR, @mumrah ! A very important fix. Do we need a test of handling a leaderAndIsrRequest that creates stray partitions, in ReplicaManagerTest? -- This is an automat

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366013492 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationZkWriter.java: ## @@ -146,6 +147,15 @@ void handleTopicsSnapshot(TopicsImage topicsImage, K

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366011893 ## core/src/main/scala/kafka/zk/migration/ZkTopicMigrationClient.scala: ## @@ -47,8 +47,14 @@ class ZkTopicMigrationClient(zkClient: KafkaZkClient) extends TopicMigrat

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366010087 ## clients/src/main/java/org/apache/kafka/common/requests/LeaderAndIsrRequest.java: ## @@ -82,6 +108,10 @@ public LeaderAndIsrRequest build(short version) {

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366008719 ## clients/src/main/java/org/apache/kafka/common/requests/LeaderAndIsrRequest.java: ## @@ -42,26 +42,52 @@ public class LeaderAndIsrRequest extends AbstractControlRe

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366007724 ## clients/src/main/java/org/apache/kafka/common/requests/LeaderAndIsrRequest.java: ## @@ -42,26 +42,52 @@ public class LeaderAndIsrRequest extends AbstractControlRe

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366007543 ## clients/src/main/java/org/apache/kafka/common/requests/LeaderAndIsrRequest.java: ## @@ -42,26 +42,52 @@ public class LeaderAndIsrRequest extends AbstractControlRe

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1366008173 ## clients/src/main/java/org/apache/kafka/common/requests/LeaderAndIsrRequest.java: ## @@ -42,26 +42,52 @@ public class LeaderAndIsrRequest extends AbstractControlRe

Re: [PR] MINOR: Don't log missing partitions at ERROR level [kafka]

2023-10-19 Thread via GitHub
cmccabe commented on PR #14579: URL: https://github.com/apache/kafka/pull/14579#issuecomment-1771568882 Perhaps we should just lower the priority to INFO instead of ERROR? I do think it's nice to know when we create a new log directory. -- This is an automated message from the Apache Git

[jira] [Resolved] (KAFKA-15581) Introduce ELR

2023-10-19 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu resolved KAFKA-15581. Reviewer: David Arthur Resolution: Fixed > Introduce ELR > - > >

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365978462 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365978462 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[PR] KAFKA-15583: Enforce HWM advance only if partition is not under-min-ISR [kafka]

2023-10-19 Thread via GitHub
CalvinConfluent opened a new pull request, #14594: URL: https://github.com/apache/kafka/pull/14594 Enforce a new rule of the HWM advance. Only advance HWM if - Using ELR - ISR size >= min ISR https://issues.apache.org/jira/browse/KAFKA-15583 -- This is an automated message fr

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on PR #1: URL: https://github.com/apache/kafka/pull/1#issuecomment-1771527507 @chb2ab is there a JIRA for this work? If not, can we create one and format the title as the jira title? -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365967820 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherThreadTest.scala: ## @@ -125,7 +125,6 @@ class ReplicaFetcherThreadTest { val replicaManager: ReplicaManager

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
ijuma commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365968314 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365967290 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherThreadTest.scala: ## @@ -125,7 +125,6 @@ class ReplicaFetcherThreadTest { val replicaManager: ReplicaManager

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365966459 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -619,6 +641,21 @@ class KafkaApis(val requestChannel: RequestChannel, request.header.clientId,

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365966194 ## clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java: ## @@ -98,13 +116,20 @@ private static ProduceResponseData toData(Map

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365958420 ## clients/src/main/resources/common/message/FetchResponse.json: ## @@ -102,6 +104,15 @@ "about": "The preferred read replica for the consumer to use on its

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365957228 ## clients/src/main/resources/common/message/ProduceResponse.json: ## @@ -59,10 +61,26 @@ "about": "The error message of the record that caused the batch t

Re: [PR] MINOR: Rename log dir UUIDs [kafka]

2023-10-19 Thread via GitHub
ijuma commented on code in PR #14517: URL: https://github.com/apache/kafka/pull/14517#discussion_r1365956252 ## clients/src/main/java/org/apache/kafka/common/Uuid.java: ## @@ -49,21 +49,21 @@ public class Uuid implements Comparable { /** * A UUID that is used to ident

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365955785 ## clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java: ## @@ -98,13 +116,20 @@ private static ProduceResponseData toData(Map

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365955504 ## clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java: ## @@ -156,7 +182,11 @@ public PartitionResponse(Errors error, long baseOffset, long lo

[PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-19 Thread via GitHub
CalvinConfluent opened a new pull request, #14593: URL: https://github.com/apache/kafka/pull/14593 The PR includes the following changes 1. Allow ISR shrink to empty. 2. Allow leader election with ELR members. 3. Allow electing the last known leader. -- This is an automated me

Re: [PR] KIP-951: Server side and protocol changes for KIP-951 [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1365954460 ## clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java: ## @@ -156,7 +182,11 @@ public PartitionResponse(Errors error, long baseOffset, long lo

Re: [PR] MINOR: cherry pick KAFKA-13973 to 3.5 [kafka]

2023-10-19 Thread via GitHub
mjsax commented on PR #14592: URL: https://github.com/apache/kafka/pull/14592#issuecomment-1771505633 Seems there is some issue on `trunk`? ``` [Error] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14592/core/src/test/scala/integration/kafka/zk/ZkMigrationIntegrationTest.

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-19 Thread via GitHub
mumrah merged PR #14312: URL: https://github.com/apache/kafka/pull/14312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-19 Thread via GitHub
ijuma commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1365925380 ## core/src/test/java/kafka/log/CleanShutdownFileHandlerTest.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-19 Thread via GitHub
ijuma commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1365924863 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contrib

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365915582 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365914738 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Updated] (KAFKA-15648) QuorumControllerTest#testBootstrapZkMigrationRecord is flaky

2023-10-19 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15648: - Description: Noticed that this test failed on Jenkins with {code} org.apache.kafka.server.faul

[jira] [Updated] (KAFKA-15648) QuorumControllerTest#testBootstrapZkMigrationRecord is flaky

2023-10-19 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15648: - Labels: flaky-test good-first-issue (was: ) > QuorumControllerTest#testBootstrapZkMigrationReco

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on PR #14575: URL: https://github.com/apache/kafka/pull/14575#issuecomment-1771452093 @kirktrue @mjsax @AndrewJSchofield Thanks for the feedback. I have tried to address the comments on code or replied. I have structured telemetry package simpler now as like other p

[jira] [Created] (KAFKA-15648) QuorumControllerTest#testBootstrapZkMigrationRecord is flaky

2023-10-19 Thread David Arthur (Jira)
David Arthur created KAFKA-15648: Summary: QuorumControllerTest#testBootstrapZkMigrationRecord is flaky Key: KAFKA-15648 URL: https://issues.apache.org/jira/browse/KAFKA-15648 Project: Kafka

[jira] [Updated] (KAFKA-15605) Topics marked for deletion in ZK are incorrectly migrated to KRaft

2023-10-19 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15605: - Description: When migrating topics from ZooKeeper, the KRaft controller reads all the topic and

[jira] [Assigned] (KAFKA-15605) Topics marked for deletion in ZK are incorrectly migrated to KRaft

2023-10-19 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur reassigned KAFKA-15605: Assignee: David Arthur > Topics marked for deletion in ZK are incorrectly migrated to KRa

[jira] [Updated] (KAFKA-15605) Topics marked for deletion in ZK are incorrectly migrated to KRaft

2023-10-19 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15605: - Description: When migrating topics from ZooKeeper, the KRaft controller reads all the topic and

[jira] [Comment Edited] (KAFKA-7699) Improve wall-clock time punctuations

2023-10-19 Thread Herman Kolstad Jakobsen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1412#comment-1412 ] Herman Kolstad Jakobsen edited comment on KAFKA-7699 at 10/19/23 5:46 PM: -

[jira] [Commented] (KAFKA-7699) Improve wall-clock time punctuations

2023-10-19 Thread Herman Kolstad Jakobsen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1412#comment-1412 ] Herman Kolstad Jakobsen commented on KAFKA-7699: I could make an attempt

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365909320 ## clients/src/main/java/org/apache/kafka/common/telemetry/collector/MetricsCollector.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
ijuma commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365908991 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365904474 ## clients/src/main/java/org/apache/kafka/common/telemetry/emitter/Emitter.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365904772 ## clients/src/main/java/org/apache/kafka/common/telemetry/metrics/MetricKey.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] KAFKA-15378: fix streams upgrade system test [kafka]

2023-10-19 Thread via GitHub
mjsax commented on PR #14539: URL: https://github.com/apache/kafka/pull/14539#issuecomment-1771437783 Test failures. Much fewer than before. A few test (ie, "From version" seems to be "stable" while others are not -- will keep digging. - test_upgrade_to_cooperative_rebalance - 0.

[jira] [Resolved] (KAFKA-15582) Clean shutdown detection, broker side

2023-10-19 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15582. - Fix Version/s: 3.7.0 Resolution: Fixed merged the PR to trunk > Clean shutdown detection, broker

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-19 Thread via GitHub
junrao merged PR #14465: URL: https://github.com/apache/kafka/pull/14465 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365883962 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (AS

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365882203 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-19 Thread via GitHub
mjsax merged PR #14477: URL: https://github.com/apache/kafka/pull/14477 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-19 Thread via GitHub
jolshan commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1365876383 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365870764 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-19 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1365869628 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (AS

<    1   2   3   >