Re: [PR] KAFKA-16696: Removed the in-memory implementation of RSM and RLMM [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15911: URL: https://github.com/apache/kafka/pull/15911#discussion_r1596287627 ## storage/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataManagerTest.java: ## @@ -49,9 +45,10 @@ public class RemoteLogMetadataManagerTest

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-09 Thread via GitHub
kamalcph commented on PR #15825: URL: https://github.com/apache/kafka/pull/15825#issuecomment-2103912972 While going through the usages, it looks to me that the LogOffsetMetadata conversion happens in the KafkaMetadataLog is not correct. Could someone please double check? ```

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15779: URL: https://github.com/apache/kafka/pull/15779#discussion_r1596279690 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ResetConsumerGroupOffsetTest.java: ## @@ -415,153 +474,246 @@ public void

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-09 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1596278649 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1424,11 +1424,18 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** * Given a message

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-09 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1596276757 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1424,11 +1424,18 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** * Given a message

Re: [PR] KAFKA-15615: Improve handling of fetching during metadata updates [kafka]

2024-05-09 Thread via GitHub
appchemist commented on code in PR #15647: URL: https://github.com/apache/kafka/pull/15647#discussion_r1580562449 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java: ## @@ -326,22 +326,34 @@ private void handleInitializeErrors(final

Re: [PR] KAFKA-14885: fix kafka client connect to the broker that offline from… [kafka]

2024-05-09 Thread via GitHub
Stephan14 commented on PR #13531: URL: https://github.com/apache/kafka/pull/13531#issuecomment-2103820670 Hi @divijvaidya, Can you help me to review it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-16652: add unit test for ClusterTemplate offering zero ClusterConfig [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15862: URL: https://github.com/apache/kafka/pull/15862#discussion_r1596214008 ## core/src/test/java/kafka/test/junit/ClusterTestExtensionsUnitTest.java: ## @@ -18,31 +18,53 @@ package kafka.test.junit; import

[jira] [Resolved] (KAFKA-12947) Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest ...

2024-05-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12947. Resolution: Duplicate this is fixed by https://github.com/apache/kafka/pull/14623 >

Re: [PR] KAFKA-14133: Move StreamTaskTest to Mockito [kafka]

2024-05-09 Thread via GitHub
chia7712 merged PR #14716: URL: https://github.com/apache/kafka/pull/14716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15821: URL: https://github.com/apache/kafka/pull/15821#discussion_r1596196298 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java: ## @@ -92,6 +94,7 @@ static void generator(ClusterGenerator

Re: [PR] KAFKA-16643 Fix chaos modifier [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on PR #15890: URL: https://github.com/apache/kafka/pull/15890#issuecomment-2103767353 > But I'm confused. Is there anything I can do about it @dongjinleekr had filed a PR (#10428) to be the start. However, the PR gets conflicts now, and not sure whether

Re: [PR] MINOR; Validate at least one control record [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15912: URL: https://github.com/apache/kafka/pull/15912#discussion_r1596192617 ## raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java: ## @@ -284,6 +284,8 @@ private int validateMemoryRecordAndReturnCount(MemoryRecords

Re: [PR] KAFKA-15951: MissingSourceTopicException should include topic names [kafka]

2024-05-09 Thread via GitHub
chickenchickenlove commented on PR #15573: URL: https://github.com/apache/kafka/pull/15573#issuecomment-2103761393 Thanks for your guideline. It was very helpful to me ‍♂️‍♂️‍♂️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15307: update/note deprecated configs [kafka]

2024-05-09 Thread via GitHub
mjsax commented on PR #14360: URL: https://github.com/apache/kafka/pull/14360#issuecomment-2103760848 Thanks for the PR @Cerchie! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-15307: update/note deprecated configs [kafka]

2024-05-09 Thread via GitHub
mjsax merged PR #14360: URL: https://github.com/apache/kafka/pull/14360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15307: update/note deprecated configs [kafka]

2024-05-09 Thread via GitHub
mjsax commented on code in PR #14360: URL: https://github.com/apache/kafka/pull/14360#discussion_r1596187852 ## docs/streams/developer-guide/config-streams.html: ## @@ -300,12 +306,12 @@ num.standby.replicasnull - default.windowed.key.serde.inner +

Re: [PR] KAFKA-15951: MissingSourceTopicException should include topic names [kafka]

2024-05-09 Thread via GitHub
mjsax commented on PR #15573: URL: https://github.com/apache/kafka/pull/15573#issuecomment-2103753531 Thanks for the PR @chickenchickenlove! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16688: Use helper method to shutdown ExecutorService [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on PR #15886: URL: https://github.com/apache/kafka/pull/15886#issuecomment-2103753542 agree to @soarez observation. I feel this patch can get merged to save all Kafka developers life. #15891 can revert this patch to observe which timer task gets hanging. -- This is

Re: [PR] KAFKA-15951: MissingSourceTopicException should include topic names [kafka]

2024-05-09 Thread via GitHub
mjsax merged PR #15573: URL: https://github.com/apache/kafka/pull/15573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Resolved] (KAFKA-16484) Support to define per broker/controller property by ClusterConfigProperty

2024-05-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16484. Fix Version/s: 3.8.0 Resolution: Fixed > Support to define per broker/controller

Re: [PR] KAFKA-16484: Support to define per broker/controller property by ClusterConfigProperty [kafka]

2024-05-09 Thread via GitHub
chia7712 merged PR #15715: URL: https://github.com/apache/kafka/pull/15715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16573: Specify node and store where serdes are needed [kafka]

2024-05-09 Thread via GitHub
mjsax commented on code in PR #15790: URL: https://github.com/apache/kafka/pull/15790#discussion_r1596180605 ## streams/src/main/java/org/apache/kafka/streams/state/internals/StoreSerdeInitializer.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16643 Fix chaos modifier [kafka]

2024-05-09 Thread via GitHub
gongxuanzhang commented on PR #15890: URL: https://github.com/apache/kafka/pull/15890#issuecomment-2103743672 @chia7712 get it! But I'm confused. Is there anything I can do about it -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15897: URL: https://github.com/apache/kafka/pull/15897#discussion_r1596178459 ## core/src/test/java/kafka/test/annotation/ClusterTest.java: ## @@ -33,7 +33,7 @@ @Retention(RUNTIME) @TestTemplate public @interface ClusterTest { -Type

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on code in PR #15897: URL: https://github.com/apache/kafka/pull/15897#discussion_r1596176943 ## core/src/test/java/kafka/test/ClusterConfig.java: ## @@ -219,8 +221,8 @@ public static class Builder { private Builder() {} -public Builder

Re: [PR] KAFKA-16643 Fix chaos modifier [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on PR #15890: URL: https://github.com/apache/kafka/pull/15890#issuecomment-2103727012 @gongxuanzhang This is still a huge patch, which contains a log of changes across all modules. This is a important improvement which can impact all Kafka developers, and so we should

Re: [PR] KAFKA-15045: (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. [kafka]

2024-05-09 Thread via GitHub
ableegoldman commented on PR #15887: URL: https://github.com/apache/kafka/pull/15887#issuecomment-2103726299 Test failures were unrelated. Merged to trunk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-15045: (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. [kafka]

2024-05-09 Thread via GitHub
ableegoldman merged PR #15887: URL: https://github.com/apache/kafka/pull/15887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: fix LogValidatorTest#checkNonCompressed [kafka]

2024-05-09 Thread via GitHub
chia7712 commented on PR #15904: URL: https://github.com/apache/kafka/pull/15904#issuecomment-2103720759 @junrao thanks for reviewing this rough patch :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-16643 Fix chaos modifier [kafka]

2024-05-09 Thread via GitHub
gongxuanzhang commented on PR #15890: URL: https://github.com/apache/kafka/pull/15890#issuecomment-2103687645 @chia7712 I add rule into checkstyle.xml you can run ``` ./gradlew checkstyleMain checkstyleTest --continue ``` -- This is an automated message from the

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1596055964 ## core/src/main/scala/kafka/server/DelayedFetch.scala: ## @@ -92,7 +92,10 @@ class DelayedFetch( // has just rolled, then the high watermark offset will

[jira] [Commented] (KAFKA-4212) Add a key-value store that is a TTL persistent cache

2024-05-09 Thread Ayoub Omari (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845142#comment-17845142 ] Ayoub Omari commented on KAFKA-4212: My company was looking for such feature earlier this year. For

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595995650 ## raft/src/test/java/org/apache/kafka/raft/internals/VoterSetTest.java: ## @@ -79,20 +81,79 @@ void testRemoveVoter() { ); } +@Test +void

Re: [PR] KAFKA-15649: Handle directory failure timeout [kafka]

2024-05-09 Thread via GitHub
soarez commented on code in PR #15697: URL: https://github.com/apache/kafka/pull/15697#discussion_r1593094798 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2466,7 +2467,6 @@ class ReplicaManager(val config: KafkaConfig, s"for partitions

Re: [PR] KAFKA-15045: (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. [kafka]

2024-05-09 Thread via GitHub
ableegoldman commented on code in PR #15887: URL: https://github.com/apache/kafka/pull/15887#discussion_r1595980303 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/AssignmentConfigs.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595963491 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-16695: Improve expired poll logging [kafka]

2024-05-09 Thread via GitHub
ableegoldman commented on code in PR #15909: URL: https://github.com/apache/kafka/pull/15909#discussion_r1595957783 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -193,11 +193,12 @@ public

Re: [PR] MINOR; Validate at least one control record [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15912: URL: https://github.com/apache/kafka/pull/15912#discussion_r1595930091 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -161,8 +161,8 @@ public VotersRecord toVotersRecord(short version) { * An overlapping

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595923193 ## raft/src/main/java/org/apache/kafka/raft/internals/TreeMapLogHistory.java: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595922267 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595916951 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSetHistory.java: ## @@ -0,0 +1,124 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595915596 ## clients/src/main/java/org/apache/kafka/common/record/MemoryRecords.java: ## @@ -728,83 +730,114 @@ public static MemoryRecords withLeaderChangeMessage(

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595916619 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595916355 ## raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java: ## @@ -245,6 +255,42 @@ private void appendControlMessage(Function valueCreat }

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595916070 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -370,8 +368,52 @@ private void maybeFireLeaderChange() { } } -@Override -

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595915829 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -1470,6 +1524,10 @@ private boolean handleFetchSnapshotResponse(

[PR] Add demo template for transactional client [kafka]

2024-05-09 Thread via GitHub
k-raina opened a new pull request, #15913: URL: https://github.com/apache/kafka/pull/15913 This is example code template for Transactional Client. This code assumes that new Exception types have already been implemented. -- This is an automated message from the Apache Git Service. To

[PR] MINOR; Validate at least one control record [kafka]

2024-05-09 Thread via GitHub
jsancio opened a new pull request, #15912: URL: https://github.com/apache/kafka/pull/15912 Validate that a control batch in the batch accumulator has at least one control record. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [

Re: [PR] KAFKA-16445: Add PATCH method for connector config [kafka]

2024-05-09 Thread via GitHub
C0urante merged PR #6934: URL: https://github.com/apache/kafka/pull/6934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595910078 ## raft/src/main/java/org/apache/kafka/snapshot/RecordsSnapshotReader.java: ## @@ -145,9 +147,7 @@ private Optional> nextBatch() { ); }

[jira] [Commented] (KAFKA-16584) Make log processing summary configurable or debug

2024-05-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845111#comment-17845111 ] Matthias J. Sax commented on KAFKA-16584: - I am on the PMC and can help you. :)  After you wiki

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595898159 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
lianetm commented on PR #15897: URL: https://github.com/apache/kafka/pull/15897#issuecomment-2103291571 Thanks for the patch @FrankYang0529, nice twist. Left some comments. Also there are examples for the `ClusterTest` annotation in the [core README

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
lianetm commented on code in PR #15897: URL: https://github.com/apache/kafka/pull/15897#discussion_r1595878217 ## core/src/test/java/kafka/test/annotation/ClusterTest.java: ## @@ -33,7 +33,7 @@ @Retention(RUNTIME) @TestTemplate public @interface ClusterTest { -Type

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
lianetm commented on code in PR #15897: URL: https://github.com/apache/kafka/pull/15897#discussion_r1595878217 ## core/src/test/java/kafka/test/annotation/ClusterTest.java: ## @@ -33,7 +33,7 @@ @Retention(RUNTIME) @TestTemplate public @interface ClusterTest { -Type

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1595871101 ## raft/src/main/java/org/apache/kafka/raft/internals/TreeMapLogHistory.java: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-16677: Replace ClusterType#ALL and ClusterType#DEFAULT by Array [kafka]

2024-05-09 Thread via GitHub
lianetm commented on code in PR #15897: URL: https://github.com/apache/kafka/pull/15897#discussion_r1595849504 ## core/src/test/java/kafka/test/ClusterConfig.java: ## @@ -219,8 +221,8 @@ public static class Builder { private Builder() {} -public Builder

[jira] [Assigned] (KAFKA-15905) Restarts of MirrorCheckpointTask should not permanently interrupt offset translation

2024-05-09 Thread Edoardo Comar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar reassigned KAFKA-15905: - Assignee: Edoardo Comar > Restarts of MirrorCheckpointTask should not permanently

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-09 Thread via GitHub
gharris1727 commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2103188026 Hey @edoardocomar and @prestona thanks for the PR! One of the reasons I thought this might require a KIP is because it requires additional permissions that the current MM2

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595796530 ## raft/src/test/java/org/apache/kafka/raft/RaftEventSimulationTest.java: ## @@ -850,14 +846,10 @@ public String toString() { } private static class

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595793667 ## raft/src/test/java/org/apache/kafka/raft/ElectionStateTest.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595784042 ## raft/src/test/java/org/apache/kafka/raft/internals/VoterSetTest.java: ## @@ -79,20 +81,79 @@ void testRemoveVoter() { ); } +@Test +void

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595780757 ## raft/src/test/java/org/apache/kafka/raft/internals/KafkaRaftMetricsTest.java: ## @@ -63,7 +66,9 @@ public void tearDown() { private QuorumState

[PR] KAFKA-16696: Removed the in-memory implementation of RSM and RLMM [kafka]

2024-05-09 Thread via GitHub
kamalcph opened a new pull request, #15911: URL: https://github.com/apache/kafka/pull/15911 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[jira] [Updated] (KAFKA-16696) Remove the in-memory implementation of RSM and RLMM

2024-05-09 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-16696: - Description: The in-memory implementation of RSM and RLMM were written to write

[jira] [Updated] (KAFKA-16696) Remove the in-memory implementation of RSM and RLMM

2024-05-09 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-16696: - Description: The in-memory implementation of RSM and RLMM were written to write

[jira] [Created] (KAFKA-16696) Remove the in-memory implementation of RSM and RLMM

2024-05-09 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-16696: Summary: Remove the in-memory implementation of RSM and RLMM Key: KAFKA-16696 URL: https://issues.apache.org/jira/browse/KAFKA-16696 Project: Kafka

[jira] [Updated] (KAFKA-16693) Kafka Users are created with ACL entries and during performing operations allowed by ACL we see Denied Operation

2024-05-09 Thread Janardhana Gopalachar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhana Gopalachar updated KAFKA-16693: -- Description: Hi We have created 2 KafkaUsers from Strimzi operator for 2

[jira] [Assigned] (KAFKA-7300) Add KafkaConsumer fetch-error-rate and fetch-error-total metrics

2024-05-09 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee reassigned KAFKA-7300: - Assignee: Philip Nee (was: Kevin Lu) > Add KafkaConsumer fetch-error-rate and

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-09 Thread via GitHub
kamalcph commented on PR #15825: URL: https://github.com/apache/kafka/pull/15825#issuecomment-2103078761 Test failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-16686: Wait for given offset in TopicBasedRemoteLogMetadataManagerTest [kafka]

2024-05-09 Thread via GitHub
satishd commented on code in PR #15885: URL: https://github.com/apache/kafka/pull/15885#discussion_r1595738359 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerTest.java: ## @@ -152,15 +152,17 @@ public void

Re: [PR] KAFKA-15045: (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. [kafka]

2024-05-09 Thread via GitHub
apourchet commented on code in PR #15887: URL: https://github.com/apache/kafka/pull/15887#discussion_r1595700958 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/KafkaStreamsState.java: ## @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] MINOR: fix LogValidatorTest#checkNonCompressed [kafka]

2024-05-09 Thread via GitHub
junrao commented on code in PR #15904: URL: https://github.com/apache/kafka/pull/15904#discussion_r1595670550 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -414,7 +414,15 @@ class LogValidatorTest { assertEquals(now + 1,

[PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-09 Thread via GitHub
edoardocomar opened a new pull request, #15910: URL: https://github.com/apache/kafka/pull/15910 KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently interrupt offset translation MirrorCheckpointTask reloads the last checkpoint at start, OffsetSyncStore stores

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-09 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845045#comment-17845045 ] Philip Nee commented on KAFKA-16687: Since you mentioned `disabled JMX reporter` would resolve the

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-09 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845042#comment-17845042 ] Philip Nee commented on KAFKA-16687: Thank you.  > Native memory leak by Unsafe_allocatememory in

[jira] [Updated] (KAFKA-16695) Improve expired poll interval logging by showing exceeded time

2024-05-09 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16695: --- Fix Version/s: 3.8.0 > Improve expired poll interval logging by showing exceeded time >

[jira] [Updated] (KAFKA-16695) Improve expired poll interval logging by showing exceeded time

2024-05-09 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16695: --- Labels: kip-848-client-support (was: ) > Improve expired poll interval logging by showing

Re: [PR] KAFKA-16695: Improve expired poll logging [kafka]

2024-05-09 Thread via GitHub
lianetm commented on PR #15909: URL: https://github.com/apache/kafka/pull/15909#issuecomment-2102939146 Hey @mjsax , here is the improved logging following your suggestion, helpful indeed I expect. Would you have a chance to take a look? Thanks! -- This is an automated message from the

Re: [PR] KAFKA-16691: [connect:transform] Support nested field paths on TimestampConverter [kafka]

2024-05-09 Thread via GitHub
C0urante commented on PR #15893: URL: https://github.com/apache/kafka/pull/15893#issuecomment-2102926693 I think this leads to a change in behavior. Right now this test (surprisingly!) passes on trunk: ```java public class TimestampConverterTest { // ... @Test

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595620870 ## raft/src/main/java/org/apache/kafka/raft/QuorumState.java: ## @@ -481,7 +517,7 @@ private void durableTransitionTo(EpochState state) { } } -

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595616944 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -1700,16 +1710,16 @@ private void handleResponse(RaftResponse.Inbound response, long

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595615609 ## raft/src/main/java/org/apache/kafka/raft/FileQuorumStateStore.java: ## @@ -59,17 +54,18 @@ * "data_version":0} * * */ -public class FileBasedStateStore

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-09 Thread via GitHub
dajac commented on PR #15835: URL: https://github.com/apache/kafka/pull/15835#issuecomment-2102880603 @jeffkbkim The unit test seems to be flaky. Could you please check? https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15785/19/tests -- This is an

[PR] MINOR: migrate DescribeConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-09 Thread via GitHub
FrankYang0529 opened a new pull request, #15908: URL: https://github.com/apache/kafka/pull/15908 By using ClusterTestExtensions, DescribeConsumerGroupTest get get away from KafkaServerTestHarness dependency. ### Committer Checklist (excluded from commit message) - [ ] Verify

Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-09 Thread via GitHub
FrankYang0529 commented on PR #15821: URL: https://github.com/apache/kafka/pull/15821#issuecomment-2102879454 > > This is used by ListConsumerGroupTest and DescribeConsumerGroupTest. I think we can create a new class SimpleConsumerGroupExecutorTestUtils for it. WDYT? Thank you. > >

Re: [PR] KAFKA-16660: reduce the check interval to speedup DelegationTokenRequestsTest [kafka]

2024-05-09 Thread via GitHub
brandboat commented on PR #15907: URL: https://github.com/apache/kafka/pull/15907#issuecomment-2102870594 Loop 131 times and all passed `I=0; while ./gradlew core:test --tests DelegationTokenRequestsTest --rerun --fail-fast; do (( I=$I+1 )); echo "Completed run: $I"; sleep 1; done`. But

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595591034 ## raft/src/main/java/org/apache/kafka/raft/EpochState.java: ## @@ -26,15 +27,16 @@ default Optional highWatermark() { } /** - * Decide whether to

[PR] KAFKA-16660: reduce the check interval to speedup DelegationTokenRequestsTest [kafka]

2024-05-09 Thread via GitHub
brandboat opened a new pull request, #15907: URL: https://github.com/apache/kafka/pull/15907 related to https://issues.apache.org/jira/browse/KAFKA-16660 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so

Re: [PR] KAFKA-14133: Move StreamTaskTest to Mockito [kafka]

2024-05-09 Thread via GitHub
clolov commented on code in PR #14716: URL: https://github.com/apache/kafka/pull/14716#discussion_r1595590794 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamTaskTest.java: ## @@ -411,25 +394,32 @@ public void seek(final TopicPartition partition,

[jira] [Updated] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-09 Thread Johnson Okorie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnson Okorie updated KAFKA-16692: --- Description: We have a kafka cluster running on version 3.5.2 that we are upgrading to

[jira] [Updated] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-09 Thread Johnson Okorie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnson Okorie updated KAFKA-16692: --- Description: We have a kafka cluster running on version 3.5.2 that we are upgrading to

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595585037 ## raft/src/main/java/org/apache/kafka/raft/ElectionState.java: ## @@ -115,15 +166,51 @@ public boolean equals(Object o) { ElectionState that =

[jira] [Created] (KAFKA-16695) Improve expired poll interval logging by showing exceeded time

2024-05-09 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16695: -- Summary: Improve expired poll interval logging by showing exceeded time Key: KAFKA-16695 URL: https://issues.apache.org/jira/browse/KAFKA-16695 Project: Kafka

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595581356 ## raft/src/main/java/org/apache/kafka/raft/ElectionState.java: ## @@ -64,47 +62,100 @@ public boolean isLeader(int nodeId) { return leaderIdOrSentinel() ==

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595578267 ## raft/src/main/java/org/apache/kafka/raft/ElectionState.java: ## @@ -16,46 +16,44 @@ */ package org.apache.kafka.raft; +import java.util.List; +import

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595568368 ## raft/src/main/java/org/apache/kafka/raft/CandidateState.java: ## @@ -51,14 +54,27 @@ public class CandidateState implements EpochState { protected

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-09 Thread via GitHub
jsancio commented on code in PR #15859: URL: https://github.com/apache/kafka/pull/15859#discussion_r1595568006 ## raft/src/main/java/org/apache/kafka/raft/internals/VoterSet.java: ## @@ -64,6 +64,43 @@ public Optional voterAddress(int voter, String listener) {

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-09 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845004#comment-17845004 ] Justine Olshan commented on KAFKA-16692: I'm working on a test to recreate the issue, and then

  1   2   >