[GitHub] [kafka] dajac commented on a diff in pull request #14192: MINOR: improve info log for memberIDRequired exception

2023-08-30 Thread via GitHub
dajac commented on code in PR #14192: URL: https://github.com/apache/kafka/pull/14192#discussion_r1309937908 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -501,7 +501,10 @@ boolean joinGroupIfNeeded(final Timer timer) {

[GitHub] [kafka] showuon commented on a diff in pull request #14192: MINOR: improve info log for memberIDRequired exception

2023-08-30 Thread via GitHub
showuon commented on code in PR #14192: URL: https://github.com/apache/kafka/pull/14192#discussion_r1309965911 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -501,7 +501,10 @@ boolean joinGroupIfNeeded(final Timer timer) {

[GitHub] [kafka] hudeqi commented on pull request #14243: KAFKA-14912:Add a dynamic config for remote index cache size

2023-08-30 Thread via GitHub
hudeqi commented on PR #14243: URL: https://github.com/apache/kafka/pull/14243#issuecomment-1698824997 > Continuing discussion on Jira at

[GitHub] [kafka] hudeqi commented on pull request #14284: KAFKA-15396:Add a metric indicating the version of the current running kafka server

2023-08-30 Thread via GitHub
hudeqi commented on PR #14284: URL: https://github.com/apache/kafka/pull/14284#issuecomment-1699113199 > Hi, I have saw the discussion thread. Maybe I need update the pr title to do the second thing in PR description. @jolshan -- This is an automated message from the Apache Git

[GitHub] [kafka] mimaison commented on pull request #14217: KAFKA-14595 ReassignPartitionsCommandArgsTest rewritten in java

2023-08-30 Thread via GitHub
mimaison commented on PR #14217: URL: https://github.com/apache/kafka/pull/14217#issuecomment-1698720236 Thanks. I'm busy with other priorities this week. Feel free to merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] dajac opened a new pull request, #14310: KAFKA-14499: [5/N] Refactor GroupCoordinator.fetchOffsets and GroupCoordinator.fetchAllOffsets

2023-08-30 Thread via GitHub
dajac opened a new pull request, #14310: URL: https://github.com/apache/kafka/pull/14310 This patch refactors the GroupCoordinator.fetchOffsets and GroupCoordinator.fetchAllOffsets methods to take an OffsetFetchRequestGroup and to return an OffsetFetchResponseGroup. It prepares the ground

[GitHub] [kafka] showuon commented on a diff in pull request #14192: MINOR: improve info log for memberIDRequired exception

2023-08-30 Thread via GitHub
showuon commented on code in PR #14192: URL: https://github.com/apache/kafka/pull/14192#discussion_r1309953193 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -501,7 +501,10 @@ boolean joinGroupIfNeeded(final Timer timer) {

[GitHub] [kafka] hudeqi commented on a diff in pull request #14243: KAFKA-14912:Add a dynamic config for remote index cache size

2023-08-30 Thread via GitHub
hudeqi commented on code in PR #14243: URL: https://github.com/apache/kafka/pull/14243#discussion_r1310123945 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -125,12 +126,37 @@ public RemoteIndexCache(RemoteStorageManager

[GitHub] [kafka] divijvaidya commented on pull request #14222: KAFKA-14133: Move RocksDBRangeIteratorTest, TimestampedKeyValueStoreBuilderTest and TimestampedSegmentTest to Mockito

2023-08-30 Thread via GitHub
divijvaidya commented on PR #14222: URL: https://github.com/apache/kafka/pull/14222#issuecomment-1698803492 All test failures unrelated to the tests which are modified in this PR. ``` [Build / JDK 20 and Scala 2.13 /

[GitHub] [kafka] divijvaidya merged pull request #14222: KAFKA-14133: Move RocksDBRangeIteratorTest, TimestampedKeyValueStoreBuilderTest and TimestampedSegmentTest to Mockito

2023-08-30 Thread via GitHub
divijvaidya merged PR #14222: URL: https://github.com/apache/kafka/pull/14222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] showuon commented on pull request #14192: MINOR: improve info log for memberIDRequired exception

2023-08-30 Thread via GitHub
showuon commented on PR #14192: URL: https://github.com/apache/kafka/pull/14192#issuecomment-1698755691 @dajac , could you help take a look for the small PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] yashmayya commented on a diff in pull request #14304: KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes

2023-08-30 Thread via GitHub
yashmayya commented on code in PR #14304: URL: https://github.com/apache/kafka/pull/14304#discussion_r1309974448 ## clients/src/main/java/org/apache/kafka/common/utils/Utils.java: ## @@ -1090,6 +1090,23 @@ public interface UncheckedCloseable extends AutoCloseable {

[GitHub] [kafka] hudeqi commented on a diff in pull request #14243: KAFKA-14912:Add a dynamic config for remote index cache size

2023-08-30 Thread via GitHub
hudeqi commented on code in PR #14243: URL: https://github.com/apache/kafka/pull/14243#discussion_r1310125666 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -669,4 +689,10 @@ public static String

[GitHub] [kafka] cadonna commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1315706213 ## streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorder.java: ## @@ -373,14 +373,14 @@ private Gauge

[GitHub] [kafka] nicktelford commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
nicktelford commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1315734550 ## streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorder.java: ## @@ -373,14 +373,14 @@ private Gauge

[GitHub] [kafka] dajac commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-05 Thread via GitHub
dajac commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1315500509 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-05 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1315883355 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3487,8 +3487,12 @@ class ReplicaManagerTest {

[GitHub] [kafka] yashmayya opened a new pull request, #14337: MINOR: Update the documentation's table of contents to add missing headings for Kafka Connect

2023-09-05 Thread via GitHub
yashmayya opened a new pull request, #14337: URL: https://github.com/apache/kafka/pull/14337 - Some of Connect's top level headings (``) and sub top level headings (``) in the documentation weren't added to the documentation's table of contents. This patch rectifies that. Before:

[GitHub] [kafka] cadonna commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1315913915 ## streams/src/test/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorderGaugesTest.java: ## @@ -211,14 +215,19 @@ private void

[GitHub] [kafka] cadonna commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706873185 @nicktelford Yes, you are right about the existing tests! Sorry I was in a hurry and only skimmed them and missed that fact. I would like to have some tests that verify the behavior

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
abhijeetk88 commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315737833 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] nicktelford commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
nicktelford commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706624785 > I think this part of the unit tests should be affected since the called method changes: > >

[GitHub] [kafka] riedelmax commented on pull request #14124: KAFKA-14509; [1/2] Define ConsumerGroupDescribe API request and response schemas and classes.

2023-09-05 Thread via GitHub
riedelmax commented on PR #14124: URL: https://github.com/apache/kafka/pull/14124#issuecomment-1706895064 @dajac Thank you for the review. I added unit tests and corrected the nits -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] satishd commented on pull request #14335: MINOR: update comment in consumeAction

2023-09-05 Thread via GitHub
satishd commented on PR #14335: URL: https://github.com/apache/kafka/pull/14335#issuecomment-1706906829 There are a few failed tests unrelated to this PR. Merging it to trunk and 3.6. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] cadonna commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1315798587 ## streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorder.java: ## @@ -373,14 +373,14 @@ private Gauge

[GitHub] [kafka] cadonna commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706493373 I think this part of the unit tests should be affected since the called method changes:

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
abhijeetk88 commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315850774 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
abhijeetk88 commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315850774 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] showuon commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-05 Thread via GitHub
showuon commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1315851160 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3487,8 +3487,12 @@ class ReplicaManagerTest {

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
abhijeetk88 commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315851891 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] yashmayya commented on a diff in pull request #14337: MINOR: Update the documentation's table of contents to add missing headings for Kafka Connect

2023-09-05 Thread via GitHub
yashmayya commented on code in PR #14337: URL: https://github.com/apache/kafka/pull/14337#discussion_r1315886711 ## docs/connect.html: ## @@ -999,7 +999,7 @@ Working with SchemasSink connectors are usually simpler because they are consuming data and therefore do not need to

[GitHub] [kafka] kamalcph commented on pull request #14340: KAFKA-15351: Update log-start-offset after leader election for topics enabled with remote storage

2023-09-05 Thread via GitHub
kamalcph commented on PR #14340: URL: https://github.com/apache/kafka/pull/14340#issuecomment-1706829016 @clolov @divijvaidya @showuon @abhijeetk88 @satishd Call for review. Please take a look! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
divijvaidya commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315744386 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] Chuckame commented on pull request #14157: KAFKA-15303: Avoid unnecessary re-serialization in FK-join

2023-09-05 Thread via GitHub
Chuckame commented on PR #14157: URL: https://github.com/apache/kafka/pull/14157#issuecomment-1706826894 Hello @mjsax, do you know if there is potential advancement about this fix ?  -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] kamalcph opened a new pull request, #14340: KAFKA-15351: Update log-start-offset after leader election for topics enabled with remote storage

2023-09-05 Thread via GitHub
kamalcph opened a new pull request, #14340: URL: https://github.com/apache/kafka/pull/14340 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[GitHub] [kafka] pprovenzano opened a new pull request, #14338: KAFKA-15422: Update documentation for Delegation Tokens in Kafka with KRaft

2023-09-05 Thread via GitHub
pprovenzano opened a new pull request, #14338: URL: https://github.com/apache/kafka/pull/14338 Minor changes to documentation to reflect that Delegation Tokens now works with Kafka with KRaft. This update was already merged to trunk. -- This is an automated message from the Apache Git

[GitHub] [kafka] pprovenzano closed pull request #14338: KAFKA-15422: Update documentation for Delegation Tokens in Kafka with KRaft

2023-09-05 Thread via GitHub
pprovenzano closed pull request #14338: KAFKA-15422: Update documentation for Delegation Tokens in Kafka with KRaft URL: https://github.com/apache/kafka/pull/14338 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] nicktelford commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
nicktelford commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1315955627 ## streams/src/test/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorderGaugesTest.java: ## @@ -211,14 +215,19 @@ private void

[GitHub] [kafka] nicktelford commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
nicktelford commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706697411 > @nicktelford Thanks for the update! > > Could you also look why we did not catch this bug in `RocksDBMetricsIntegrationTest`or other metrics integration tests and add tests

[GitHub] [kafka] Cerchie commented on a diff in pull request #14322: KAFKA-15418: update statement on decompression

2023-09-05 Thread via GitHub
Cerchie commented on code in PR #14322: URL: https://github.com/apache/kafka/pull/14322#discussion_r1316003070 ## docs/design.html: ## @@ -136,8 +136,10 @@ -Kafka supports this with an efficient batching format. A batch of messages can be clumped together compressed and

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-05 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1316096433 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/RangeAssignor.java: ## @@ -89,16 +93,17 @@ public

[GitHub] [kafka] nicktelford commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
nicktelford commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706403793 > @nicktelford Could you please add new unit tests for the case of multiple column families and adapt existing unit tests to your fix? No existing tests appear to be affected

[GitHub] [kafka] yashmayya opened a new pull request, #14336: KAFKA-14876: Add stopped state to Kafka Connect Administration docs section

2023-09-05 Thread via GitHub
yashmayya opened a new pull request, #14336: URL: https://github.com/apache/kafka/pull/14336 - https://github.com/apache/kafka/pull/13657 added documentation for the new `PUT /connectors/{name}/stop` REST API (see

[GitHub] [kafka] AndrewJSchofield commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-09-05 Thread via GitHub
AndrewJSchofield commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1315868466 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -408,11 +463,13 @@ private Optional updateLatestMetadata(

[GitHub] [kafka] pprovenzano opened a new pull request, #14339: KAFKA-15422: Update documenttion for delegation tokens when working with Kafka with KRaft

2023-09-05 Thread via GitHub
pprovenzano opened a new pull request, #14339: URL: https://github.com/apache/kafka/pull/14339 Minor changes to documentation to reflect that Delegation Tokens now works with Kafka with KRaft. Cherry-Pick of https://github.com/apache/kafka/pull/14318 Reviewers: Manikumar Reddy --

[GitHub] [kafka] satishd merged pull request #14335: MINOR: update comment in consumeAction

2023-09-05 Thread via GitHub
satishd merged PR #14335: URL: https://github.com/apache/kafka/pull/14335 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cadonna commented on pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-05 Thread via GitHub
cadonna commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1706367819 @nicktelford Could you please add new unit tests for the case of multiple column families and adapt existing unit tests to your fix? -- This is an automated message from the Apache Git

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
abhijeetk88 commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315738312 ## docs/ops.html: ## @@ -1545,6 +1545,51 @@

[GitHub] [kafka] lucasbru commented on a diff in pull request #14180: KAFKA-15326: [7/N] Processing thread non-busy waiting

2023-09-05 Thread via GitHub
lucasbru commented on code in PR #14180: URL: https://github.com/apache/kafka/pull/14180#discussion_r1315783830 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/tasks/DefaultTaskManagerTest.java: ## @@ -94,6 +103,114 @@ public void

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14331: KAFKA-15293 Add documentation for tiered storage metrics

2023-09-05 Thread via GitHub
divijvaidya commented on code in PR #14331: URL: https://github.com/apache/kafka/pull/14331#discussion_r1315859578 ## docs/ops.html: ## @@ -1841,6 +1841,63 @@ Tiered Storage Monitoring Metrics + The following set of metrics are available for monitoring of the tiered storage

[GitHub] [kafka] ableegoldman merged pull request #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-02 Thread via GitHub
ableegoldman merged PR #14326: URL: https://github.com/apache/kafka/pull/14326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] ableegoldman commented on pull request #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-02 Thread via GitHub
ableegoldman commented on PR #14326: URL: https://github.com/apache/kafka/pull/14326#issuecomment-1703978483 Test failures are unrelated, merging to trunk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] ableegoldman commented on pull request #14325: KAFKA-15429: catch+log errors from unsubscribe in streamthread shutdown

2023-09-02 Thread via GitHub
ableegoldman commented on PR #14325: URL: https://github.com/apache/kafka/pull/14325#issuecomment-1703980374 Merged to trunk and cherrypicked to 3.6 and 3.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] ableegoldman commented on pull request #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-02 Thread via GitHub
ableegoldman commented on PR #14326: URL: https://github.com/apache/kafka/pull/14326#issuecomment-1703980104 Merged to trunk and cherrypicked to 3.6 and 3.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314218928 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -309,6 +311,21 @@ public CoordinatorResult commitOffset(

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314264907 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerManager.java: ## @@ -98,35 +98,28 @@ public void

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314264907 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerManager.java: ## @@ -98,35 +98,28 @@ public void

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314265438 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -377,7 +377,9 @@ private void

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314265053 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -157,9 +157,10 @@ private void

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-03 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1314290595 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,62 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314264907 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerManager.java: ## @@ -98,35 +98,28 @@ public void

[GitHub] [kafka] zhaohaidao commented on pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on PR #14271: URL: https://github.com/apache/kafka/pull/14271#issuecomment-1704074948 > @zhaohaidao Thanks for the update! I just made another pass and I left a few more comments for consideration. Could you take a look at them and let me know what you think?

[GitHub] [kafka] showuon commented on pull request #13908: KAFKA-15052 Fix the flaky testBalancePartitionLeaders - part II

2023-09-03 Thread via GitHub
showuon commented on PR #13908: URL: https://github.com/apache/kafka/pull/13908#issuecomment-1704289222 @dimitarndimitrov , thanks for fixing it. Re-triggering the CI build now. If by bumping the session timeout to 2 sec (from 1 sec) can fix the flaky testBalancePartitionLeaders

[GitHub] [kafka] kamalcph opened a new pull request, #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph opened a new pull request, #14329: URL: https://github.com/apache/kafka/pull/14329 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[GitHub] [kafka] kamalcph commented on pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on PR #14329: URL: https://github.com/apache/kafka/pull/14329#issuecomment-1704311904 @clolov @divijvaidya @showuon @abhijeetk88 @satishd This PR is stacked on top of #14307 and #14328 to review the change-set separately (we can merge them if required). Please take

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314217880 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -420,6 +423,18 @@ public Group group(String groupId, long

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314217786 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -420,6 +423,18 @@ public Group group(String groupId, long

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314217697 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -747,4 +749,15 @@ private static Integer decValue(String

[GitHub] [kafka] kamalcph commented on a diff in pull request #14329: KAFKA-15410: Delete topic integration test with LocalTieredStorage and TBRLMM (3/4)

2023-09-03 Thread via GitHub
kamalcph commented on code in PR #14329: URL: https://github.com/apache/kafka/pull/14329#discussion_r1314264284 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -379,9 +379,9 @@ public void stopPartitions(Set topicPartitions,

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314219340 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -428,9 +430,44 @@ public CompletableFuture listGroups(

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314218928 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -309,6 +311,21 @@ public CoordinatorResult commitOffset(

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-03 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1314219391 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -428,9 +430,44 @@ public CompletableFuture listGroups(

[GitHub] [kafka] kamalcph opened a new pull request, #14328: KAFKA-15410: Reassign replica expand, move and shrink integration tests.

2023-09-03 Thread via GitHub
kamalcph opened a new pull request, #14328: URL: https://github.com/apache/kafka/pull/14328 - Updated the log-start-offset to correct value while building the replica state in ReplicaFetcherTierStateMachine#buildRemoteLogAuxState This is a stacked PR on top of #14307 ###

[GitHub] [kafka] kamalcph opened a new pull request, #14330: KAFKA-15410: Delete segments due to log-start-offset breach integration test (4/4)

2023-09-03 Thread via GitHub
kamalcph opened a new pull request, #14330: URL: https://github.com/apache/kafka/pull/14330 Added the integration test for `DELETE_RECORDS` API for tiered storage enabled topic. The test currently in the failed state. ### Committer Checklist (excluded from commit message) - [ ]

[GitHub] [kafka] kamalcph commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-02 Thread via GitHub
kamalcph commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1314112162 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -776,6 +777,8 @@ public void run() { if (!isCancelled()) {

[GitHub] [kafka] showuon commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-03 Thread via GitHub
showuon commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1704045285 Failed tests are unrelated: ``` Build / JDK 20 and Scala 2.13 /

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-03 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1314155819 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -776,6 +777,8 @@ public void run() { if (!isCancelled()) {

[GitHub] [kafka] showuon commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-03 Thread via GitHub
showuon commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1704046955 > For my understanding, the issue was from tests disabled previously (ie, not the 2 we just disabled?) And if so can we re-enable the 2 we just disabled while investigating? Yes,

[GitHub] [kafka] ableegoldman merged pull request #14325: KAFKA-15429: catch+log errors from unsubscribe in streamthread shutdown

2023-09-02 Thread via GitHub
ableegoldman merged PR #14325: URL: https://github.com/apache/kafka/pull/14325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] ableegoldman commented on pull request #14325: KAFKA-15429: catch+log errors from unsubscribe in streamthread shutdown

2023-09-02 Thread via GitHub
ableegoldman commented on PR #14325: URL: https://github.com/apache/kafka/pull/14325#issuecomment-1703978353 Test failures are unrelated. Merging to trunk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] showuon commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-03 Thread via GitHub
showuon commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1704047268 > Did you find something that we can do in our test framework that can help us investigate such leaks quickly? Yes, there should be something we can do to improve it. Let me think

[GitHub] [kafka] satishd commented on pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-02 Thread via GitHub
satishd commented on PR #14113: URL: https://github.com/apache/kafka/pull/14113#issuecomment-1704007735 @kamalcph @showuon Please look into the replies from @abhijeetk88 on your respective comments. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] showuon commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-03 Thread via GitHub
showuon commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1314161952 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,62 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[GitHub] [kafka] showuon merged pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-03 Thread via GitHub
showuon merged PR #14320: URL: https://github.com/apache/kafka/pull/14320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] showuon commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-03 Thread via GitHub
showuon commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1704045909 Backported into 3.6 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] kamalcph commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-02 Thread via GitHub
kamalcph commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1314112162 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -776,6 +777,8 @@ public void run() { if (!isCancelled()) {

[GitHub] [kafka] mumrah opened a new pull request, #14342: KAFKA-15435 Fix counts in MigrationManifest

2023-09-05 Thread via GitHub
mumrah opened a new pull request, #14342: URL: https://github.com/apache/kafka/pull/14342 This patch fixes an issue in MigrationManifest where we weren't computing the record counts properly. A unit test was added to verify the fix. This patch also changes the counters in

[GitHub] [kafka] yashmayya commented on pull request #14337: MINOR: Update the documentation's table of contents to add missing headings for Kafka Connect

2023-09-05 Thread via GitHub
yashmayya commented on PR #14337: URL: https://github.com/apache/kafka/pull/14337#issuecomment-1707112259 Thanks for the review and suggestion Chris! I've raised https://github.com/apache/kafka-site/pull/538 to port over these changes. -- This is an automated message from the Apache Git

[GitHub] [kafka] C0urante commented on pull request #14336: KAFKA-14876: Add stopped state to Kafka Connect Administration docs section

2023-09-05 Thread via GitHub
C0urante commented on PR #14336: URL: https://github.com/apache/kafka/pull/14336#issuecomment-1707142324 BTW, same thought about backporting--I'm going to cherry-pick these changes on the `apache/kafka` repo onto the 3.5 and 3.6 branches, and for them to show up immediately, feel free to

[GitHub] [kafka] tylerbertrand opened a new pull request, #14344: Resolve checkstyle cache miss

2023-09-05 Thread via GitHub
tylerbertrand opened a new pull request, #14344: URL: https://github.com/apache/kafka/pull/14344 Resolves cache misses in `checkstyle` tasks due to absolute paths in `configProperties`. Sets `configDirectory` extension property, which is made available by the `checkstyle` plugin as

[GitHub] [kafka] philipnee commented on a diff in pull request #14313: KAFKA-15416: Fix flaky TopicAdminTest::retryEndOffsetsShouldRetryWhenTopicNotFound test case

2023-09-05 Thread via GitHub
philipnee commented on code in PR #14313: URL: https://github.com/apache/kafka/pull/14313#discussion_r1316332331 ## connect/runtime/src/main/java/org/apache/kafka/connect/util/TopicAdmin.java: ## @@ -286,11 +286,16 @@ public static NewTopicBuilder defineTopic(String topicName)

[GitHub] [kafka] jolshan commented on a diff in pull request #14314: KAFKA-15425: Fail fast in Admin::listOffsets when topic (but not partition) metadata is not found

2023-09-05 Thread via GitHub
jolshan commented on code in PR #14314: URL: https://github.com/apache/kafka/pull/14314#discussion_r1316414818 ## clients/src/main/java/org/apache/kafka/clients/admin/internals/PartitionLeaderStrategy.java: ## @@ -72,6 +79,12 @@ private void handleTopicError( ) {

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-05 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1316429235 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,393 @@ +/* + * Licensed to the

[GitHub] [kafka] C0urante commented on pull request #14337: MINOR: Update the documentation's table of contents to add missing headings for Kafka Connect

2023-09-05 Thread via GitHub
C0urante commented on PR #14337: URL: https://github.com/apache/kafka/pull/14337#issuecomment-1707068710 I've backported this to the 3.5 and 3.6 branches; it should be reflected once a new release is put out on those. @yashmayya if you'd like, feel free to file a PR against

[GitHub] [kafka] C0urante commented on a diff in pull request #14337: MINOR: Update the documentation's table of contents to add missing headings for Kafka Connect

2023-09-05 Thread via GitHub
C0urante commented on code in PR #14337: URL: https://github.com/apache/kafka/pull/14337#discussion_r1316159088 ## docs/toc.html: ## @@ -206,6 +206,14 @@ Plugin Discovery 8.3 Connector Development Guide +

[GitHub] [kafka] yashmayya commented on pull request #14336: KAFKA-14876: Add stopped state to Kafka Connect Administration docs section

2023-09-05 Thread via GitHub
yashmayya commented on PR #14336: URL: https://github.com/apache/kafka/pull/14336#issuecomment-1707093218 Ah whoops, that was not intentional. Good catch, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] mumrah commented on a diff in pull request #14342: KAFKA-15435 Fix counts in MigrationManifest

2023-09-05 Thread via GitHub
mumrah commented on code in PR #14342: URL: https://github.com/apache/kafka/pull/14342#discussion_r1316249748 ## metadata/src/main/java/org/apache/kafka/metadata/migration/MigrationManifest.java: ## @@ -60,7 +61,8 @@ public MigrationManifest build() { if

[GitHub] [kafka] C0urante commented on a diff in pull request #14309: KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters

2023-09-05 Thread via GitHub
C0urante commented on code in PR #14309: URL: https://github.com/apache/kafka/pull/14309#discussion_r1316338908 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java: ## @@ -380,6 +387,110 @@ protected Map

[GitHub] [kafka] C0urante commented on pull request #14309: KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters

2023-09-05 Thread via GitHub
C0urante commented on PR #14309: URL: https://github.com/apache/kafka/pull/14309#issuecomment-1707264425 > I wonder if we can unify these approaches, and maybe even use the "enrich" patten for producer/consumer/admin instead of the "merge" style. I honestly find the

[GitHub] [kafka] junrao commented on a diff in pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-05 Thread via GitHub
junrao commented on code in PR #14305: URL: https://github.com/apache/kafka/pull/14305#discussion_r1316404620 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -363,6 +365,20 @@ private CompletableFuture> chainFuture(fi

  1   2   3   4   5   6   7   8   9   10   >