Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364998096 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364998096 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364998096 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364998096 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15644: Fix CVE-2023-4586 in netty:handler [kafka]

2023-10-18 Thread via GitHub
atu-sharm commented on PR #14584: URL: https://github.com/apache/kafka/pull/14584#issuecomment-1770161791 Hi @mimaison can you please review this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[PR] KAFKA-15644: Fix CVE-2023-4586 in netty:handler [kafka]

2023-10-18 Thread via GitHub
atu-sharm opened a new pull request, #14584: URL: https://github.com/apache/kafka/pull/14584 MInor Update: Upgrading Netty to 4.1.100.Final to remove CVE-2023-4586 Ref: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-4586 -- This is an automated message from the Apache Git

[jira] [Created] (KAFKA-15644) Fix CVE-2023-4586 in netty:handler

2023-10-18 Thread Atul Sharma (Jira)
Atul Sharma created KAFKA-15644: --- Summary: Fix CVE-2023-4586 in netty:handler Key: KAFKA-15644 URL: https://issues.apache.org/jira/browse/KAFKA-15644 Project: Kafka Issue Type: Bug

[jira] [Assigned] (KAFKA-15610) Fix `CoreUtils.swallow()` test gaps

2023-10-18 Thread Atul Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Atul Sharma reassigned KAFKA-15610: --- Assignee: Atul Sharma > Fix `CoreUtils.swallow()` test gaps > -

Re: [PR] KAFKA-13988: Enable replicating from latest offset with MirrorMaker 2 [kafka]

2023-10-18 Thread via GitHub
fvaleri commented on code in PR #14567: URL: https://github.com/apache/kafka/pull/14567#discussion_r1364960461 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceTask.java: ## @@ -266,7 +262,29 @@ private Map loadOffsets(Set topicPartition private

Re: [PR] KAFKA-15610: Fix `CoreUtils.swallow()` test gaps [kafka]

2023-10-18 Thread via GitHub
atu-sharm commented on PR #14583: URL: https://github.com/apache/kafka/pull/14583#issuecomment-1770123147 Hi, @ijuma @divijvaidya can you please review this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] KAFKA-15610: Fix `CoreUtils.swallow()` test gaps [kafka]

2023-10-18 Thread via GitHub
atu-sharm opened a new pull request, #14583: URL: https://github.com/apache/kafka/pull/14583 Adding test to check that the passed in `logging` is used in case of an exception for CoreUtils.swallow. https://github.com/apache/kafka/pull/14529#discussion_r1355277747. -- Th

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
showuon commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364955572 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count = 1

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
showuon commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364955572 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count = 1

[jira] [Commented] (KAFKA-15610) Fix `CoreUtils.swallow()` test gaps

2023-10-18 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1044#comment-1044 ] Ismael Juma commented on KAFKA-15610: - Yes. > Fix `CoreUtils.swallow()` test gaps >

[jira] [Commented] (KAFKA-15610) Fix `CoreUtils.swallow()` test gaps

2023-10-18 Thread Atul Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1038#comment-1038 ] Atul Sharma commented on KAFKA-15610: - Hi [~ijuma], can i take this? > Fix `CoreUt

[jira] [Created] (KAFKA-15643) Improve unloading logging

2023-10-18 Thread David Jacot (Jira)
David Jacot created KAFKA-15643: --- Summary: Improve unloading logging Key: KAFKA-15643 URL: https://issues.apache.org/jira/browse/KAFKA-15643 Project: Kafka Issue Type: Sub-task Repo

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1364913876 ## clients/src/main/java/org/apache/kafka/common/telemetry/collector/MetricsCollector.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[jira] [Updated] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

2023-10-18 Thread Hanyu Zheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanyu Zheng updated KAFKA-15629: Summary: proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery (w

[jira] [Updated] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimeStampedKeyQuery and TimeStampedRangeQuery

2023-10-18 Thread Hanyu Zheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanyu Zheng updated KAFKA-15629: Summary: proposal to introduce IQv2 Query Types: TimeStampedKeyQuery and TimeStampedRangeQuery (w

[PR] KAFKA-15578: Migrating other system tests to use the group coordinator [kafka]

2023-10-18 Thread via GitHub
rreddy-22 opened a new pull request, #14582: URL: https://github.com/apache/kafka/pull/14582 Adding the use new coordinator flag to all the decorators in all the existing tests that use consumer groups. The new group coordinator cannot be used with zookeeper so this combination will not

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1364873194 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1364874333 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetryReceiver.java: ## @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1364873194 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetrySender.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-18 Thread via GitHub
CalvinConfluent commented on PR #14465: URL: https://github.com/apache/kafka/pull/14465#issuecomment-1769903140 @junrao Thanks for the review, no relevant tests failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-18 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1364872040 ## metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java: ## @@ -327,6 +328,10 @@ public ControllerResult registerBroker( }

[jira] [Updated] (KAFKA-15616) Define client telemetry states and their transitions

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15616: Fix Version/s: 3.7.0 > Define client telemetry states and their transitions >

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax merged PR #14566: URL: https://github.com/apache/kafka/pull/14566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14566: URL: https://github.com/apache/kafka/pull/14566#issuecomment-1769879629 Again a lot of flaky test noise: ``` Build / JDK 11 and Scala 2.13 / testFollowerCompleteDelayedFetchesOnReplication(String).quorum=kraft – integration.kafka.server.FetchFromFollowerI

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14566: URL: https://github.com/apache/kafka/pull/14566#discussion_r1364865721 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetryState.java: ## @@ -0,0 +1,166 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on PR #14532: URL: https://github.com/apache/kafka/pull/14532#issuecomment-1769876947 Hi @cadonna Thank you for spending time reviewing the PR again. I have answered your comments, and attached a followup ticket and a minor PR: https://github.com/apache/kafka/pull/14581

[PR] MINOR: Simplify UnsentRequest constructor [kafka]

2023-10-18 Thread via GitHub
philipnee opened a new pull request, #14581: URL: https://github.com/apache/kafka/pull/14581 Per: https://github.com/apache/kafka/pull/14532 The constructor of providing a BiConsumer seems confusing. Instead, here I provide a whenComplete() interface to chain the user provided callba

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-18 Thread via GitHub
ijuma commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1364857293 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Commented] (KAFKA-15620) Duplicate remote log DELETE_SEGMENT metadata is generated when there are multiple leader epochs in the segment

2023-10-18 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1004#comment-1004 ] Henry Cai commented on KAFKA-15620: --- Yes, non-blocking warning in the log.  But it als

Re: [PR] KAFKA-15626: Replace verification guard object with an specific type [kafka]

2023-10-18 Thread via GitHub
ijuma commented on code in PR #14568: URL: https://github.com/apache/kafka/pull/14568#discussion_r1364857293 ## storage/src/main/java/org/apache/kafka/storage/internals/log/VerificationGuard.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14566: URL: https://github.com/apache/kafka/pull/14566#discussion_r1364857199 ## clients/src/main/java/org/apache/kafka/common/errors/IllegalClientTelemetryStateException.java: ## @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14566: URL: https://github.com/apache/kafka/pull/14566#discussion_r1364856191 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryState.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364855206 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
iit2009060 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364855206 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,84 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count

Re: [PR] KAFKA-15616: Client telemetry states and transition (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14566: URL: https://github.com/apache/kafka/pull/14566#discussion_r1364855062 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryState.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

[jira] [Commented] (KAFKA-15620) Duplicate remote log DELETE_SEGMENT metadata is generated when there are multiple leader epochs in the segment

2023-10-18 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776999#comment-17776999 ] Kamal Chandraprakash commented on KAFKA-15620: -- [~h...@pinterest.com] What

[jira] [Commented] (KAFKA-15620) Duplicate remote log DELETE_SEGMENT metadata is generated when there are multiple leader epochs in the segment

2023-10-18 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776996#comment-17776996 ] Henry Cai commented on KAFKA-15620: --- Thanks.  Will give it a try. > Duplicate remote

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364845881 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -188,9 +188,7 @@ private NetworkClientDelegate.UnsentRequest

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-18 Thread via GitHub
showuon commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1364844466 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -672,16 +673,89 @@ class RemoteIndexCacheTest { verifyFetchIndexInvocation(count = 1

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax merged PR #14560: URL: https://github.com/apache/kafka/pull/14560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364842391 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManagerTest.java: ## @@ -151,6 +152,36 @@ void testHardFailures(Exception exc

[jira] [Created] (KAFKA-15642) Ensure offset fetcher behaves correctly when the request is Timeout

2023-10-18 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15642: -- Summary: Ensure offset fetcher behaves correctly when the request is Timeout Key: KAFKA-15642 URL: https://issues.apache.org/jira/browse/KAFKA-15642 Project: Kafka

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14560: URL: https://github.com/apache/kafka/pull/14560#issuecomment-1769844494 The Jenkins UI is a mess -- you can click on "Tests" in the top menu bar to see failing tests. ``` New failing - 16 Build / JDK 8 and Scala 2.12 / testTimeouts() – org.apache.k

Re: [PR] KAFKA-15604: Telemetry API request and response schemas and classes (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14554: URL: https://github.com/apache/kafka/pull/14554#issuecomment-1769842128 Triggered a Jenkins re-run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] KAFKA-15604: Telemetry API request and response schemas and classes (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14554: URL: https://github.com/apache/kafka/pull/14554#discussion_r1364838008 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -6222,4 +6222,62 @@ class KafkaApisTest { assertEquals(expectedResponse, response.data) } +

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-18 Thread via GitHub
hanyuzheng7 commented on code in PR #14477: URL: https://github.com/apache/kafka/pull/14477#discussion_r1364836021 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1586,12 +1593,15 @@ public void shouldHandleKeyQuery( publ

Re: [PR] KAFKA-14133: Migrate StateDirectory mock in TaskManagerTest to Mockito [kafka]

2023-10-18 Thread via GitHub
github-actions[bot] commented on PR #13897: URL: https://github.com/apache/kafka/pull/13897#issuecomment-1769836677 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] KAFKA-15604: Telemetry API request and response schemas and classes (KIP-714) [kafka]

2023-10-18 Thread via GitHub
mjsax commented on code in PR #14554: URL: https://github.com/apache/kafka/pull/14554#discussion_r1364836200 ## clients/src/main/java/org/apache/kafka/common/requests/GetTelemetrySubscriptionsRequest.java: ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-18 Thread via GitHub
hanyuzheng7 commented on code in PR #14477: URL: https://github.com/apache/kafka/pull/14477#discussion_r1364836021 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1586,12 +1593,15 @@ public void shouldHandleKeyQuery( publ

Re: [PR] Kip951 poc [kafka]

2023-10-18 Thread via GitHub
github-actions[bot] commented on PR #14063: URL: https://github.com/apache/kafka/pull/14063#issuecomment-1769836600 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[jira] [Resolved] (KAFKA-15620) Duplicate remote log DELETE_SEGMENT metadata is generated when there are multiple leader epochs in the segment

2023-10-18 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15620. --- Fix Version/s: 3.7.0 Resolution: Duplicate > Duplicate remote log DELETE_SEGMENT metadata is

[jira] [Commented] (KAFKA-15620) Duplicate remote log DELETE_SEGMENT metadata is generated when there are multiple leader epochs in the segment

2023-10-18 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776989#comment-17776989 ] Luke Chen commented on KAFKA-15620: --- [~ckamal], thanks for the info. [~h...@pinterest.

[jira] [Updated] (KAFKA-15479) Remote log segments should be considered once for retention breach

2023-10-18 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15479: -- Fix Version/s: 3.6.1 > Remote log segments should be considered once for retention breach > --

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364833452 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManagerTest.java: ## @@ -151,6 +152,36 @@ void testHardFailures(Exception exc

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-18 Thread via GitHub
hanyuzheng7 commented on code in PR #14477: URL: https://github.com/apache/kafka/pull/14477#discussion_r1364832374 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1586,12 +1593,15 @@ public void shouldHandleKeyQuery( publ

[jira] [Updated] (KAFKA-13973) block-cache-capacity metrics worth twice as much as normal

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-13973: Fix Version/s: 3.6.1 > block-cache-capacity metrics worth twice as much as normal > --

Re: [PR] KAFKA-13973: Fix inflated block cache metrics [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14317: URL: https://github.com/apache/kafka/pull/14317#issuecomment-1769829743 Cherry-picked this to `3.6` branch. Also tries to cherry-pick to `3.5` but the newly added test fails on 3.5 -- not sure why yet. -- This is an automated message from the Apache Git Servi

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-18 Thread via GitHub
hanyuzheng7 commented on PR #14477: URL: https://github.com/apache/kafka/pull/14477#issuecomment-1769829820 > LGTM. Two nits to improve variable names. > > As discussed in person, we also should update JavaDocs for `RangeQuery`, and `ReadOnlyKeyValueStore` to make it more explicit wha

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364830057 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManagerTest.java: ## @@ -151,6 +152,36 @@ void testHardFailures(Exception exc

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364829502 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -254,14 +254,14 @@ public NetworkClientDelegate.UnsentRequest t

[jira] [Updated] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-18 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15562: --- Description: Both fetchOffsetRequest and commitOffsetRequest handlers don't have sufficient logic t

[jira] [Updated] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-18 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15562: --- Summary: Ensure fetch offset and commit offset handler handles both timeout and various error types

[jira] [Updated] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-18 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15562: --- Description: (was: Review the code in Consumercoordinator#OffsetCommitResponseHandler  Implemen

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-18 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1364820542 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -256,27 +256,39 @@ public String toString() { public sta

Re: [PR] Worker sink task threaded test mockito [kafka]

2023-10-18 Thread via GitHub
hgeraldino closed pull request #14580: Worker sink task threaded test mockito URL: https://github.com/apache/kafka/pull/14580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[PR] Worker sink task threaded test mockito [kafka]

2023-10-18 Thread via GitHub
hgeraldino opened a new pull request, #14580: URL: https://github.com/apache/kafka/pull/14580 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of test

Re: [PR] wip: investigate Flaky tests [kafka]

2023-10-18 Thread via GitHub
dengziming closed pull request #14512: wip: investigate Flaky tests URL: https://github.com/apache/kafka/pull/14512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] KAFKA-13988: Enable replicating from latest offset with MirrorMaker 2 [kafka]

2023-10-18 Thread via GitHub
hudeqi commented on PR #14567: URL: https://github.com/apache/kafka/pull/14567#issuecomment-1769788407 LGTM, thanks for this PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] KAFKA-15566: Fix flaky tests in FetchRequestTest.scala in KRaft mode [kafka]

2023-10-18 Thread via GitHub
showuon commented on PR #14573: URL: https://github.com/apache/kafka/pull/14573#issuecomment-1769750941 @dengziming , do you want to have a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [WIP] [kafka]

2023-10-18 Thread via GitHub
pprovenzano commented on PR #14578: URL: https://github.com/apache/kafka/pull/14578#issuecomment-1769746613 Adding of the directory.id in any form will prevent the cluster from rolling back to a previous ZK version. -- This is an automated message from the Apache Git Service. To respond t

[jira] [Commented] (KAFKA-15479) Remote log segments should be considered once for retention breach

2023-10-18 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776960#comment-17776960 ] Henry Cai commented on KAFKA-15479: --- This is also related to https://issues.apache.org

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364758468 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3573 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Issue Type: Bug (was: Improvement) > Investigate CompletedFetch handleInitializeErrors for accuracy >

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15641: -- Description: The {{FetcherTest}} and {{FetchRequestManagerTest}} classes have a test named testFetche

[jira] [Updated] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15338: Affects Version/s: (was: 3.4.0) (was: 3.3.1)

[jira] [Created] (KAFKA-15641) Investigate CompletedFetch handleInitializeErrors for accuracy

2023-10-18 Thread Kirk True (Jira)
Kirk True created KAFKA-15641: - Summary: Investigate CompletedFetch handleInitializeErrors for accuracy Key: KAFKA-15641 URL: https://issues.apache.org/jira/browse/KAFKA-15641 Project: Kafka Iss

[jira] [Updated] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15338: Fix Version/s: 3.3.3 3.4.2 3.5.2 > The metric group

Re: [PR] KAFKA-15378: fix streams upgrade system test [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14539: URL: https://github.com/apache/kafka/pull/14539#issuecomment-1769719774 The following system tests failed: - test_upgrade_to_cooperative_rebalance - 0.10.1.1 - 0.10.2.2 - 1.0.2 - 1.1.1 - 2.0.1 - 2.3.1 - test_app_upgrade

Re: [PR] MINOR: add KIP-941 to Kafka Streams upgrade docs [kafka]

2023-10-18 Thread via GitHub
mjsax commented on PR #14577: URL: https://github.com/apache/kafka/pull/14577#issuecomment-1769695164 Merged to `trunk` and cherry-picked to `3.6` branch. Also ported to `kafka-site`: https://github.com/apache/kafka-site/pull/563 (merged right away). -- This is an automated message

Re: [PR] MINOR: add KIP-941 to Kafka Streams upgrade docs [kafka]

2023-10-18 Thread via GitHub
mjsax merged PR #14577: URL: https://github.com/apache/kafka/pull/14577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Updated] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimeStampKeyQuery and TimeStampRangeQuery

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15629: Description: KIP-992: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-992%3A+Propo

[jira] [Updated] (KAFKA-15629) proposal to introduce IQv2 Query Types: TimeStampKeyQuery and TimeStampRangeQuery

2023-10-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15629: Labels: kip (was: ) > proposal to introduce IQv2 Query Types: TimeStampKeyQuery and > Ti

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364715414 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java: ## @@ -162,7 +165,9 @@ private Fetch fetchRecords(final CompletedFetch nextI

[jira] [Updated] (KAFKA-15640) Refactor CompletedFetch initialization

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15640: -- Description: The interaction between {{{}FetchBuffer{}}}, {{{}FetchCollector{}}}, and {{CompletedFetc

[jira] [Updated] (KAFKA-15640) Refactor CompletedFetch initialization

2023-10-18 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15640: -- Priority: Major (was: Minor) > Refactor CompletedFetch initialization > -

[jira] [Created] (KAFKA-15640) Refactor CompletedFetch initialization

2023-10-18 Thread Kirk True (Jira)
Kirk True created KAFKA-15640: - Summary: Refactor CompletedFetch initialization Key: KAFKA-15640 URL: https://issues.apache.org/jira/browse/KAFKA-15640 Project: Kafka Issue Type: Improvement

[PR] MINOR: Don't log missing partitions at ERROR level [kafka]

2023-10-18 Thread via GitHub
splett2 opened a new pull request, #14579: URL: https://github.com/apache/kafka/pull/14579 ### What We are logging missing partitions as errors in the `state.change.logger` which may be a bit alarming. There are valid cases where the partition may not exist, eg: partition reassignment.

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364712500 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -35,19 +41,24 @@ * * * - * Note: this class is not thread-safe and

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364707771 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3533 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364706025 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3541 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-18 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1364704339 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThreadTest.java: ## @@ -0,0 +1,270 @@ +/* + * Licensed to the Apache Software Found

  1   2   3   >