[jira] [Updated] (KAFKA-15044) Snappy v.1.1.9.1 NoClassDefFound on ARM machines

2023-06-01 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15044: -- Fix Version/s: 3.5.0 > Snappy v.1.1.9.1 NoClassDefFound on ARM machines >

[GitHub] [kafka] cadonna merged pull request #13774: KAFKA-12562: Fix KafkaStreams#store old references in comments

2023-06-01 Thread via GitHub
cadonna merged PR #13774: URL: https://github.com/apache/kafka/pull/13774 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Updated] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-15046: - Description: * Phenomenon: ** !image-2023-06-01-12-46-30-058.png|width=259,height=236! ** Prod

[GitHub] [kafka] showuon commented on a diff in pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
showuon commented on code in PR #13790: URL: https://github.com/apache/kafka/pull/13790#discussion_r1212741250 ## .asf.yaml: ## @@ -25,28 +25,29 @@ notifications: # https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA&title=Git+-+.asf.yaml+features#Git.asf.

[GitHub] [kafka] jlprat commented on pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
jlprat commented on PR #13790: URL: https://github.com/apache/kafka/pull/13790#issuecomment-1571555084 We could just do the reordering, to see if the invites are sent again (for the 2 that didn't accept in time) -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [kafka] showuon closed pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
showuon closed pull request #13790: MINOR: update collaborator (v3.4.1 release) URL: https://github.com/apache/kafka/pull/13790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] cadonna commented on pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
cadonna commented on PR #13790: URL: https://github.com/apache/kafka/pull/13790#issuecomment-1571560100 I would also opt for major releases. Minor releases are usually done around major releases and sometimes we also do multiple minor releases around a major release. So we would update the

[GitHub] [kafka] jlprat commented on pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
jlprat commented on PR #13790: URL: https://github.com/apache/kafka/pull/13790#issuecomment-1571562621 @showuon we could still do the reordering to see if the invites are sent again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [kafka] showuon commented on pull request #13790: MINOR: update collaborator (v3.4.1 release)

2023-06-01 Thread via GitHub
showuon commented on PR #13790: URL: https://github.com/apache/kafka/pull/13790#issuecomment-1571574041 > @showuon we could still do the reordering to see if the invites are sent again Let's give it a try! -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-06-01 Thread via GitHub
Hangleton commented on code in PR #13584: URL: https://github.com/apache/kafka/pull/13584#discussion_r1212777323 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -65,6 +68,32 @@ class LogSegmentTest { Utils.delete(logDir) } + /** + * LogSegmentOffse

[GitHub] [kafka] Hangleton commented on pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-06-01 Thread via GitHub
Hangleton commented on PR #13584: URL: https://github.com/apache/kafka/pull/13584#issuecomment-1571584730 @showuon Thanks for the follow-up and apologies for the delay, the PR seems good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728260#comment-17728260 ] Divij Vaidya commented on KAFKA-15046: -- Thank you for the investigation folks. We h

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728262#comment-17728262 ] Haruki Okada commented on KAFKA-15046: -- Oh I haven't noticed there's another ticket

[jira] [Updated] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-9693: Fix Version/s: 3.6.0 > Kafka latency spikes caused by log segment flush on roll > -

[jira] [Commented] (KAFKA-9693) Kafka latency spikes caused by log segment flush on roll

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728265#comment-17728265 ] Divij Vaidya commented on KAFKA-9693: - Asynchronously rolling producer snapshot looks

[jira] [Updated] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15046: - Component/s: core > Produce performance issue under high disk load > ---

[jira] [Updated] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15046: - Labels: performance (was: ) > Produce performance issue under high disk load >

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728266#comment-17728266 ] Haruki Okada commented on KAFKA-15046: -- Hm, when I dug into further this, I noticed

[jira] [Comment Edited] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728266#comment-17728266 ] Haruki Okada edited comment on KAFKA-15046 at 6/1/23 8:39 AM:

[GitHub] [kafka] divijvaidya commented on pull request #13786: KAFKA-15044: Upgrade to snappy v1.1.10.0

2023-06-01 Thread via GitHub
divijvaidya commented on PR #13786: URL: https://github.com/apache/kafka/pull/13786#issuecomment-1571610910 As a thought exercise, could we have detected this ARM specific bug if we were running our tests on ARM machines? What requirement do we have to enable testing on ARM again? -- Thi

[GitHub] [kafka] kamalcph commented on pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
kamalcph commented on PR #13561: URL: https://github.com/apache/kafka/pull/13561#issuecomment-1571615955 @satishd Will you open a separate PR to delete the active segment once it breaches the retention time? Or, will handle it in this patch. -- This is an automated message from th

[GitHub] [kafka] mimaison opened a new pull request, #13791: MINOR: Update LICENSE-binary following snappy upgrade

2023-06-01 Thread via GitHub
mimaison opened a new pull request, #13791: URL: https://github.com/apache/kafka/pull/13791 To be backported to 3.5 too ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify do

[GitHub] [kafka] showuon merged pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-06-01 Thread via GitHub
showuon merged PR #13584: URL: https://github.com/apache/kafka/pull/13584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728275#comment-17728275 ] Divij Vaidya commented on KAFKA-15046: -- Yes that is right, leaderEpochCheckpoint is

[GitHub] [kafka] mimaison merged pull request #13791: MINOR: Update LICENSE-binary following snappy upgrade

2023-06-01 Thread via GitHub
mimaison merged PR #13791: URL: https://github.com/apache/kafka/pull/13791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-15046) Produce performance issue under high disk load

2023-06-01 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-15046: - Description: * Phenomenon: ** !image-2023-06-01-12-46-30-058.png|width=259,height=236! ** Prod

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212931402 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -98,8 +97,7 @@ private final Metrics metrics; private f

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212936370 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -130,20 +136,20 @@ public PrototypeAsyncConsumer(final ConsumerCo

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212938962 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -85,7 +87,6 @@ private static final String CLIENT_ID_METRIC_

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212945606 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -96,6 +97,10 @@ private final Metrics metrics; private

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212948810 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -96,6 +97,10 @@ private final Metrics metrics; private

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212959825 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -114,14 +119,14 @@ public PrototypeAsyncConsumer(final ConsumerCo

[GitHub] [kafka] mimaison opened a new pull request, #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison opened a new pull request, #13792: URL: https://github.com/apache/kafka/pull/13792 Moved the ZK and KRaft steps into `h5` sections so they are below the `h4` Upgrading to 3.5.0 from any version 0.8.x through 3.4.x section. ### Committer Checklist (excluded from commit message

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212959825 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -114,14 +119,14 @@ public PrototypeAsyncConsumer(final ConsumerCo

[GitHub] [kafka] mimaison commented on pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison commented on PR #13792: URL: https://github.com/apache/kafka/pull/13792#issuecomment-1571810081 @tombentley @dajac Can you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] andymg3 commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
andymg3 commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1212978727 ## metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java: ## @@ -33,6 +33,92 @@ public class PartitionRegistration { + +/** + * A bui

[GitHub] [kafka] andymg3 commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
andymg3 commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1212980304 ## metadata/src/test/java/org/apache/kafka/controller/PartitionChangeBuilderTest.java: ## @@ -77,39 +77,75 @@ public void testChangeRecordIsNoOp() { ); }

[GitHub] [kafka] andymg3 commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
andymg3 commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1212980561 ## metadata/src/test/java/org/apache/kafka/controller/PartitionChangeBuilderTest.java: ## @@ -77,39 +77,75 @@ public void testChangeRecordIsNoOp() { ); }

[GitHub] [kafka] gr-m-9 commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-06-01 Thread via GitHub
gr-m-9 commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1212990392 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -447,6 +482,19 @@ public void close() { @Override pub

[jira] [Created] (KAFKA-15047) Handle rolling segments when the active segment's retention is breached incase of tiered storage is enabled.

2023-06-01 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15047: -- Summary: Handle rolling segments when the active segment's retention is breached incase of tiered storage is enabled. Key: KAFKA-15047 URL: https://issues.apache.org/jira/brow

[GitHub] [kafka] Hangleton commented on pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-06-01 Thread via GitHub
Hangleton commented on PR #13240: URL: https://github.com/apache/kafka/pull/13240#issuecomment-1571989946 Hello David (@dajac), I was discussing this with Christo today as part of his work on the OffsetFetch API. Would you like this PR on OffsetCommit to be split to make the review easier a

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213100593 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +629,193 @@ public void run() { } } +public void handleLogSta

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213102668 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +629,193 @@ public void run() { } } +public void handleLogSta

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213107255 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +629,193 @@ public void run() { } } +public void handleLogSta

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213107956 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +629,193 @@ public void run() { } } +public void handleLogSta

[GitHub] [kafka] dajac opened a new pull request, #13793: KAFKA-14462; [15/N] Make Result generic and rename it

2023-06-01 Thread via GitHub
dajac opened a new pull request, #13793: URL: https://github.com/apache/kafka/pull/13793 As the title says. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation

[GitHub] [kafka] dajac opened a new pull request, #13794: KAFKA-14462; [16/N] Add CoordinatorLoader and Replayable interfaces

2023-06-01 Thread via GitHub
dajac opened a new pull request, #13794: URL: https://github.com/apache/kafka/pull/13794 As the title says. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213112946 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -154,16 +154,41 @@ class UnifiedLog(@volatile var logStartOffset: Long, locally { initializePartitionMe

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-06-01 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1213113458 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1383,20 +1421,30 @@ class UnifiedLog(@volatile var logStartOffset: Long, } } + private[log] def loc

[GitHub] [kafka] dajac opened a new pull request, #13795: KAFKA-14462; [17/N] Add CoordinatorRuntime

2023-06-01 Thread via GitHub
dajac opened a new pull request, #13795: URL: https://github.com/apache/kafka/pull/13795 This PR includes temporary https://github.com/apache/kafka/pull/13666, https://github.com/apache/kafka/pull/13675, https://github.com/apache/kafka/pull/13793 and https://github.com/apache/kafka/pull/13

[GitHub] [kafka] hudeqi commented on pull request #13421: KAFKA-14824: ReplicaAlterLogDirsThread may cause serious disk growing in case of potential exception

2023-06-01 Thread via GitHub
hudeqi commented on PR #13421: URL: https://github.com/apache/kafka/pull/13421#issuecomment-1572025066 > No, you are correct, I had another look at the neighbouring code after I wrote my previous comment and I agree that this appears to not cause any leadership movements. Okay, I am happy t

[GitHub] [kafka] novosibman commented on pull request #13782: Suggest for performance fix: KAFKA-9693 Kafka latency spikes caused by log segment flush on roll - trunk version

2023-06-01 Thread via GitHub
novosibman commented on PR #13782: URL: https://github.com/apache/kafka/pull/13782#issuecomment-1572027303 > Are all the graphs shared for OMB and Kafka Tussle generated for Kafka with the fix in this PR? Graphs with the fix noted in first description comment - marked with `kafka_2.13-3

[jira] [Updated] (KAFKA-15047) Handle rolling segments when the active segment's retention is breached incase of tiered storage is enabled.

2023-06-01 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15047: --- Description: Active segments are not copied by remote storage subsystem. But they can be el

[GitHub] [kafka] dajac commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
dajac commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213163941 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorEventProcessor.java: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] dajac commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
dajac commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213165026 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
jeffkbkim commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213190623 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache So

[GitHub] [kafka] dajac commented on a diff in pull request #13769: MINOR: Covering all epoch cases in add partitions to txn manager

2023-06-01 Thread via GitHub
dajac commented on code in PR #13769: URL: https://github.com/apache/kafka/pull/13769#discussion_r1213186879 ## core/src/main/scala/kafka/server/AddPartitionsToTxnManager.scala: ## @@ -36,45 +36,58 @@ class TransactionDataAndCallbacks(val transactionData: AddPartitionsToTxnTran

[GitHub] [kafka] dajac commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
dajac commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213213734 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
jeffkbkim commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213191767 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorEvent.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
jeffkbkim commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213221009 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache So

[GitHub] [kafka] dajac commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
dajac commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213222820 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorEvent.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] dajac commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
dajac commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213232889 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [kafka] tombentley commented on a diff in pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
tombentley commented on code in PR #13792: URL: https://github.com/apache/kafka/pull/13792#discussion_r1213226340 ## docs/upgrade.html: ## @@ -59,6 +59,65 @@ Notable changes in 3 +Upgrading ZooKeeper-based clusters +If you are upgrading from a version

[GitHub] [kafka] mumrah merged pull request #13758: KAFKA-15010 ZK migration failover support

2023-06-01 Thread via GitHub
mumrah merged PR #13758: URL: https://github.com/apache/kafka/pull/13758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

[GitHub] [kafka] jsancio commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
jsancio commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1213240584 ## metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java: ## @@ -33,6 +33,92 @@ public class PartitionRegistration { + +/** + * A bui

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
jeffkbkim commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213254844 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache So

[GitHub] [kafka] urbandan opened a new pull request, #13796: KAFKA-14034 Idempotent producer should wait for preceding in-flight b…

2023-06-01 Thread via GitHub
urbandan opened a new pull request, #13796: URL: https://github.com/apache/kafka/pull/13796 …atches to complete before resetting the sequence number The idempotent producer resets the sequence number when a batch fails with a non-retriable error. This can violate the idempotent guaran

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213267178 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1019,13 +1029,14 @@ class ReplicaManager(val config: KafkaConfig, } /** - * Append the messages

[GitHub] [kafka] urbandan commented on pull request #13796: KAFKA-14034 Idempotent producer should wait for preceding in-flight b…

2023-06-01 Thread via GitHub
urbandan commented on PR #13796: URL: https://github.com/apache/kafka/pull/13796#issuecomment-1572188562 @viktorsomogyi Could you please review this fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213268934 ## core/src/main/scala/kafka/coordinator/group/PartitionWriterImpl.scala: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213271843 ## core/src/test/scala/unit/kafka/coordinator/group/PartitionWriterImplTest.scala: ## @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213273088 ## core/src/test/scala/unit/kafka/coordinator/group/PartitionWriterImplTest.scala: ## @@ -0,0 +1,225 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213274631 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordSerializer.java: ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] andymg3 commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
andymg3 commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1213276160 ## metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java: ## @@ -33,6 +33,92 @@ public class PartitionRegistration { + +/** + * A bui

[jira] [Resolved] (KAFKA-15017) New ClientQuotas are not written to ZK from snapshot

2023-06-01 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15017. --- Resolution: Fixed > New ClientQuotas are not written to ZK from snapshot >

[jira] [Commented] (KAFKA-15017) New ClientQuotas are not written to ZK from snapshot

2023-06-01 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728390#comment-17728390 ] Proven Provenzano commented on KAFKA-15017: --- Fix is on 3.5 branch. > New Clie

[GitHub] [kafka] mimaison commented on a diff in pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison commented on code in PR #13792: URL: https://github.com/apache/kafka/pull/13792#discussion_r1213332860 ## docs/upgrade.html: ## @@ -59,6 +59,65 @@ Notable changes in 3 +Upgrading ZooKeeper-based clusters +If you are upgrading from a version pr

[GitHub] [kafka] mimaison commented on a diff in pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison commented on code in PR #13792: URL: https://github.com/apache/kafka/pull/13792#discussion_r121808 ## docs/upgrade.html: ## @@ -59,6 +59,65 @@ Notable changes in 3 +Upgrading ZooKeeper-based clusters +If you are upgrading from a version pr

[GitHub] [kafka] jsancio commented on pull request #13765: KAFKA-15021; Skip leader epoch bump on ISR shrink

2023-06-01 Thread via GitHub
jsancio commented on PR #13765: URL: https://github.com/apache/kafka/pull/13765#issuecomment-1572275711 Thanks for your feedback @divijvaidya @dajac. I am replying to both your comments in you message. > With this change, we are changing the semantics of what a leadership epoch means. Pr

[GitHub] [kafka] mimaison commented on a diff in pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison commented on code in PR #13792: URL: https://github.com/apache/kafka/pull/13792#discussion_r1213336329 ## docs/upgrade.html: ## @@ -59,6 +59,65 @@ Notable changes in 3 +Upgrading ZooKeeper-based clusters +If you are upgrading from a version pr

[GitHub] [kafka] jsancio commented on a diff in pull request #13788: KAFKA-14791: Create a builder for PartitionRegistration

2023-06-01 Thread via GitHub
jsancio commented on code in PR #13788: URL: https://github.com/apache/kafka/pull/13788#discussion_r1213346871 ## metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java: ## @@ -33,6 +33,92 @@ public class PartitionRegistration { + +/** + * A bui

[GitHub] [kafka] mimaison merged pull request #13792: MINOR: Add 3.5 upgrade steps for ZK and KRaft

2023-06-01 Thread via GitHub
mimaison merged PR #13792: URL: https://github.com/apache/kafka/pull/13792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] dajac commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
dajac commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213267178 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1019,13 +1029,14 @@ class ReplicaManager(val config: KafkaConfig, } /** - * Append the messages

[GitHub] [kafka] jolshan commented on a diff in pull request #13769: MINOR: Covering all epoch cases in add partitions to txn manager

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13769: URL: https://github.com/apache/kafka/pull/13769#discussion_r1213389003 ## core/src/test/scala/unit/kafka/server/AddPartitionsToTxnManagerTest.scala: ## @@ -93,17 +93,22 @@ class AddPartitionsToTxnManagerTest { val transaction1AgainE

[GitHub] [kafka] jolshan commented on a diff in pull request #13769: MINOR: Covering all epoch cases in add partitions to txn manager

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13769: URL: https://github.com/apache/kafka/pull/13769#discussion_r1213389903 ## core/src/test/scala/unit/kafka/server/AddPartitionsToTxnManagerTest.scala: ## @@ -93,17 +93,22 @@ class AddPartitionsToTxnManagerTest { val transaction1AgainE

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13756: KAFKA-14936: Add On Disk Time Ordered Buffer (1/N)

2023-06-01 Thread via GitHub
vcrfxia commented on code in PR #13756: URL: https://github.com/apache/kafka/pull/13756#discussion_r1212267772 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimeOrderedKeyValueBuffer.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] philipnee opened a new pull request, #13797: KAFKA-14950: implement assign() and assignment()

2023-06-01 Thread via GitHub
philipnee opened a new pull request, #13797: URL: https://github.com/apache/kafka/pull/13797 In this PR: I implemented assign() and assignment(). Ported the original tests from the KafkaConsumerTest.java Different to the original Implementation: We will explicitly send a commit

[GitHub] [kafka] philipnee commented on pull request #13605: KAFKA-14950: implement assign() and assignment()

2023-06-01 Thread via GitHub
philipnee commented on PR #13605: URL: https://github.com/apache/kafka/pull/13605#issuecomment-1572469830 closingi this in light of https://github.com/apache/kafka/pull/13797 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] philipnee closed pull request #13605: KAFKA-14950: implement assign() and assignment()

2023-06-01 Thread via GitHub
philipnee closed pull request #13605: KAFKA-14950: implement assign() and assignment() URL: https://github.com/apache/kafka/pull/13605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [kafka] jolshan commented on a diff in pull request #13666: KAFKA-14462; [13/N] CoordinatorEvent and CoordinatorEventProcessor

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13666: URL: https://github.com/apache/kafka/pull/13666#discussion_r1213460495 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] jolshan commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213469954 ## core/src/main/scala/kafka/coordinator/group/PartitionWriterImpl.scala: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] jolshan commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213473971 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1019,13 +1029,14 @@ class ReplicaManager(val config: KafkaConfig, } /** - * Append the messag

[GitHub] [kafka] jolshan commented on a diff in pull request #13675: KAFKA-14462; [14/N] Add PartitionWriter

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13675: URL: https://github.com/apache/kafka/pull/13675#discussion_r1213474996 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordSerializer.java: ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] jolshan commented on pull request #13793: KAFKA-14462; [15/N] Make Result generic and rename it

2023-06-01 Thread via GitHub
jolshan commented on PR #13793: URL: https://github.com/apache/kafka/pull/13793#issuecomment-1572514821 Can we update the description to give the rationale for this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] dajac commented on pull request #13793: KAFKA-14462; [15/N] Make Result generic and rename it

2023-06-01 Thread via GitHub
dajac commented on PR #13793: URL: https://github.com/apache/kafka/pull/13793#issuecomment-1572521296 @jolshan Updated the description. This is basically a minor refactoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] jolshan commented on pull request #13794: KAFKA-14462; [16/N] Add CoordinatorLoader and Replayable interfaces

2023-06-01 Thread via GitHub
jolshan commented on PR #13794: URL: https://github.com/apache/kafka/pull/13794#issuecomment-1572526297 Let's add a brief description to explain the rationale for these interfaces to the description -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [kafka] jolshan commented on a diff in pull request #13794: KAFKA-14462; [16/N] Add CoordinatorLoader and Replayable interfaces

2023-06-01 Thread via GitHub
jolshan commented on code in PR #13794: URL: https://github.com/apache/kafka/pull/13794#discussion_r1213491922 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/Replayable.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] jsancio commented on a diff in pull request #13765: KAFKA-15021; Skip leader epoch bump on ISR shrink

2023-06-01 Thread via GitHub
jsancio commented on code in PR #13765: URL: https://github.com/apache/kafka/pull/13765#discussion_r1213501789 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1087,12 +1087,14 @@ class Partition(val topicPartition: TopicPartition, // avoid unnecessary collectio

[GitHub] [kafka] novosibman commented on pull request #13782: Suggest for performance fix: KAFKA-9693 Kafka latency spikes caused by log segment flush on roll - trunk version

2023-06-01 Thread via GitHub
novosibman commented on PR #13782: URL: https://github.com/apache/kafka/pull/13782#issuecomment-1572620634 Provided updated change: returned original try-with-resource on writing, added utility method for flushing: ``` try (FileChannel fileChannel = FileChannel.open(file.toP

[GitHub] [kafka] C0urante commented on a diff in pull request #13776: KAFKA-15034: Improve performance of the ReplaceField SMT; add JMH benchmark

2023-06-01 Thread via GitHub
C0urante commented on code in PR #13776: URL: https://github.com/apache/kafka/pull/13776#discussion_r1213569347 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/connect/ReplaceFieldBenchmark.java: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

  1   2   >