Re: [PR] KAFKA-16014: add RemoteLogMetadataCount metric [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15026: URL: https://github.com/apache/kafka/pull/15026#issuecomment-1858739478 @clolov @satishd , call for review. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] KAFKA-16014: add RemoteLogMetadataCount metric [kafka]

2023-12-15 Thread via GitHub
showuon opened a new pull request, #15026: URL: https://github.com/apache/kafka/pull/15026 Add `RemoteLogMetadataCount` count metric as described in KIP-963. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15005: URL: https://github.com/apache/kafka/pull/15005#issuecomment-1858738944 > The following metrics are related to each other: > > 1. RemoteCopyLagBytes and RemoteCopyLagSegments > > 2. RemoteDeleteLagBytes and RemoteDeleteLagSegments >

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-15 Thread via GitHub
philipnee commented on code in PR #15023: URL: https://github.com/apache/kafka/pull/15023#discussion_r1428703397 ## clients/src/test/java/org/apache/kafka/clients/MetadataTest.java: ## @@ -1071,18 +1071,25 @@ protected boolean retainTopic(String topic, boolean isInternal, long

[PR] KAFKA-16023: Disable flaky tests [kafka]

2023-12-15 Thread via GitHub
philipnee opened a new pull request, #15025: URL: https://github.com/apache/kafka/pull/15025 I observed several failed tests in PR builds. Let's first disable them and try to find a different way to test the async consumer with these tests. cc @dajac -- This is an automated

[jira] [Updated] (KAFKA-16023) PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding

2023-12-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-16023: --- Description: Several tests in PlaintextConsumerTest.scala (such as

[jira] [Updated] (KAFKA-16023) PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding

2023-12-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-16023: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Bug) > PlaintextConsumerTest needs to wait

[jira] [Created] (KAFKA-16023) PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16023: -- Summary: PlaintextConsumerTest needs to wait for reconciliation to complete before proceeding Key: KAFKA-16023 URL: https://issues.apache.org/jira/browse/KAFKA-16023

[jira] [Created] (KAFKA-16022) AsyncKafkaConsumer sometimes complains "No current assignment for partition {}"

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16022: -- Summary: AsyncKafkaConsumer sometimes complains "No current assignment for partition {}" Key: KAFKA-16022 URL: https://issues.apache.org/jira/browse/KAFKA-16022 Project:

[jira] [Updated] (KAFKA-16022) AsyncKafkaConsumer sometimes complains "No current assignment for partition {}"

2023-12-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-16022: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Bug) > AsyncKafkaConsumer sometimes

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
kamalcph commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428646042 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -926,16 +926,27 @@ private boolean deleteLogSegmentsDueToLeaderEpochCacheTruncation(EpochEntry

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
kamalcph commented on code in PR #15005: URL: https://github.com/apache/kafka/pull/15005#discussion_r1428638499 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1073,13 +1088,28 @@ void cleanupExpiredRemoteLogSegments() throws RemoteStorageException,

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
kamalcph commented on PR #15005: URL: https://github.com/apache/kafka/pull/15005#issuecomment-1858669208 The following metrics are related to each other: 1. RemoteCopyBytesLag and RemoteCopySegmentsLag 2. RemoteDeleteBytesLag and RemoteDeleteSegmentsLag We should update

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah merged PR #15007: URL: https://github.com/apache/kafka/pull/15007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah commented on PR #15007: URL: https://github.com/apache/kafka/pull/15007#issuecomment-1858638896 Test failures look unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #14375: URL: https://github.com/apache/kafka/pull/14375#issuecomment-1858633479 There is a compilation failure in java 8 scala 12. Please help fix it. https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14375/16/pipeline -- This

Re: [PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-15 Thread via GitHub
srdo commented on PR #15024: URL: https://github.com/apache/kafka/pull/15024#issuecomment-1858616582 I couldn't spot any testing that deals specifically with the behavior when the encoding can't be looked up. This change will alter that behavior so people will get a failure earlier during

[PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-15 Thread via GitHub
srdo opened a new pull request, #15024: URL: https://github.com/apache/kafka/pull/15024 The encoding for a StringSerializer doesn't change during the life of the serializer, so there isn't a good reason to look up the corresponding Charset every time a String needs to be serialized, which

[jira] [Created] (KAFKA-16021) Eagerly load Charset for StringSerializer/StringDeserializer

2023-12-15 Thread Jira
Stig Rohde Døssing created KAFKA-16021: -- Summary: Eagerly load Charset for StringSerializer/StringDeserializer Key: KAFKA-16021 URL: https://issues.apache.org/jira/browse/KAFKA-16021 Project:

Re: [PR] KAFKA-15215: docs for KIP-954 [kafka]

2023-12-15 Thread via GitHub
ableegoldman commented on code in PR #14949: URL: https://github.com/apache/kafka/pull/14949#discussion_r1428577334 ## docs/streams/upgrade-guide.html: ## @@ -185,6 +185,13 @@ Streams API + +default.dsl.store was replaced with

[PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-15 Thread via GitHub
philipnee opened a new pull request, #15023: URL: https://github.com/apache/kafka/pull/15023 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.16.0 [kafka]

2023-12-15 Thread via GitHub
bmscomp commented on code in PR #13662: URL: https://github.com/apache/kafka/pull/13662#discussion_r1428518449 ## connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java: ## @@ -143,7 +143,7 @@ public class JsonConverter implements Converter,

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.16.0 [kafka]

2023-12-15 Thread via GitHub
bmscomp commented on code in PR #13662: URL: https://github.com/apache/kafka/pull/13662#discussion_r1428518449 ## connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java: ## @@ -143,7 +143,7 @@ public class JsonConverter implements Converter,

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.16.0 [kafka]

2023-12-15 Thread via GitHub
bmscomp commented on code in PR #13662: URL: https://github.com/apache/kafka/pull/13662#discussion_r1428518449 ## connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java: ## @@ -143,7 +143,7 @@ public class JsonConverter implements Converter,

Re: [PR] MINOR: Minor update patch versions [kafka]

2023-12-15 Thread via GitHub
jolshan merged PR #14641: URL: https://github.com/apache/kafka/pull/14641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15366: Modify LogDirFailureTest for KRaft [kafka]

2023-12-15 Thread via GitHub
rondagostino commented on PR #14977: URL: https://github.com/apache/kafka/pull/14977#issuecomment-1858462616 @viktorsomogyi I'm seeing these `kafka.server.LogDirFailureTest ` failures, reproducible locally as well as on the build: ``` testIOExceptionDuringLogRoll(String).quorum=kraft

[jira] [Updated] (KAFKA-15365) Broker-side replica management changes

2023-12-15 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino updated KAFKA-15365: -- Fix Version/s: 3.7.0 > Broker-side replica management changes >

[jira] [Updated] (KAFKA-15358) QueuedReplicaToDirAssignments metric

2023-12-15 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino updated KAFKA-15358: -- Fix Version/s: 3.7.0 > QueuedReplicaToDirAssignments metric >

[jira] [Updated] (KAFKA-15471) Allow independently stop KRaft controllers or brokers

2023-12-15 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino updated KAFKA-15471: -- Fix Version/s: 3.7.0 > Allow independently stop KRaft controllers or brokers >

[jira] [Resolved] (KAFKA-15471) Allow independently stop KRaft controllers or brokers

2023-12-15 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Dagostino resolved KAFKA-15471. --- Resolution: Fixed > Allow independently stop KRaft controllers or brokers >

[jira] [Created] (KAFKA-16020) Time#waitForFuture should tolerate nanosecond overflow

2023-12-15 Thread David Arthur (Jira)
David Arthur created KAFKA-16020: Summary: Time#waitForFuture should tolerate nanosecond overflow Key: KAFKA-16020 URL: https://issues.apache.org/jira/browse/KAFKA-16020 Project: Kafka Issue

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah commented on code in PR #15007: URL: https://github.com/apache/kafka/pull/15007#discussion_r1428410182 ## metadata/src/main/java/org/apache/kafka/metadata/migration/MigrationManifest.java: ## @@ -41,15 +41,17 @@ public static class Builder { private final Map

[jira] [Commented] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2023-12-15 Thread Sabit (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797281#comment-17797281 ] Sabit commented on KAFKA-12679: --- Would it be possible to add some logging as to which thread owns the lock

[jira] [Updated] (KAFKA-16019) Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback

2023-12-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-16019: --- Description: I was running the PlaintextConsumer to test the async consumer; however, a few tests

[jira] [Updated] (KAFKA-16019) Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback

2023-12-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-16019: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Task) > Some of the tests in

[jira] [Created] (KAFKA-16019) Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback

2023-12-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-16019: -- Summary: Some of the tests in PlaintextConsumer can't seem to deterministically invokes and verify the consumer callback Key: KAFKA-16019 URL:

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
jsancio commented on code in PR #15007: URL: https://github.com/apache/kafka/pull/15007#discussion_r1428364538 ## metadata/src/main/java/org/apache/kafka/metadata/migration/MigrationManifest.java: ## @@ -41,15 +41,17 @@ public static class Builder { private final Map

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
jsancio commented on code in PR #15007: URL: https://github.com/apache/kafka/pull/15007#discussion_r1428347367 ## metadata/src/main/java/org/apache/kafka/metadata/migration/BufferingBatchConsumer.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[PR] KAFKA-16003: Always create the /config/topics ZNode even for topics w… [kafka]

2023-12-15 Thread via GitHub
mimaison opened a new pull request, #15022: URL: https://github.com/apache/kafka/pull/15022 …ithout configs ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify

Re: [PR] KAFKA-15556: Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect [kafka]

2023-12-15 Thread via GitHub
Phuc-Hong-Tran commented on PR #15020: URL: https://github.com/apache/kafka/pull/15020#issuecomment-1858303193 @philipnee, all builds succeed, test failures looks unrelated. PTAL, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Created] (KAFKA-16018) KafkaStreams can go into a zombie state if UncaughtExceptionHandler is specified via the deprecated method

2023-12-15 Thread Tommy Becker (Jira)
Tommy Becker created KAFKA-16018: Summary: KafkaStreams can go into a zombie state if UncaughtExceptionHandler is specified via the deprecated method Key: KAFKA-16018 URL:

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-12-15 Thread via GitHub
jolshan merged PR #14702: URL: https://github.com/apache/kafka/pull/14702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-12-15 Thread via GitHub
jolshan commented on PR #14702: URL: https://github.com/apache/kafka/pull/14702#issuecomment-1858246311 Given that this failed on a storage test (storage tests shouldn't be affected here), and that it was able to build on some of the many other runs I tried, I am going to merge this after

Re: [PR] KAFKA-15764: Missing Tests for Transactions [kafka]

2023-12-15 Thread via GitHub
jolshan commented on PR #14702: URL: https://github.com/apache/kafka/pull/14702#issuecomment-1858240819 I'm convinced this PR is cursed. Still failing builds. :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] MINOR: Minor update patch versions [kafka]

2023-12-15 Thread via GitHub
jolshan commented on PR #14641: URL: https://github.com/apache/kafka/pull/14641#issuecomment-1858231854 Reran and the majority of the 2.6.3 and 2.7.2 tests passed. The ones that didn't had a different issue. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-15 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1428202192 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -892,6 +895,43 @@ public void replay( } } +

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-15 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1428196798 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -299,14 +304,14 @@ class TransactionsTest extends IntegrationTestHarness {

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-15 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1428195220 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -892,6 +895,43 @@ public void replay( } } +

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-15 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1428194671 ## core/src/main/scala/kafka/coordinator/group/CoordinatorLoaderImpl.scala: ## @@ -135,7 +136,22 @@ class CoordinatorLoaderImpl[T](

[jira] [Commented] (KAFKA-16003) The znode /config/topics is not updated during KRaft migration in "dual-write" mode

2023-12-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797249#comment-17797249 ] Mickael Maison commented on KAFKA-16003: [~davidarthur] Yes this is what I had in mind. I should

Re: [PR] KAFKA-14412: Generalise over RocksDB WriteBatch [kafka]

2023-12-15 Thread via GitHub
nicktelford commented on PR #14853: URL: https://github.com/apache/kafka/pull/14853#issuecomment-1858152008 The build still appears to be failing, this time with an error from git about updating the HEAD... It doesnt look like other PRs are affected by this, so I'm guessing the build is

Re: [PR] MINOR: Stop leaking threads in BlockingConnectorTest [kafka]

2023-12-15 Thread via GitHub
C0urante commented on code in PR #12290: URL: https://github.com/apache/kafka/pull/12290#discussion_r1428127814 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/BlockingConnectorTest.java: ## @@ -368,31 +374,54 @@ private static ConfigDef config() {

[jira] [Updated] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Troiano updated KAFKA-16015: --- Description: Using the *kafka-leader-election.sh* I was getting random timeouts like these:

[jira] [Commented] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797219#comment-17797219 ] Sergio Troiano commented on KAFKA-16015: Thanks as usual [~pprovenzano]  :)   I will open then

[jira] [Commented] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797218#comment-17797218 ] Proven Provenzano commented on KAFKA-16015: --- Hi [~sergio_troi...@hotmail.com]  I would

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-16017: -- Description: Streams checkpoints the wrong offset when a task is revived after a

[jira] [Commented] (KAFKA-16003) The znode /config/topics is not updated during KRaft migration in "dual-write" mode

2023-12-15 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797215#comment-17797215 ] David Arthur commented on KAFKA-16003: -- Ah great. Thanks again for the report [~ppatierno].

[jira] [Comment Edited] (KAFKA-16003) The znode /config/topics is not updated during KRaft migration in "dual-write" mode

2023-12-15 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797215#comment-17797215 ] David Arthur edited comment on KAFKA-16003 at 12/15/23 2:54 PM: Ah,

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-16017: -- Description: Streams checkpoints the wrong offset when a task is revived after a

[jira] [Assigned] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reassigned KAFKA-16017: - Assignee: Bruno Cadonna > Checkpointed offset is incorrect when task is revived and

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-16017: -- Description: Streams checkpoints the wrong offset when a task is revived after a

Re: [PR] MINOR: transform new coordinator error before returning to client [kafka]

2023-12-15 Thread via GitHub
dajac merged PR #15001: URL: https://github.com/apache/kafka/pull/15001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: transform new coordinator error before returning to client [kafka]

2023-12-15 Thread via GitHub
dajac commented on PR #15001: URL: https://github.com/apache/kafka/pull/15001#issuecomment-1857981350 Merged to trunk and 3.7. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on PR #14375: URL: https://github.com/apache/kafka/pull/14375#issuecomment-1857973003 @showuon thank you so much for reviewing the PR. I believe I address the comments that I can. Please review again, when you get a chance. Thank you. -- This is an automated

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428034292 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3205,13 +3220,52 @@ class ReplicaManagerTest { threadNamePrefix: Option[String],

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428033100 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3205,13 +3220,52 @@ class ReplicaManagerTest { threadNamePrefix: Option[String],

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428033100 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3205,13 +3220,52 @@ class ReplicaManagerTest { threadNamePrefix: Option[String],

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428032730 ## core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java: ## @@ -229,6 +229,9 @@ private Long buildRemoteLogAuxState(TopicPartition topicPartition,

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428029820 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1042,8 +1053,9 @@ void cleanupExpiredRemoteLogSegments() throws RemoteStorageException,

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-12-15 Thread via GitHub
tinaselenge commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1428029215 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3884,6 +3938,116 @@ class ReplicaManagerTest { } } + @Test + def

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-16017: -- Description: Streams checkpoints the wrong offset when a task is revived after a

[jira] [Created] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-15 Thread Bruno Cadonna (Jira)
Bruno Cadonna created KAFKA-16017: - Summary: Checkpointed offset is incorrect when task is revived and restoring Key: KAFKA-16017 URL: https://issues.apache.org/jira/browse/KAFKA-16017 Project:

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah commented on PR #15007: URL: https://github.com/apache/kafka/pull/15007#issuecomment-1857890630 Thanks for taking a look @jsancio! I'll answer some related questions here. If an error occurs inside the `readAllMetadata` call in KRaftMetadataDriver, the catch block will explicitly

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah commented on code in PR #15007: URL: https://github.com/apache/kafka/pull/15007#discussion_r1427975054 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -645,6 +648,29 @@ public void run() throws Exception { } }

Re: [PR] KAFKA-16007 Merge batch records during ZK migration [kafka]

2023-12-15 Thread via GitHub
mumrah commented on code in PR #15007: URL: https://github.com/apache/kafka/pull/15007#discussion_r1427972960 ## metadata/src/main/java/org/apache/kafka/metadata/migration/BufferingBatchConsumer.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Resolved] (KAFKA-15971) Re-enable consumer integration tests for new consumer

2023-12-15 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15971. - Resolution: Fixed > Re-enable consumer integration tests for new consumer >

Re: [PR] KAFKA-15971: Re-enable consumer integration tests for new consumer [kafka]

2023-12-15 Thread via GitHub
dajac merged PR #14925: URL: https://github.com/apache/kafka/pull/14925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR Minor Cleanup KRaft code [kafka]

2023-12-15 Thread via GitHub
jlprat commented on PR #15002: URL: https://github.com/apache/kafka/pull/15002#issuecomment-1857846340 Failures in CI seem unrelated to the change and they pass locally. Cherry-picked the change to `3.7` as well (cc @stanislavkozlovski). -- This is an automated message from the

Re: [PR] MINOR Minor Cleanup KRaft code [kafka]

2023-12-15 Thread via GitHub
jlprat merged PR #15002: URL: https://github.com/apache/kafka/pull/15002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
kamalcph commented on PR #15005: URL: https://github.com/apache/kafka/pull/15005#issuecomment-1857818458 I'll provide a review today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15373: fix exception thrown in Admin#describeTopics for unknown ID [kafka]

2023-12-15 Thread via GitHub
MikeEdgar commented on PR #14599: URL: https://github.com/apache/kafka/pull/14599#issuecomment-1857793048 @jolshan , I've updated the test and it's passing, but the CI is failing on a seemingly unrelated test. ``` Gradle Test Run :storage:test > Gradle Test Executor 76 >

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15005: URL: https://github.com/apache/kafka/pull/15005#issuecomment-1857734261 @clolov , there are checkstyle error. Please fix them: https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15005/7/pipeline . Thanks. -- This is an

Re: [PR] KAFKA-16013: impelement ExpiresPerSec for remote fetch [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15015: URL: https://github.com/apache/kafka/pull/15015#issuecomment-1857731719 @satishd , addressed your comment in this PR. Thanks. https://github.com/apache/kafka/pull/15014#discussion_r1426937685 -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-16013: Add metric for expiration rate of delayed remote fetch [kafka]

2023-12-15 Thread via GitHub
showuon merged PR #15014: URL: https://github.com/apache/kafka/pull/15014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16013: Add metric for expiration rate of delayed remote fetch [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15014: URL: https://github.com/apache/kafka/pull/15014#issuecomment-1857712774 Failed tests are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Troiano updated KAFKA-16015: --- Description: Using the *kafka-leader-election.sh* I was getting random timeouts like these:

Re: [PR] KAFKA-15945 : Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest [kafka]

2023-12-15 Thread via GitHub
OmniaGM commented on code in PR #14893: URL: https://github.com/apache/kafka/pull/14893#discussion_r1427819819 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationBaseTest.java: ## @@ -920,13 +923,13 @@ String

Re: [PR] KAFKA-16002: Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments [kafka]

2023-12-15 Thread via GitHub
clolov commented on code in PR #15005: URL: https://github.com/apache/kafka/pull/15005#discussion_r1427814012 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1073,13 +1088,29 @@ void cleanupExpiredRemoteLogSegments() throws RemoteStorageException,

[jira] [Created] (KAFKA-16016) Migrate utility scripts to kafka codebase

2023-12-15 Thread Vedarth Sharma (Jira)
Vedarth Sharma created KAFKA-16016: -- Summary: Migrate utility scripts to kafka codebase Key: KAFKA-16016 URL: https://issues.apache.org/jira/browse/KAFKA-16016 Project: Kafka Issue Type:

Re: [PR] KAFKA-16014: add RemoteLogSizeComputationTime metric [kafka]

2023-12-15 Thread via GitHub
clolov commented on code in PR #15021: URL: https://github.com/apache/kafka/pull/15021#discussion_r1427799678 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -408,6 +409,17 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

[jira] [Updated] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Troiano updated KAFKA-16015: --- Affects Version/s: 3.5.1 > kafka-leader-election timeout values always overwritten by

[jira] [Updated] (KAFKA-16015) kafka-leader-election timeout values always overwritten by default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Troiano updated KAFKA-16015: --- Summary: kafka-leader-election timeout values always overwritten by default values (was:

[jira] [Updated] (KAFKA-16015) kafka-leader-election timeout values always overwritten b default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Troiano updated KAFKA-16015: --- Description: Using the *kafka-leader-election.sh* I was getting random timeouts like these:

[jira] [Commented] (KAFKA-16015) kafka-leader-election timeout values always overwritten b default values

2023-12-15 Thread Sergio Troiano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17797113#comment-17797113 ] Sergio Troiano commented on KAFKA-16015: hi [~pprovenzano] , I will open a PR in the 3.6 branch

[jira] [Created] (KAFKA-16015) kafka-leader-election timeout values always overwritten b default values

2023-12-15 Thread Sergio Troiano (Jira)
Sergio Troiano created KAFKA-16015: -- Summary: kafka-leader-election timeout values always overwritten b default values Key: KAFKA-16015 URL: https://issues.apache.org/jira/browse/KAFKA-16015

Re: [PR] KAFKA-16014: add RemoteLogSizeComputationTime metric [kafka]

2023-12-15 Thread via GitHub
showuon commented on PR #15021: URL: https://github.com/apache/kafka/pull/15021#issuecomment-1857570946 @clolov @satishd , call for review. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] KAFKA-16014: add RemoteLogSizeComputationTime metric [kafka]

2023-12-15 Thread via GitHub
showuon opened a new pull request, #15021: URL: https://github.com/apache/kafka/pull/15021 Implementing `RemoteLogSizeComputationTime` metric in KIP-963. Waiting for https://github.com/apache/kafka/pull/15005 merged to add this metric into `removeRemoteTopicPartitionMetrics` method.

Re: [PR] KAFKA-15971: Re-enable consumer integration tests for new consumer [kafka]

2023-12-15 Thread via GitHub
dajac commented on PR #14925: URL: https://github.com/apache/kafka/pull/14925#issuecomment-1857510091 The last few builds failed due to infrastructure issues but we got a "good" one here: https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14925/10/tests.

Re: [PR] KAFKA-15870: Move new group coordinator metrics from Yammer to Metrics [kafka]

2023-12-15 Thread via GitHub
dajac commented on PR #14848: URL: https://github.com/apache/kafka/pull/14848#issuecomment-1857508292 Triggered a new build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-15 Thread via GitHub
dajac commented on PR #14640: URL: https://github.com/apache/kafka/pull/14640#issuecomment-1857500665 The failed tests are not related. Merged it to trunk and 3.7. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

  1   2   >