[jira] [Commented] (KAFKA-15793) Flaky test ZkMigrationIntegrationTest.testMigrateTopicDeletions
[
https://issues.apache.org/jira/browse/KAFKA-15793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835652#comment-17835652
]
Igor Soarez commented on KAFKA-15793:
-
This has come up again:
{code:java}
[2024-04-09T21:06:17.307Z] Gradle Test Run :core:test > Gradle Test Executor 97
> DeleteTopicsRequestTest >
testTopicDeletionClusterHasOfflinePartitions(String) >
"testTopicDeletionClusterHasOfflinePartitions(String).quorum=zk" STARTED
[2024-04-09T21:06:17.307Z]
kafka.zk.ZkMigrationIntegrationTest.testMigrateTopicDeletions(ClusterInstance)[7]
failed, log available in
/home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-15522/core/build/reports/testOutput/kafka.zk.ZkMigrationIntegrationTest.testMigrateTopicDeletions(ClusterInstance)[7].test.stdout
[2024-04-09T21:06:17.307Z]
[2024-04-09T21:06:17.307Z] Gradle Test Run :core:test > Gradle Test Executor 96
> ZkMigrationIntegrationTest > testMigrateTopicDeletions(ClusterInstance) >
testMigrateTopicDeletions [7] Type=ZK, MetadataVersion=3.7-IV4,
Security=PLAINTEXT FAILED
[2024-04-09T21:06:17.307Z]
org.apache.kafka.server.fault.FaultHandlerException: nonFatalFaultHandler:
Unhandled error in MetadataChangeEvent: Check op on KRaft Migration ZNode
failed. Expected zkVersion = 5. This indicates that another KRaft controller is
making writes to ZooKeeper.
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.KafkaZkClient.handleUnwrappedMigrationResult$1(KafkaZkClient.scala:2001)
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.KafkaZkClient.unwrapMigrationResponse$1(KafkaZkClient.scala:2027)
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.KafkaZkClient.$anonfun$retryMigrationRequestsUntilConnected$2(KafkaZkClient.scala:2052)
[2024-04-09T21:06:17.307Z] at
app//scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
[2024-04-09T21:06:17.307Z] at
app//scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
[2024-04-09T21:06:17.307Z] at
app//scala.collection.mutable.ArrayBuffer.map(ArrayBuffer.scala:43)
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.KafkaZkClient.retryMigrationRequestsUntilConnected(KafkaZkClient.scala:2052)
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.migration.ZkTopicMigrationClient.$anonfun$updateTopicPartitions$1(ZkTopicMigrationClient.scala:265)
[2024-04-09T21:06:17.307Z] at
app//kafka.zk.migration.ZkTopicMigrationClient.updateTopicPartitions(ZkTopicMigrationClient.scala:255)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationZkWriter.lambda$handleTopicsDelta$20(KRaftMigrationZkWriter.java:334)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationDriver.applyMigrationOperation(KRaftMigrationDriver.java:248)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationDriver.access$300(KRaftMigrationDriver.java:62)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationDriver$MetadataChangeEvent.lambda$run$1(KRaftMigrationDriver.java:532)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationDriver.lambda$countingOperationConsumer$6(KRaftMigrationDriver.java:845)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationZkWriter.lambda$handleTopicsDelta$21(KRaftMigrationZkWriter.java:331)
[2024-04-09T21:06:17.307Z] at
java.base@17.0.7/java.util.HashMap.forEach(HashMap.java:1421)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationZkWriter.handleTopicsDelta(KRaftMigrationZkWriter.java:297)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationZkWriter.handleDelta(KRaftMigrationZkWriter.java:112)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.metadata.migration.KRaftMigrationDriver$MetadataChangeEvent.run(KRaftMigrationDriver.java:531)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:128)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:211)
[2024-04-09T21:06:17.307Z] at
app//org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:182)
[2024-04-09T21:06:17.307Z] at
java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
[2024-04-09T21:06:17.307Z]
[2024-04-09T21:06:17.307Z] Caused by:
[2024-04-09T21:06:17.307Z] java.lang.RuntimeException: Check op on
KRaft Migration ZNode failed. Expected zkVersion = 5. This indicates that
another KRaft controller is making writes to ZooKeeper.
[2024-04-09T21:06:17.307Z] at
kafka.zk.KafkaZkClient.handleUnwrappedMigrationResult$1(KafkaZkClient.scala:2001)
[jira] [Created] (KAFKA-16504) Flaky test org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations
Igor Soarez created KAFKA-16504:
---
Summary: Flaky test
org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations
Key: KAFKA-16504
URL: https://issues.apache.org/jira/browse/KAFKA-16504
Project: Kafka
Issue Type: Test
Components: controller
Reporter: Igor Soarez
{code:java}
[2024-04-09T20:26:55.840Z] Gradle Test Run :metadata:test > Gradle Test
Executor 54 > QuorumControllerTest > testConfigurationOperations() STARTED
[2024-04-09T20:26:55.840Z]
org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations()
failed, log available in
/home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-15522/metadata/build/reports/testOutput/org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations().test.stdout
[2024-04-09T20:26:55.840Z]
[2024-04-09T20:26:55.840Z] Gradle Test Run :metadata:test > Gradle Test
Executor 54 > QuorumControllerTest > testConfigurationOperations() FAILED
[2024-04-09T20:26:55.840Z] java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.NotControllerException: No controller appears to
be active.
[2024-04-09T20:26:55.840Z] at
java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
[2024-04-09T20:26:55.840Z] at
java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
[2024-04-09T20:26:55.840Z] at
org.apache.kafka.controller.QuorumControllerTest.testConfigurationOperations(QuorumControllerTest.java:202)
[2024-04-09T20:26:55.840Z]
[2024-04-09T20:26:55.840Z] Caused by:
[2024-04-09T20:26:55.840Z]
org.apache.kafka.common.errors.NotControllerException: No controller appears to
be active. {code}
[https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15522/5/tests/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] KAFKA-15951: MissingSourceTopicException should include topic names [kafka]
chickenchickenlove commented on PR #15573: URL: https://github.com/apache/kafka/pull/15573#issuecomment-2046916043 Gently ping, @mjsax . Would you please take a look when you have some free time? ♂️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15853: Move consumer group and group coordinator configs out of core [kafka]
OmniaGM commented on code in PR #15684:
URL: https://github.com/apache/kafka/pull/15684#discussion_r1559064580
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetConfig.java:
##
@@ -20,16 +20,51 @@
import org.apache.kafka.common.record.CompressionType;
public class OffsetConfig {
Review Comment:
I pushed the changes let me know if it makes sense or if you have other
suggestions/feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-15853: Move consumer group and group coordinator configs out of core [kafka]
OmniaGM commented on code in PR #15684:
URL: https://github.com/apache/kafka/pull/15684#discussion_r1559054203
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetConfig.java:
##
@@ -20,16 +20,51 @@
import org.apache.kafka.common.record.CompressionType;
public class OffsetConfig {
Review Comment:
I'll move all props, defaults and docs into `GroupCoordinatorConfig` and
move the constructor for `OffsetConfig` as static class within
`GroupCoordinatorConfig` as `GroupMetadataManager` which mainly cares about
`OffsetConfig` doesn't need the rest of the configs in `GroupCoordinatorConfig`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-15853: Move consumer group and group coordinator configs out of core [kafka]
OmniaGM commented on code in PR #15684:
URL: https://github.com/apache/kafka/pull/15684#discussion_r1559022658
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetConfig.java:
##
@@ -20,16 +20,51 @@
import org.apache.kafka.common.record.CompressionType;
public class OffsetConfig {
Review Comment:
Good call, this actually will make it simplier. Will update the PR to
reflict this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] KAFKA-15853: Move consumer group and group coordinator configs out of core [kafka]
dajac commented on code in PR #15684:
URL: https://github.com/apache/kafka/pull/15684#discussion_r1558996031
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetConfig.java:
##
@@ -20,16 +20,51 @@
import org.apache.kafka.common.record.CompressionType;
public class OffsetConfig {
Review Comment:
@OmniaGM Thanks for working on this. I wonder whether we could merge
`OffsetConfig` into `GroupCoordinatorConfig` in order to have one config for
the entire package. It would bring us closer to the goal described in
https://issues.apache.org/jira/browse/KAFKA-15089. Would it be possible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] MINOR: Add type check to classic group timeout operations [kafka]
dajac merged PR #15587: URL: https://github.com/apache/kafka/pull/15587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Add type check to classic group timeout operations [kafka]
dajac commented on code in PR #15587:
URL: https://github.com/apache/kafka/pull/15587#discussion_r1558986456
##
group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java:
##
@@ -2415,6 +2415,22 @@ private CoordinatorResult
classicGroupJoinExistingMember(
return EMPTY_RESULT;
}
+/**
+ * An overload of {@link
GroupMetadataManager#completeClassicGroupJoin(ClassicGroup)} used as
+ * timeout operation. It additionally looks up the group by the id and
checks the group type.
+ * completeClassicGroupJoin will only be called if the group is CLASSIC.
+ */
+private CoordinatorResult completeClassicGroupJoin(String
groupId) {
+ClassicGroup group;
+try {
+group = getOrMaybeCreateClassicGroup(groupId, false);
+} catch (UnknownMemberIdException | GroupIdNotFoundException
exception) {
Review Comment:
Filed https://issues.apache.org/jira/browse/KAFKA-16503.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (KAFKA-16503) getOrMaybeCreateClassicGroup should not thrown GroupIdNotFoundException
David Jacot created KAFKA-16503: --- Summary: getOrMaybeCreateClassicGroup should not thrown GroupIdNotFoundException Key: KAFKA-16503 URL: https://issues.apache.org/jira/browse/KAFKA-16503 Project: Kafka Issue Type: Sub-task Reporter: David Jacot It looks like `getOrMaybeCreateClassicGroup` method throws an `GroupIdNotFoundException` error when the group exists but with the wrong type. As `getOrMaybeCreateClassicGroup` is mainly used on the join-group/sync-group APIs, this seems incorrect. We need to double check and fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] KAFKA:-16399: Add JBOD support in tiered storage [kafka]
showuon opened a new pull request, #15690: URL: https://github.com/apache/kafka/pull/15690 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.* ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] MINOR: Remove dead code of metric forward-rate [kafka]
cadonna merged PR #15686: URL: https://github.com/apache/kafka/pull/15686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-16294: Add group protocol migration enabling config [kafka]
dajac commented on PR #15411: URL: https://github.com/apache/kafka/pull/15411#issuecomment-2046712467 Re-triggered a build as last one had errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-16004: auto-commit inflight improved logs, docs and tests [kafka]
cadonna merged PR #15669: URL: https://github.com/apache/kafka/pull/15669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (KAFKA-15538) Client support for java regex based subscription
[ https://issues.apache.org/jira/browse/KAFKA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reopened KAFKA-15538: --- > Client support for java regex based subscription > > > Key: KAFKA-15538 > URL: https://issues.apache.org/jira/browse/KAFKA-15538 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Lianet Magrans >Assignee: Phuc Hong Tran >Priority: Blocker > Labels: kip-848-client-support, newbie, regex > Fix For: 3.8.0 > > > When using subscribe with a java regex (Pattern), we need to resolve it on > the client side to send the broker a list of topic names to subscribe to. > Context: > The new consumer group protocol uses [Google > RE2/J|https://github.com/google/re2j] for regular expressions and introduces > new methods in the consumer API to subscribe using a `SubscribePattern`. The > subscribe using a java `Pattern` will be still supported for a while but > eventually removed. > * When the subscribe with SubscriptionPattern is used, the client should > just send the regex to the broker and it will be resolved on the server side. > * In the case of the subscribe with Pattern, the regex should be resolved on > the client side. > As part of this task, we should re-enable all integration tests defined in > the PlainTextAsyncConsumer that relate to subscription with pattern and that > are currently disabled for the new consumer + new protocol -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] KAFKA-15538: Client support for java regex based subscription [kafka]
cadonna merged PR #15585: URL: https://github.com/apache/kafka/pull/15585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15588 ConfigCommandIntegrationTest rewritten in java [kafka]
nizhikov commented on PR #15645: URL: https://github.com/apache/kafka/pull/15645#issuecomment-2046644817 > I prefer to don't touch it. This class is all about zk and we can remove it once we remove zk support. So we want to rewrite in java KRaft tests only. Is it correct? I thought we want to move all code related to the `ConfigCommand` to java and remove it when ZK support will be dropped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
