[jira] [Resolved] (KAFKA-16759) Invalid client telemetry transition on consumer close
[
https://issues.apache.org/jira/browse/KAFKA-16759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Schofield resolved KAFKA-16759.
--
Resolution: Fixed
> Invalid client telemetry transition on consumer close
> -
>
> Key: KAFKA-16759
> URL: https://issues.apache.org/jira/browse/KAFKA-16759
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 3.7.0
>Reporter: Andrew Schofield
>Assignee: Andrew Schofield
>Priority: Minor
> Fix For: 3.8.0
>
>
> Using the console consumer with client telemetry enabled, I hit an invalid
> state transition when closing the consumer with CTRL-C. The consumer sends a
> final "terminating" telemetry push which puts the client telemetry reporter
> into TERMINATING_PUSH_IN_PROGRESS state. When it receives a response in this
> state, it attempts an invalid state transition.
>
> {noformat}
> [2024-05-13 19:19:35,804] WARN Error updating client telemetry state,
> disabled telemetry
> (org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter)
> java.lang.IllegalStateException: Invalid telemetry state transition from
> TERMINATING_PUSH_IN_PROGRESS to PUSH_NEEDED; the valid telemetry state
> transitions from TERMINATING_PUSH_IN_PROGRESS are: TERMINATED
> at
> org.apache.kafka.common.telemetry.ClientTelemetryState.validateTransition(ClientTelemetryState.java:163)
> at
> org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter$DefaultClientTelemetrySender.maybeSetState(ClientTelemetryReporter.java:827)
> at
> org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter$DefaultClientTelemetrySender.handleResponse(ClientTelemetryReporter.java:520)
> at
> org.apache.kafka.clients.NetworkClient$TelemetrySender.handleResponse(NetworkClient.java:1321)
> at
> org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:948)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:594)
> at
> org.apache.kafka.clients.consumer.internals.NetworkClientDelegate.poll(NetworkClientDelegate.java:130)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.sendUnsentRequests(ConsumerNetworkThread.java:262)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.cleanup(ConsumerNetworkThread.java:275)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.run(ConsumerNetworkThread.java:95)
> [2024-05-13 19:19:35,805] WARN Unable to transition state after successful
> push telemetry from state TERMINATING_PUSH_IN_PROGRESS
> (org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16759) Invalid client telemetry transition on consumer close
Andrew Schofield created KAFKA-16759:
Summary: Invalid client telemetry transition on consumer close
Key: KAFKA-16759
URL: https://issues.apache.org/jira/browse/KAFKA-16759
Project: Kafka
Issue Type: Bug
Affects Versions: 3.7.0
Reporter: Andrew Schofield
Assignee: Andrew Schofield
Fix For: 3.8.0
Using the console consumer with client telemetry enabled, I hit an invalid
state transition when closing the consumer with CTRL-C. The consumer sends a
final "terminating" telemetry push which puts the client telemetry reporter
into TERMINATING_PUSH_IN_PROGRESS state. When it receives a response in this
state, it attempts an invalid state transition.
{noformat}
[2024-05-13 19:19:35,804] WARN Error updating client telemetry state, disabled
telemetry (org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter)
java.lang.IllegalStateException: Invalid telemetry state transition from
TERMINATING_PUSH_IN_PROGRESS to PUSH_NEEDED; the valid telemetry state
transitions from TERMINATING_PUSH_IN_PROGRESS are: TERMINATED
at
org.apache.kafka.common.telemetry.ClientTelemetryState.validateTransition(ClientTelemetryState.java:163)
at
org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter$DefaultClientTelemetrySender.maybeSetState(ClientTelemetryReporter.java:827)
at
org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter$DefaultClientTelemetrySender.handleResponse(ClientTelemetryReporter.java:520)
at
org.apache.kafka.clients.NetworkClient$TelemetrySender.handleResponse(NetworkClient.java:1321)
at
org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:948)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:594)
at
org.apache.kafka.clients.consumer.internals.NetworkClientDelegate.poll(NetworkClientDelegate.java:130)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.sendUnsentRequests(ConsumerNetworkThread.java:262)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.cleanup(ConsumerNetworkThread.java:275)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkThread.run(ConsumerNetworkThread.java:95)
[2024-05-13 19:19:35,805] WARN Unable to transition state after successful push
telemetry from state TERMINATING_PUSH_IN_PROGRESS
(org.apache.kafka.common.telemetry.internals.ClientTelemetryReporter){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-16734) Add support for formatting records written to share-group state topic
Andrew Schofield created KAFKA-16734: Summary: Add support for formatting records written to share-group state topic Key: KAFKA-16734 URL: https://issues.apache.org/jira/browse/KAFKA-16734 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16730) Initial code for share-group consumer
Andrew Schofield created KAFKA-16730: Summary: Initial code for share-group consumer Key: KAFKA-16730 URL: https://issues.apache.org/jira/browse/KAFKA-16730 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16733) Add support for formatting new records written to offsets topic
Andrew Schofield created KAFKA-16733: Summary: Add support for formatting new records written to offsets topic Key: KAFKA-16733 URL: https://issues.apache.org/jira/browse/KAFKA-16733 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16732) Support for share-coordinator-metrics in the broker
Andrew Schofield created KAFKA-16732: Summary: Support for share-coordinator-metrics in the broker Key: KAFKA-16732 URL: https://issues.apache.org/jira/browse/KAFKA-16732 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16731) Support for share-group-metrics in the broker
Andrew Schofield created KAFKA-16731: Summary: Support for share-group-metrics in the broker Key: KAFKA-16731 URL: https://issues.apache.org/jira/browse/KAFKA-16731 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16728) Add dynamic group configuration for heartbeat interval and session timeout
Andrew Schofield created KAFKA-16728: Summary: Add dynamic group configuration for heartbeat interval and session timeout Key: KAFKA-16728 URL: https://issues.apache.org/jira/browse/KAFKA-16728 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16729) Support for read-committed isolation level
Andrew Schofield created KAFKA-16729: Summary: Support for read-committed isolation level Key: KAFKA-16729 URL: https://issues.apache.org/jira/browse/KAFKA-16729 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16726) Add dynamic group configuration for offset reset
Andrew Schofield created KAFKA-16726: Summary: Add dynamic group configuration for offset reset Key: KAFKA-16726 URL: https://issues.apache.org/jira/browse/KAFKA-16726 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16725) Add broker configurations
Andrew Schofield created KAFKA-16725: Summary: Add broker configurations Key: KAFKA-16725 URL: https://issues.apache.org/jira/browse/KAFKA-16725 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16727) Add dynamic group configuration for record lock duration
Andrew Schofield created KAFKA-16727: Summary: Add dynamic group configuration for record lock duration Key: KAFKA-16727 URL: https://issues.apache.org/jira/browse/KAFKA-16727 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16724) Add new options for kafka-producer-perf-test.sh
Andrew Schofield created KAFKA-16724: Summary: Add new options for kafka-producer-perf-test.sh Key: KAFKA-16724 URL: https://issues.apache.org/jira/browse/KAFKA-16724 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16722) Add ConsumerGroupPartitionAssignor and ShareGroupPartitionAssignor
Andrew Schofield created KAFKA-16722: Summary: Add ConsumerGroupPartitionAssignor and ShareGroupPartitionAssignor Key: KAFKA-16722 URL: https://issues.apache.org/jira/browse/KAFKA-16722 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16723) Add kafka-console-share-consumer.sh
Andrew Schofield created KAFKA-16723: Summary: Add kafka-console-share-consumer.sh Key: KAFKA-16723 URL: https://issues.apache.org/jira/browse/KAFKA-16723 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16721) Add exceptions for the new error codes
Andrew Schofield created KAFKA-16721: Summary: Add exceptions for the new error codes Key: KAFKA-16721 URL: https://issues.apache.org/jira/browse/KAFKA-16721 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16718) Add AdminClient.deleteShareGroupOffsets
Andrew Schofield created KAFKA-16718: Summary: Add AdminClient.deleteShareGroupOffsets Key: KAFKA-16718 URL: https://issues.apache.org/jira/browse/KAFKA-16718 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16719) Add AdminClient.deleteShareGroups
Andrew Schofield created KAFKA-16719: Summary: Add AdminClient.deleteShareGroups Key: KAFKA-16719 URL: https://issues.apache.org/jira/browse/KAFKA-16719 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16717) Add AdminClient.alterShareGroupOffsets
Andrew Schofield created KAFKA-16717: Summary: Add AdminClient.alterShareGroupOffsets Key: KAFKA-16717 URL: https://issues.apache.org/jira/browse/KAFKA-16717 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16720) Add AdminClient.listShareGroupOffsets
Andrew Schofield created KAFKA-16720: Summary: Add AdminClient.listShareGroupOffsets Key: KAFKA-16720 URL: https://issues.apache.org/jira/browse/KAFKA-16720 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16716) Add AdminClient.describeShareGroups and AdminClient.listShareGroups
Andrew Schofield created KAFKA-16716: Summary: Add AdminClient.describeShareGroups and AdminClient.listShareGroups Key: KAFKA-16716 URL: https://issues.apache.org/jira/browse/KAFKA-16716 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16715) Create KafkaShareConsumer interface
Andrew Schofield created KAFKA-16715: Summary: Create KafkaShareConsumer interface Key: KAFKA-16715 URL: https://issues.apache.org/jira/browse/KAFKA-16715 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16714) kafka-share-groups.sh supporting list and describe
Andrew Schofield created KAFKA-16714: Summary: kafka-share-groups.sh supporting list and describe Key: KAFKA-16714 URL: https://issues.apache.org/jira/browse/KAFKA-16714 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16713) Add new RPC definitions
Andrew Schofield created KAFKA-16713: Summary: Add new RPC definitions Key: KAFKA-16713 URL: https://issues.apache.org/jira/browse/KAFKA-16713 Project: Kafka Issue Type: Sub-task Reporter: Andrew Schofield Assignee: Andrew Schofield -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-16676) Security docs missing RPCs from KIP-714 and KIP-1000
[ https://issues.apache.org/jira/browse/KAFKA-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-16676. -- Resolution: Fixed > Security docs missing RPCs from KIP-714 and KIP-1000 > > > Key: KAFKA-16676 > URL: https://issues.apache.org/jira/browse/KAFKA-16676 > Project: Kafka > Issue Type: Improvement > Components: docs >Affects Versions: 3.7.0, 3.8.0 >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Minor > Fix For: 3.8.0 > > > KIPs 714 and 1000 introduced 3 new RPCs to do with client metrics. None of > them was added to the list of RPCs in the security documentation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16676) Security docs missing RPCs from KIP-714 and KIP-1000
Andrew Schofield created KAFKA-16676: Summary: Security docs missing RPCs from KIP-714 and KIP-1000 Key: KAFKA-16676 URL: https://issues.apache.org/jira/browse/KAFKA-16676 Project: Kafka Issue Type: Improvement Components: docs Affects Versions: 3.7.0, 3.8.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.8.0 KIPs 714 and 1000 introduced 3 new RPCs to do with client metrics. None of them was added to the list of RPCs in the security documentation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16608) AsyncKafkaConsumer doesn't honour interrupted thread status on KafkaConsumer.poll(Duration)
Andrew Schofield created KAFKA-16608: Summary: AsyncKafkaConsumer doesn't honour interrupted thread status on KafkaConsumer.poll(Duration) Key: KAFKA-16608 URL: https://issues.apache.org/jira/browse/KAFKA-16608 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 3.8.0 Reporter: Andrew Schofield The behaviour for KafkaConsumer.poll(Duration) when the calling thread is in interrupted state is to throw InterruptException. The AsyncKafkaConsumer doesn't do this. It only throws that exception if the interruption occurs while it is waiting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16261) MembershipManagerImpl.updateSubscription fails if already empty subscription
Andrew Schofield created KAFKA-16261: Summary: MembershipManagerImpl.updateSubscription fails if already empty subscription Key: KAFKA-16261 URL: https://issues.apache.org/jira/browse/KAFKA-16261 Project: Kafka Issue Type: Bug Components: clients Reporter: Andrew Schofield Assignee: Lianet Magrans The internal SubscriptionState object keeps track of whether the assignment is user-assigned, or auto-assigned. If there are no assigned partitions, the assignment resets to NONE. If you call SubscriptionState.assignFromSubscribed in this state it fails. The easiest thing is perhaps to check SubscriptionState.hasAutoAssignedPartitions() to make sure that assignFromSubscribed is going to be permitted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16243) Idle kafka-console-consumer with new consumer group protocol preemptively leaves group
Andrew Schofield created KAFKA-16243: Summary: Idle kafka-console-consumer with new consumer group protocol preemptively leaves group Key: KAFKA-16243 URL: https://issues.apache.org/jira/browse/KAFKA-16243 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Using the new consumer group protocol with kafka-console-consumer.sh, I find that if I leave the consumer with no records to process for 5 minutes (max.poll.interval.ms = 30ms), the tool logs the following warning message and leaves the group. "consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records." With the older consumer, this did not occur. The reason is that the consumer keeps a poll timer which is used to ensure liveness of the application thread. The poll timer automatically updates while the `Consumer.poll(Duration)` method is blocked, while the newer consumer only updates the poll timer when a new call to `Consumer.poll(Duration)` is issued. This means that the kafka-console-consumer.sh tools, which uses a very long timeout by default, works differently with the new consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-8606) Provide a method to fetch committed offsets for a collection of TopicPartition
[ https://issues.apache.org/jira/browse/KAFKA-8606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-8606. - Fix Version/s: 2.4.0 Resolution: Fixed Method added in AK 2.4. > Provide a method to fetch committed offsets for a collection of TopicPartition > -- > > Key: KAFKA-8606 > URL: https://issues.apache.org/jira/browse/KAFKA-8606 > Project: Kafka > Issue Type: Improvement > Components: clients, consumer >Affects Versions: 2.3.0, 2.2.1 >Reporter: ov7a >Priority: Major > Fix For: 2.4.0 > > > Currently KafkaConsumer has methods for fetching begging offsets, end offsets > and offsets for times, all of them accepting a collection of TopicPartition. > There is a method to fetch committed offset for single TopicPartition, but > there is no public API to fetch commited offsets for a collection of > TopicPartition. So, If one wants to fetch all committed offsets for topic, a > request per partition is created. > Note that ConsumerCoordinator.fetchCommittedOffsets which called internally > in "committed" method does accept a collection of TopicPartition. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16184) Flaky test - testAlterReplicaLogDirs(String).quorum=kraft – kafka.api.PlaintextAdminIntegrationTest
Andrew Schofield created KAFKA-16184: Summary: Flaky test - testAlterReplicaLogDirs(String).quorum=kraft – kafka.api.PlaintextAdminIntegrationTest Key: KAFKA-16184 URL: https://issues.apache.org/jira/browse/KAFKA-16184 Project: Kafka Issue Type: Bug Reporter: Andrew Schofield [https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15214/1/tests/] h4. Error org.opentest4j.AssertionFailedError: timed out waiting for replica movement h4. Stacktrace org.opentest4j.AssertionFailedError: timed out waiting for replica movement at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at org.junit.jupiter.api.Assertions.fail(Assertions.java:134) at kafka.api.PlaintextAdminIntegrationTest.$anonfun$testAlterReplicaLogDirs$7(PlaintextAdminIntegrationTest.scala:317) at kafka.api.PlaintextAdminIntegrationTest.$anonfun$testAlterReplicaLogDirs$7$adapted(PlaintextAdminIntegrationTest.scala:316) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at kafka.api.PlaintextAdminIntegrationTest.testAlterReplicaLogDirs(PlaintextAdminIntegrationTest.scala:316) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at
[jira] [Created] (KAFKA-16183) Flaky test - testMetricsDuringTopicCreateDelete(String).quorum=zk – kafka.integration.MetricsDuringTopicCreationDeletionTest
Andrew Schofield created KAFKA-16183: Summary: Flaky test - testMetricsDuringTopicCreateDelete(String).quorum=zk – kafka.integration.MetricsDuringTopicCreationDeletionTest Key: KAFKA-16183 URL: https://issues.apache.org/jira/browse/KAFKA-16183 Project: Kafka Issue Type: Bug Reporter: Andrew Schofield [https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15214/1/pipeline] h4. Error java.lang.AssertionError: assertion failed: Expect UnderReplicatedPartitionCount to be 0, but got: 1 h4. Stacktrace java.lang.AssertionError: assertion failed: Expect UnderReplicatedPartitionCount to be 0, but got: 1 at scala.Predef$.assert(Predef.scala:279) at kafka.integration.MetricsDuringTopicCreationDeletionTest.testMetricsDuringTopicCreateDelete(MetricsDuringTopicCreationDeletionTest.scala:123) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:226) at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:204) at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:142) at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:110) at
[jira] [Created] (KAFKA-16182) Flaky test - testClientInstanceId() - org.apache.kafka.clients.admin.KafkaAdminClientTest
Andrew Schofield created KAFKA-16182: Summary: Flaky test - testClientInstanceId() - org.apache.kafka.clients.admin.KafkaAdminClientTest Key: KAFKA-16182 URL: https://issues.apache.org/jira/browse/KAFKA-16182 Project: Kafka Issue Type: Bug Components: admin Affects Versions: 3.7.0 Reporter: Andrew Schofield h3. Error org.apache.kafka.common.KafkaException: Error occurred while fetching client instance id Stacktrace org.apache.kafka.common.KafkaException: Error occurred while fetching client instance id at app//org.apache.kafka.clients.admin.KafkaAdminClient.clientInstanceId(KafkaAdminClient.java:4477) at app//org.apache.kafka.clients.admin.KafkaAdminClientTest.testClientInstanceId(KafkaAdminClientTest.java:7082) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45) at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) at java.base@17.0.7/java.util.ArrayList.forEach(ArrayList.java:1511) at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at
[jira] [Created] (KAFKA-16137) ListClientMetricsResourcesResponse definition is missing field descriptions
Andrew Schofield created KAFKA-16137: Summary: ListClientMetricsResourcesResponse definition is missing field descriptions Key: KAFKA-16137 URL: https://issues.apache.org/jira/browse/KAFKA-16137 Project: Kafka Issue Type: Task Components: admin Affects Versions: 3.7.0 Reporter: Andrew Schofield This is purely improving the readability of the Kafka protocol documentation by adding missing description information for the fields of the `ListClientMetricsResources` response. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16104) Enable additional PlaintextConsumerTest tests for new consumer
Andrew Schofield created KAFKA-16104: Summary: Enable additional PlaintextConsumerTest tests for new consumer Key: KAFKA-16104 URL: https://issues.apache.org/jira/browse/KAFKA-16104 Project: Kafka Issue Type: Improvement Components: clients Reporter: Andrew Schofield It should be possible to enable: * testAutoCommitOnClose * testAutoCommitOnCloseAfterWakeup * testExpandingTopicSubscriptions * testShrinkingTopicSubscriptions * testMultiConsumerSessionTimeoutOnClose (KAFKA-16011 has been fixed) * testAutoCommitOnRebalance * testPerPartitionLeadMetricsCleanUpWithSubscribe * testPerPartitionLagMetricsCleanUpWithSubscribe * testStaticConsumerDetectsNewPartitionCreatedAfterRestart -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16100) Consistent handling of timeouts and responses for new consumer ApplicationEvents
Andrew Schofield created KAFKA-16100: Summary: Consistent handling of timeouts and responses for new consumer ApplicationEvents Key: KAFKA-16100 URL: https://issues.apache.org/jira/browse/KAFKA-16100 Project: Kafka Issue Type: Sub-task Components: clients Reporter: Andrew Schofield The handling of timeouts and responses for the various kinds of ApplicationEvents in the new consumer is not consistent. A small amount of refactoring would make the code more maintainable and give consistent behaviour for the different requests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16099) Handle timeouts for AsyncKafkaConsumer.commitSync
Andrew Schofield created KAFKA-16099: Summary: Handle timeouts for AsyncKafkaConsumer.commitSync Key: KAFKA-16099 URL: https://issues.apache.org/jira/browse/KAFKA-16099 Project: Kafka Issue Type: Sub-task Components: clients Reporter: Andrew Schofield The handling of synchronous offset commits in the background thread does not observe the caller's timeout. In the situation that a commit request needs to be retried, the retries should not extend beyond the caller's timeout. The CommitApplicationEvent should contain the timeout and not continue beyond that time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-16092) Queues for Kafka
Andrew Schofield created KAFKA-16092: Summary: Queues for Kafka Key: KAFKA-16092 URL: https://issues.apache.org/jira/browse/KAFKA-16092 Project: Kafka Issue Type: Improvement Reporter: Andrew Schofield Assignee: Andrew Schofield This Jira tracks the development of KIP-932: https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15775) Implement listTopics() and partitionFor() for the AsyncKafkaConsumer
[
https://issues.apache.org/jira/browse/KAFKA-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Schofield resolved KAFKA-15775.
--
Resolution: Fixed
> Implement listTopics() and partitionFor() for the AsyncKafkaConsumer
>
>
> Key: KAFKA-15775
> URL: https://issues.apache.org/jira/browse/KAFKA-15775
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
>Reporter: Philip Nee
>Assignee: Andrew Schofield
>Priority: Major
> Labels: consumer-threading-refactor, kip-848,
> kip-848-client-support, kip-848-e2e, kip-848-preview
>
> {code:java}
> @Override
> public List partitionsFor(String topic) {
> return partitionsFor(topic, Duration.ofMillis(defaultApiTimeoutMs));
> }
> @Override
> public List partitionsFor(String topic, Duration timeout) {
> throw new KafkaException("method not implemented");
> }
> @Override
> public Map> listTopics() {
> return listTopics(Duration.ofMillis(defaultApiTimeoutMs));
> }
> @Override
> public Map> listTopics(Duration timeout) {
> throw new KafkaException("method not implemented");
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (KAFKA-15986) New consumer group protocol integration test failures
[ https://issues.apache.org/jira/browse/KAFKA-15986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15986. -- Resolution: Fixed > New consumer group protocol integration test failures > - > > Key: KAFKA-15986 > URL: https://issues.apache.org/jira/browse/KAFKA-15986 > Project: Kafka > Issue Type: Bug > Components: clients >Affects Versions: 3.7.0 >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Major > Labels: CTR > Fix For: 3.7.0 > > > A recent change in `AsyncKafkaConsumer.updateFetchPositions` has made > fetching fail without returning records in some situations. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15986) New consumer group protocol integration test failures
Andrew Schofield created KAFKA-15986: Summary: New consumer group protocol integration test failures Key: KAFKA-15986 URL: https://issues.apache.org/jira/browse/KAFKA-15986 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.7.0 A recent change in `AsyncKafkaConsumer.updateFetchPositions` has made fetching fail without returning records in some situations. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15932) Flaky test - PlaintextConsumerTest.testSeek("kraft+kip-848","consumer")
[
https://issues.apache.org/jira/browse/KAFKA-15932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Schofield resolved KAFKA-15932.
--
Resolution: Fixed
> Flaky test - PlaintextConsumerTest.testSeek("kraft+kip-848","consumer")
> ---
>
> Key: KAFKA-15932
> URL: https://issues.apache.org/jira/browse/KAFKA-15932
> Project: Kafka
> Issue Type: Bug
> Components: clients
>Affects Versions: 3.7.0
>Reporter: Andrew Schofield
>Assignee: Andrew Schofield
>Priority: Major
> Labels: flaky-test
>
> Intermittently failing test for the new consumer.
> https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14859/1/tests/
> ```Error
> org.opentest4j.AssertionFailedError: Timed out before consuming expected 1
> records. The number consumed was 0.
> Stacktrace
> org.opentest4j.AssertionFailedError: Timed out before consuming expected 1
> records. The number consumed was 0.
> at
> app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)
> at app//org.junit.jupiter.api.Assertions.fail(Assertions.java:134)
> at
> app//kafka.api.AbstractConsumerTest.consumeRecords(AbstractConsumerTest.scala:161)
> at
> app//kafka.api.AbstractConsumerTest.consumeAndVerifyRecords(AbstractConsumerTest.scala:128)
> at
> app//kafka.api.PlaintextConsumerTest.testSeek(PlaintextConsumerTest.scala:616)
> at
> java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base@11.0.16.1/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base@11.0.16.1/java.lang.reflect.Method.invoke(Method.java:566)
> at
> app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
> at
> app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
> at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
> at
> app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
> at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
> at
> app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
> at
> app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
> at
>
[jira] [Created] (KAFKA-15978) New consumer sends OffsetCommit with empty member ID
Andrew Schofield created KAFKA-15978: Summary: New consumer sends OffsetCommit with empty member ID Key: KAFKA-15978 URL: https://issues.apache.org/jira/browse/KAFKA-15978 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.7.0 Running the trogdor tests with the new consumer, it seemed that offsets were not being committed correctly, although the records were being fetched successfully. Upon investigation, it seems that the commit request manager uses a cached member ID which means that its OffsetCommit requests are rejected by the group coordinator. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once
[
https://issues.apache.org/jira/browse/KAFKA-15533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Schofield resolved KAFKA-15533.
--
Resolution: Resolved
Already resolved
> Ensure HeartbeatRequestManager only send out some fields once
> -
>
> Key: KAFKA-15533
> URL: https://issues.apache.org/jira/browse/KAFKA-15533
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
>Reporter: Philip Nee
>Assignee: Philip Nee
>Priority: Minor
> Labels: kip-848, kip-848-client-support
>
> We want to ensure ConsumerGroupHeartbeatRequest is as lightweight as
> possible, so a lot of fields in it don't need to be resend. An example would
> be the rebalanceTimeoutMs, currently we have the following code:
>
>
> {code:java}
> ConsumerGroupHeartbeatRequestData data = new
> ConsumerGroupHeartbeatRequestData()
> .setGroupId(membershipManager.groupId())
> .setMemberEpoch(membershipManager.memberEpoch())
> .setMemberId(membershipManager.memberId())
> .setRebalanceTimeoutMs(rebalanceTimeoutMs); {code}
>
>
> We should encapsulate these once-used fields into a class such as
> HeartbeatMetdataBuilder, and it should maintain a state of whether a certain
> field needs to be sent or not.
>
> Note that, currently only 3 fields are mandatory in the request:
> * groupId
> * memberEpoch
> * memberId
> Note that on retriable errors and network errors (ex. timeout) a full request
> should be sent to the broker.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (KAFKA-15971) Re-enable consumer integration tests for new consumer
Andrew Schofield created KAFKA-15971: Summary: Re-enable consumer integration tests for new consumer Key: KAFKA-15971 URL: https://issues.apache.org/jira/browse/KAFKA-15971 Project: Kafka Issue Type: Improvement Components: clients Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.7.0 Re-enable the consumer integration tests for the new consumer making sure that build stability is not impacted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15966) Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy
[ https://issues.apache.org/jira/browse/KAFKA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15966. -- Resolution: Duplicate Just fixed by https://github.com/apache/kafka/pull/14895 > Test failure - ClassCastException in > ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy > - > > Key: KAFKA-15966 > URL: https://issues.apache.org/jira/browse/KAFKA-15966 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 3.7.0 >Reporter: Andrew Schofield >Priority: Major > Labels: test-failure > > Gradle Test Run :streams:test > Gradle Test Executor 3 > > ConsistencyVectorIntegrationTest > > shouldHaveSamePositionBoundActiveAndStandBy FAILED > java.lang.ClassCastException: java.lang.Integer cannot be cast to > org.apache.kafka.streams.state.ValueAndTimestamp > at > org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.checkPosition(ConsistencyVectorIntegrationTest.java:169) > at > org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy(ConsistencyVectorIntegrationTest.java:140) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15953) Refactor polling delays
[ https://issues.apache.org/jira/browse/KAFKA-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15953. -- Resolution: Fixed > Refactor polling delays > --- > > Key: KAFKA-15953 > URL: https://issues.apache.org/jira/browse/KAFKA-15953 > Project: Kafka > Issue Type: Sub-task > Components: clients >Affects Versions: 3.7.0 >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Major > Fix For: 3.7.0 > > > This is a follow-on tasks for > https://issues.apache.org/jira/browse/KAFKA-15890. The idea is to reduce the > interaction between the application thread and the request managers which was > introduced in that earlier JIRA's patch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15966) Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy
Andrew Schofield created KAFKA-15966: Summary: Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy Key: KAFKA-15966 URL: https://issues.apache.org/jira/browse/KAFKA-15966 Project: Kafka Issue Type: Bug Components: streams Affects Versions: 3.7.0 Reporter: Andrew Schofield Gradle Test Run :streams:test > Gradle Test Executor 3 > ConsistencyVectorIntegrationTest > shouldHaveSamePositionBoundActiveAndStandBy FAILED java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.kafka.streams.state.ValueAndTimestamp at org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.checkPosition(ConsistencyVectorIntegrationTest.java:169) at org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy(ConsistencyVectorIntegrationTest.java:140) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15953) Refactor polling delays
Andrew Schofield created KAFKA-15953: Summary: Refactor polling delays Key: KAFKA-15953 URL: https://issues.apache.org/jira/browse/KAFKA-15953 Project: Kafka Issue Type: Sub-task Components: clients Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.7.0 This is a follow-on tasks for https://issues.apache.org/jira/browse/KAFKA-15890. The idea is to reduce the interaction between the application thread and the request managers which was introduced in that earlier JIRA's patch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15890) Consumer.poll with long timeout unaware of assigned partitions
[ https://issues.apache.org/jira/browse/KAFKA-15890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15890. -- Resolution: Fixed > Consumer.poll with long timeout unaware of assigned partitions > -- > > Key: KAFKA-15890 > URL: https://issues.apache.org/jira/browse/KAFKA-15890 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Major > Labels: consumer-threading-refactor, kip-848-client-support, > kip-848-preview > > Various problems found testing `kafka-console-consumer.sh` with the new > consumer, including NPEs, never-ending reconcilation states and failure to > fetch records. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15842) Correct handling of KafkaConsumer.committed for new consumer
[ https://issues.apache.org/jira/browse/KAFKA-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15842. -- Resolution: Fixed > Correct handling of KafkaConsumer.committed for new consumer > > > Key: KAFKA-15842 > URL: https://issues.apache.org/jira/browse/KAFKA-15842 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Minor > Labels: kip-848-client-support, kip-848-e2e, kip-848-preview > > KafkaConsumer.committed throws TimeOutException when there is no response. > The new consumer currently returns a null. Changing the new consumer to > behave like the old consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15945) Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest
Andrew Schofield created KAFKA-15945: Summary: Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest Key: KAFKA-15945 URL: https://issues.apache.org/jira/browse/KAFKA-15945 Project: Kafka Issue Type: Bug Components: mirrormaker Affects Versions: 3.7.0 Reporter: Andrew Schofield Last seen: https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14811/7/tests Error org.opentest4j.AssertionFailedError: `delete.retention.ms` should be 2000, because it's explicitly defined on the target topic! ==> expected: <2000> but was: <8640> Stacktrace org.opentest4j.AssertionFailedError: `delete.retention.ms` should be 2000, because it's explicitly defined on the target topic! ==> expected: <2000> but was: <8640> at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) at app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182) at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1152) at app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.lambda$testSyncTopicConfigs$8(MirrorConnectorsIntegrationBaseTest.java:780) at app//org.apache.kafka.test.TestUtils.lambda$waitForCondition$3(TestUtils.java:331) at app//org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:379) at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:328) at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:312) at app//org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:302) at app//org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testSyncTopicConfigs(MirrorConnectorsIntegrationBaseTest.java:774) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.7/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.7/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.7/java.lang.reflect.Method.invoke(Method.java:568) at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at
[jira] [Created] (KAFKA-15944) Flaky test - verifyStore[cache=false, log=true, supplier=ROCKS_KV, kind=DSL] – org.apache.kafka.streams.integration.PositionRestartIntegrationTest
Andrew Schofield created KAFKA-15944:
Summary: Flaky test - verifyStore[cache=false, log=true,
supplier=ROCKS_KV, kind=DSL] –
org.apache.kafka.streams.integration.PositionRestartIntegrationTest
Key: KAFKA-15944
URL: https://issues.apache.org/jira/browse/KAFKA-15944
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 3.7.0
Reporter: Andrew Schofield
Error
org.apache.kafka.common.errors.TimeoutException: The query never returned
within the bound. Last result:
StateQueryResult{partitionResults={0=SucceededQueryResult{result=org.apache.kafka.streams.state.internals.MeteredKeyValueStore$MeteredKeyValueTimestampedIterator@61b360a4,
executionInfo=[], position=Position{position={input-topic={0=1,
1=FailedQueryResult{failureReason=NOT_PRESENT, failure='The requested partition
was not present at the time of the query.', executionInfo=[], position=null}},
globalResult=null}
Stacktrace
org.apache.kafka.common.errors.TimeoutException: The query never returned
within the bound. Last result:
StateQueryResult{partitionResults={0=SucceededQueryResult{result=org.apache.kafka.streams.state.internals.MeteredKeyValueStore$MeteredKeyValueTimestampedIterator@61b360a4,
executionInfo=[], position=Position{position={input-topic={0=1,
1=FailedQueryResult{failureReason=NOT_PRESENT, failure='The requested partition
was not present at the time of the query.', executionInfo=[], position=null}},
globalResult=null}
Standard Output
[2023-11-28 22:52:47,353] INFO [Producer clientId=producer-129] Instantiated an
idempotent producer. (org.apache.kafka.clients.producer.KafkaProducer:587)
[2023-11-28 22:52:47,466] INFO [Producer clientId=producer-129] ProducerId set
to 0 with epoch 0
(org.apache.kafka.clients.producer.internals.TransactionManager:502)
[2023-11-28 22:52:47,473] INFO [Producer clientId=producer-129] Closing the
Kafka producer with timeoutMillis = 9223372036854775807 ms.
(org.apache.kafka.clients.producer.KafkaProducer:1332)
[2023-11-28 22:52:47,531] INFO stream-client
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57]
Kafka Streams version: test-version (org.apache.kafka.streams.KafkaStreams:914)
[2023-11-28 22:52:47,531] INFO stream-client
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57]
Kafka Streams commit ID: test-commit-ID
(org.apache.kafka.streams.KafkaStreams:915)
[2023-11-28 22:52:47,532] INFO stream-thread
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1]
Creating restore consumer client
(org.apache.kafka.streams.processor.internals.StreamThread:365)
[2023-11-28 22:52:47,537] INFO stream-thread
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1]
Creating thread producer client
(org.apache.kafka.streams.processor.internals.StreamThread:105)
[2023-11-28 22:52:47,538] INFO [Producer
clientId=app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1-producer]
Instantiated an idempotent producer.
(org.apache.kafka.clients.producer.KafkaProducer:587)
[2023-11-28 22:52:47,545] INFO stream-thread
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1]
Creating consumer client
(org.apache.kafka.streams.processor.internals.StreamThread:432)
[2023-11-28 22:52:47,545] INFO state-updater
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StateUpdater-1]
State updater thread started
(org.apache.kafka.streams.processor.internals.DefaultStateUpdater:135)
[2023-11-28 22:52:47,547] INFO [Producer
clientId=app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1-producer]
ProducerId set to 1 with epoch 0
(org.apache.kafka.clients.producer.internals.TransactionManager:502)
[2023-11-28 22:52:47,550] INFO stream-thread
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57-StreamThread-1-consumer]
Cooperative rebalancing protocol is enabled now
(org.apache.kafka.streams.processor.internals.assignment.AssignorConfiguration:141)
[2023-11-28 22:52:47,552] INFO stream-client
[app-org.apache.kafka.streams.integration.PositionRestartIntegrationTest-true-true-IN_MEMORY_KV-DSL-456895e1-b230-4a83-9164-8e6d65d1fc57]
State
[jira] [Created] (KAFKA-15943) Flaky test - testHighWaterMarkAfterPartitionReassignment(String).quorum=kraft – org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest
Andrew Schofield created KAFKA-15943: Summary: Flaky test - testHighWaterMarkAfterPartitionReassignment(String).quorum=kraft – org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest Key: KAFKA-15943 URL: https://issues.apache.org/jira/browse/KAFKA-15943 Project: Kafka Issue Type: Bug Components: tools Affects Versions: 3.7.0 Reporter: Andrew Schofield Error org.opentest4j.AssertionFailedError: Timed out waiting for verifyAssignment result org.apache.kafka.tools.reassign.VerifyAssignmentResult@9c4dd302. The latest result was org.apache.kafka.tools.reassign.VerifyAssignmentResult@cc845dc Stacktrace org.opentest4j.AssertionFailedError: Timed out waiting for verifyAssignment result org.apache.kafka.tools.reassign.VerifyAssignmentResult@9c4dd302. The latest result was org.apache.kafka.tools.reassign.VerifyAssignmentResult@cc845dc at app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38) at app//org.junit.jupiter.api.Assertions.fail(Assertions.java:134) at app//kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:1141) at app//kafka.utils.TestUtils.waitUntilTrue(TestUtils.scala) at app//org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest.waitForVerifyAssignment(ReassignPartitionsIntegrationTest.java:685) at app//org.apache.kafka.tools.reassign.ReassignPartitionsIntegrationTest.testHighWaterMarkAfterPartitionReassignment(ReassignPartitionsIntegrationTest.java:216) at java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base@11.0.16.1/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@11.0.16.1/java.lang.reflect.Method.invoke(Method.java:566) at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728) at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45) at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139) at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) at
[jira] [Created] (KAFKA-15932) Flaky test - PlaintextConsumerTest.testSeek("kraft+kip-848","consumer")
Andrew Schofield created KAFKA-15932:
Summary: Flaky test -
PlaintextConsumerTest.testSeek("kraft+kip-848","consumer")
Key: KAFKA-15932
URL: https://issues.apache.org/jira/browse/KAFKA-15932
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 3.7.0
Reporter: Andrew Schofield
Intermittently failing test for the new consumer.
https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14859/1/tests/
```Error
org.opentest4j.AssertionFailedError: Timed out before consuming expected 1
records. The number consumed was 0.
Stacktrace
org.opentest4j.AssertionFailedError: Timed out before consuming expected 1
records. The number consumed was 0.
at
app//org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:38)
at app//org.junit.jupiter.api.Assertions.fail(Assertions.java:134)
at
app//kafka.api.AbstractConsumerTest.consumeRecords(AbstractConsumerTest.scala:161)
at
app//kafka.api.AbstractConsumerTest.consumeAndVerifyRecords(AbstractConsumerTest.scala:128)
at
app//kafka.api.PlaintextConsumerTest.testSeek(PlaintextConsumerTest.scala:616)
at
java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base@11.0.16.1/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base@11.0.16.1/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@11.0.16.1/java.lang.reflect.Method.invoke(Method.java:566)
at
app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at
app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at
app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at
app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at
app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
at
app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at
app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at
app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at
app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at
[jira] [Resolved] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client
[ https://issues.apache.org/jira/browse/KAFKA-15840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15840. -- Resolution: Duplicate > Correct initialization of ConsumerGroupHeartbeat by client > -- > > Key: KAFKA-15840 > URL: https://issues.apache.org/jira/browse/KAFKA-15840 > Project: Kafka > Issue Type: Sub-task > Components: clients, consumer >Reporter: Andrew Schofield >Assignee: Andrew Schofield >Priority: Major > Labels: kip-848-client-support, kip-848-e2e, kip-848-preview > > The new consumer using the KIP-848 protocol currently leaves the > TopicPartitions set to null for the ConsumerGroupHeartbeat request, even when > the MemberEpoch is zero. This violates the KIP which expects the list to be > empty (but not null). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15890) kafka-console-consumer.sh problems with new consumer
Andrew Schofield created KAFKA-15890: Summary: kafka-console-consumer.sh problems with new consumer Key: KAFKA-15890 URL: https://issues.apache.org/jira/browse/KAFKA-15890 Project: Kafka Issue Type: Bug Components: clients, consumer Reporter: Andrew Schofield Assignee: Andrew Schofield Various problems found testing `kafka-console-consumer.sh` with the new consumer, including NPEs, never-ending reconcilation states and failure to fetch records. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15871) Implement kafka-client-metrics.sh tool
Andrew Schofield created KAFKA-15871: Summary: Implement kafka-client-metrics.sh tool Key: KAFKA-15871 URL: https://issues.apache.org/jira/browse/KAFKA-15871 Project: Kafka Issue Type: Sub-task Components: admin Affects Versions: 3.7.0 Reporter: Andrew Schofield Assignee: Andrew Schofield Fix For: 3.7.0 Implement the `kafka-client-metrics.sh` tool which is introduced in KIP-714 and enhanced in KIP-1000. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KAFKA-15837) Throw error on use of Consumer.poll(long timeout)
[ https://issues.apache.org/jira/browse/KAFKA-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15837. -- Resolution: Fixed > Throw error on use of Consumer.poll(long timeout) > - > > Key: KAFKA-15837 > URL: https://issues.apache.org/jira/browse/KAFKA-15837 > Project: Kafka > Issue Type: Improvement > Components: clients, consumer >Reporter: Kirk True >Assignee: Andrew Schofield >Priority: Major > Fix For: 3.7.0 > > > Per [KIP-266|https://cwiki.apache.org/confluence/x/5kiHB], the > Consumer.poll(long timeout) method was deprecated back in 2.0.0. The method > will now throw a KafkaException. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15842) Correct handling of KafkaConsumer.committed for new consumer
Andrew Schofield created KAFKA-15842: Summary: Correct handling of KafkaConsumer.committed for new consumer Key: KAFKA-15842 URL: https://issues.apache.org/jira/browse/KAFKA-15842 Project: Kafka Issue Type: Sub-task Components: clients Reporter: Andrew Schofield Assignee: Andrew Schofield KafkaConsumer.committed throws TimeOutException when there is no response. The new consumer currently returns a null. Changing the new consumer to behave like the old consumer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client
Andrew Schofield created KAFKA-15840: Summary: Correct initialization of ConsumerGroupHeartbeat by client Key: KAFKA-15840 URL: https://issues.apache.org/jira/browse/KAFKA-15840 Project: Kafka Issue Type: Sub-task Components: clients Reporter: Andrew Schofield Assignee: Andrew Schofield The new consumer using the KIP-848 protocol currently leaves the TopicPartitions set to null for the ConsumerGroupHeartbeat request, even when the MemberEpoch is zero. This violates the KIP which expects the list to be empty (but not null). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-15831) List Client Metrics Configuration Resources
Andrew Schofield created KAFKA-15831: Summary: List Client Metrics Configuration Resources Key: KAFKA-15831 URL: https://issues.apache.org/jira/browse/KAFKA-15831 Project: Kafka Issue Type: Improvement Components: admin Reporter: Andrew Schofield Assignee: Andrew Schofield This JIRA tracks the development of KIP-1000 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (KAFKA-7841) KIP-419 Safely notify Kafka Connect SourceTask is stopped
Andrew Schofield created KAFKA-7841: --- Summary: KIP-419 Safely notify Kafka Connect SourceTask is stopped Key: KAFKA-7841 URL: https://issues.apache.org/jira/browse/KAFKA-7841 Project: Kafka Issue Type: New Feature Components: KafkaConnect Affects Versions: 2.2.0 Reporter: Andrew Schofield Implements KIP 419. https://cwiki.apache.org/confluence/display/KAFKA/KIP-419%3A+Safely+notify+Kafka+Connect+SourceTask+is+stopped -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KAFKA-7776) Kafka Connect values converter parsing of ISO8601 not working properly
Andrew Schofield created KAFKA-7776: --- Summary: Kafka Connect values converter parsing of ISO8601 not working properly Key: KAFKA-7776 URL: https://issues.apache.org/jira/browse/KAFKA-7776 Project: Kafka Issue Type: Bug Components: KafkaConnect Affects Versions: 2.0.0, 1.1.0 Reporter: Andrew Schofield In org.apache.kafka.connect.data.Values, the Values.convertToDate/Time/Timestamp methods are intended to be able to accept Strings in ISO8601 format and convert into the Kafka Connect formats. However, the parser for strings incorrectly tokenizes the strings (having real trouble with colons) which means that the correct ISO8601 format is never actual presented as a single piece to the code that converts it into java.util.Date. The parser needs to be enhanced to accept an ISO8601 string as a single token, probably only when it knows that the intended use is as one of the date-based logical types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
