[
https://issues.apache.org/jira/browse/KAFKA-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18056188#comment-18056188
]
Jochen Schalanda edited comment on KAFKA-13009 at 2/3/26 11:50 AM:
-------------------------------------------------------------------
We just encountered the same situation after introducing a new GlobalKTable in
our topology with Kafka Streams 4.1.1
{code:java}
org.apache.kafka.streams.errors.StreamsException: Encounter unexpected fatal
error for task 1_0
at
org.apache.kafka.streams.processor.internals.TaskManager.maybeThrowTaskExceptions(TaskManager.java:428)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:1009)
at
org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:918)
at
org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1433)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1239)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:926)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:886)
Caused by: java.lang.IllegalStateException: Metrics recorder is re-initialised
with different task: previous task is -1_-1 whereas current task is 1_0. This
is a bug in Kafka Streams. Please open a bug report under
https://issues.apache.org/jira/projects/KAFKA/issues
at
org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorder.init(RocksDBMetricsRecorder.java:140)
at
org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:161)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:63)
at
org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:106)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:63)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$2(MeteredKeyValueStore.java:135)
at
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:927)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:135)
at
org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:233)
at
org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:103)
at
org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:112)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTaskToStateUpdater(TaskManager.java:1016)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:1001)
... 5 more
{code}
was (Author: joschi):
We just encountered the same situation after introducing a new GlobalKTable in
our topology with Kafka Streams 4.1.1
```
org.apache.kafka.streams.errors.StreamsException: Encounter unexpected fatal
error for task 1_0
at
org.apache.kafka.streams.processor.internals.TaskManager.maybeThrowTaskExceptions(TaskManager.java:428)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:1009)
at
org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:918)
at
org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1433)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1239)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:926)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:886)
Caused by: java.lang.IllegalStateException: Metrics recorder is re-initialised
with different task: previous task is -1_-1 whereas current task is 1_0. This
is a bug in Kafka Streams. Please open a bug report under
https://issues.apache.org/jira/projects/KAFKA/issues
at
org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorder.init(RocksDBMetricsRecorder.java:140)
at
org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:161)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:63)
at
org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:106)
at
org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:63)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$2(MeteredKeyValueStore.java:135)
at
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:927)
at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:135)
at
org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:233)
at
org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:103)
at
org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:112)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTaskToStateUpdater(TaskManager.java:1016)
at
org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:1001)
... 5 more
```
> Metrics recorder is re-initialised with different task
> ------------------------------------------------------
>
> Key: KAFKA-13009
> URL: https://issues.apache.org/jira/browse/KAFKA-13009
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.7.0
> Environment: Docker container
> Reporter: Victor van den Hoven
> Priority: Major
>
> When starting my Kafka Stream application, I get in the Logs:
>
> [SmartMeterActionService-e0d0f403-87c7-4502-b1be-875d544899e2-StreamThread-1]
> State transition from STARTING to
> PARTITIONS_ASSIGNED[SmartMeterActionService-e0d0f403-87c7-4502-b1be-875d544899e2-StreamThread-1]
> State transition from STARTING to PARTITIONS_ASSIGNED2021-06-29 07:35:58.258
> ERROR 1 — [-StreamThread-1] o.a.k.s.p.internals.StreamThread :
> stream-thread
> [SmartMeterActionService-e0d0f403-87c7-4502-b1be-875d544899e2-StreamThread-1]
> Encountered the following exception during processing and the thread is going
> to shut down:
> java.lang.IllegalStateException: Metrics recorder is re-initialised with
> different task: previous task is -1_-1 whereas current task is 0_1.
> *{color:#ff0000}This is a bug in Kafka Streams. Please open a bug report
> under [https://issues.apache.org/jira/projects/KAFKA/issues] at{color}*
> org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorder.init(RocksDBMetricsRecorder.java:137)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:252)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:55)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:74)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:55)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$1(MeteredKeyValueStore.java:120)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:883)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:120)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:201)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:103)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StandbyTask.initializeIfNeeded(StandbyTask.java:93)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.TaskManager.tryToCompleteRestoration(TaskManager.java:473)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StreamThread.initializeAndRestorePhase(StreamThread.java:728)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:625)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:553)
> ~[kafka-streams-2.7.0.jar!/:na] at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:512)
> ~[kafka-streams-2.7.0.jar!/:na]
> 2021-06-29 07:35:58.259 INFO 1 — [-StreamThread-1]
> o.a.k.s.p.internals.StreamThread : stream-thread
> [SmartMeterActionService-e0d0f403-87c7-4502-b1be-875d544899e2-StreamThread-1]
> State transition from PARTITIONS_ASSIGNED to PENDING_SHUTDOWN2021-06-29
> 07:35:58.259 INFO 1 — [-StreamThread-1] o.a.k.s.p.internals.StreamThread
> : stream-thread
> [SmartMeterActionService-e0d0f403-87c7-4502-b1be-875d544899e2-StreamThread-1]
> Shutting down
>
> After this the application shuts down!
>
>
> After removing the internal change-log-topic the application could start
> again without the issue.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
