[
https://issues.apache.org/jira/browse/KAFKA-19906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colt McNealy updated KAFKA-19906:
---------------------------------
Description:
We saw this stacktrace when starting up a new instance. It appears to be very
non-deterministic.
```
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.transitRestoredTaskToRunning(TaskManager.java:980)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.handleRestoredTasksFromStateUpdater(TaskManager.java:1056)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:923)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1433)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1239)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:926)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:886)
[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928Caused by: java.lang.NullPointerException: Cannot invoke
"Object.hashCode()" because "key" is null
2025-11-21 12:56:11.928 at
java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1125)
~[?:?]
2025-11-21 12:56:11.928 at
java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1116)
~[?:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.common.metrics.Metrics.removeMetric(Metrics.java:550)
~[kafka-clients-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.removeMetric(StreamsMetricsImpl.java:341)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.internals.metrics.OpenIterators.remove(OpenIterators.java:68)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore$MeteredKeyValueIterator.close(MeteredKeyValueStore.java:467)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.store.LHKeyValueIterator.close(LHKeyValueIterator.java:33)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.topology.core.processors.CommandProcessor.onPartitionClaimed(CommandProcessor.java:142)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.topology.core.processors.CommandProcessor.init(CommandProcessor.java:85)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.ProcessorNode.init(ProcessorNode.java:123)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 ... 10 more
```
We were running 4.1.1 with 2 streamthreads. This was when a new instance was
joining the group.
was:
We saw this stacktrace when starting up a new instance. It appears to be very
non-deterministic.
```
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.transitRestoredTaskToRunning(TaskManager.java:980)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.handleRestoredTasksFromStateUpdater(TaskManager.java:1056)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:923)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1433)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1239)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:926)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:886)
[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928Caused by: java.lang.NullPointerException: Cannot invoke
"Object.hashCode()" because "key" is null
2025-11-21 12:56:11.928 at
java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1125)
~[?:?]
2025-11-21 12:56:11.928 at
java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1116)
~[?:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.common.metrics.Metrics.removeMetric(Metrics.java:550)
~[kafka-clients-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.removeMetric(StreamsMetricsImpl.java:341)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.internals.metrics.OpenIterators.remove(OpenIterators.java:68)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.state.internals.MeteredKeyValueStore$MeteredKeyValueIterator.close(MeteredKeyValueStore.java:467)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.store.LHKeyValueIterator.close(LHKeyValueIterator.java:33)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.topology.core.processors.CommandProcessor.onPartitionClaimed(CommandProcessor.java:142)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
io.littlehorse.server.streams.topology.core.processors.CommandProcessor.init(CommandProcessor.java:85)
~[server-0.16.0-SNAPSHOT.jar:?]
2025-11-21 12:56:11.928 at
org.apache.kafka.streams.processor.internals.ProcessorNode.init(ProcessorNode.java:123)
~[kafka-streams-4.1.1.jar:?]
2025-11-21 12:56:11.928 ... 10 more
```
> NullPointerException from Streams Metrics Infra on Topology Startup
> -------------------------------------------------------------------
>
> Key: KAFKA-19906
> URL: https://issues.apache.org/jira/browse/KAFKA-19906
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.1.1
> Reporter: Colt McNealy
> Priority: Major
>
> We saw this stacktrace when starting up a new instance. It appears to be very
> non-deterministic.
> ```
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.TaskManager.transitRestoredTaskToRunning(TaskManager.java:980)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.TaskManager.handleRestoredTasksFromStateUpdater(TaskManager.java:1056)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:923)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.StreamThread.checkStateUpdater(StreamThread.java:1433)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.StreamThread.runOnceWithoutProcessingThreads(StreamThread.java:1239)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:926)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:886)
> [kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928Caused by: java.lang.NullPointerException: Cannot
> invoke "Object.hashCode()" because "key" is null
> 2025-11-21 12:56:11.928 at
> java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1125)
> ~[?:?]
> 2025-11-21 12:56:11.928 at
> java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1116)
> ~[?:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.common.metrics.Metrics.removeMetric(Metrics.java:550)
> ~[kafka-clients-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.removeMetric(StreamsMetricsImpl.java:341)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.internals.metrics.OpenIterators.remove(OpenIterators.java:68)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.state.internals.MeteredKeyValueStore$MeteredKeyValueIterator.close(MeteredKeyValueStore.java:467)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 at
> io.littlehorse.server.streams.store.LHKeyValueIterator.close(LHKeyValueIterator.java:33)
> ~[server-0.16.0-SNAPSHOT.jar:?]
> 2025-11-21 12:56:11.928 at
> io.littlehorse.server.streams.topology.core.processors.CommandProcessor.onPartitionClaimed(CommandProcessor.java:142)
> ~[server-0.16.0-SNAPSHOT.jar:?]
> 2025-11-21 12:56:11.928 at
> io.littlehorse.server.streams.topology.core.processors.CommandProcessor.init(CommandProcessor.java:85)
> ~[server-0.16.0-SNAPSHOT.jar:?]
> 2025-11-21 12:56:11.928 at
> org.apache.kafka.streams.processor.internals.ProcessorNode.init(ProcessorNode.java:123)
> ~[kafka-streams-4.1.1.jar:?]
> 2025-11-21 12:56:11.928 ... 10 more
> ```
>
> We were running 4.1.1 with 2 streamthreads. This was when a new instance was
> joining the group.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
