Boyang Chen created KAFKA-10011:
-----------------------------------
Summary: lockedTaskDirectories should be cleared when task gets
closed dirty in HandleLostAll
Key: KAFKA-10011
URL: https://issues.apache.org/jira/browse/KAFKA-10011
Project: Kafka
Issue Type: Bug
Reporter: Boyang Chen
Assignee: Boyang Chen
Tasks who get closed in handleLostAll don't clear out their position inside
lockedTaskDirectories, which causes an illegal state afterwards:
{code:java}
[2020-05-17T06:21:54-07:00]
(streams-soak-trunk-eos-beta_soak_i-0b021dbf00474b6aa_streamslog) [2020-05-17
13:21:54,127] ERROR
[stream-soak-test-150cf9ae-793b-4aac-bea0-0fb61d228b39-StreamThread-3]
stream-thread
[stream-soak-test-150cf9ae-793b-4aac-bea0-0fb61d228b39-StreamThread-3]
Encountered the following exception during processing and the thread is going
to shut down: (org.apache.kafka.streams.processor.internals.StreamThread)
[2020-05-17T06:21:54-07:00]
(streams-soak-trunk-eos-beta_soak_i-0b021dbf00474b6aa_streamslog)
org.apache.kafka.streams.errors.ProcessorStateException: task directory
[/mnt/run/streams/state/stream-soak-test/3_1] doesn't exist and couldn't be
created
at
org.apache.kafka.streams.processor.internals.StateDirectory.directoryForTask(StateDirectory.java:112)
at
org.apache.kafka.streams.processor.internals.StateDirectory.checkpointFileFor(StateDirectory.java:121)
at
org.apache.kafka.streams.processor.internals.TaskManager.getTaskOffsetSums(TaskManager.java:498)
at
org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor.subscriptionUserData(StreamsPartitionAssignor.java:239)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.metadata(ConsumerCoordinator.java:222)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest(AbstractCoordinator.java:560)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.initiateJoinGroup(AbstractCoordinator.java:495)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:417)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:358)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:506)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1265)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1231)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1206)
at
org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:770)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:630)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:550)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
