[
https://issues.apache.org/jira/browse/KAFKA-9645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang resolved KAFKA-9645.
----------------------------------
Fix Version/s: 2.6.0
Resolution: Fixed
> Records could not find corresponding partition/task
> ---------------------------------------------------
>
> Key: KAFKA-9645
> URL: https://issues.apache.org/jira/browse/KAFKA-9645
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 2.6.0
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
> Fix For: 2.6.0
>
>
> We could be hitting the illegal state when stream kicks off a rebalance with
> all tasks closed:
> ```
> [2020-03-03T18:36:09-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,105] WARN
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> Detected that the thread is being fenced. This implies that this thread
> missed a rebalance and dropped out of the consumer group. Will close out all
> assigned tasks and rejoin the consumer group.
> (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:09-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,105] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Consumer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer,
> groupId=null] Subscribed to partition(s):
> stream-soak-test-KSTREAM-AGGREGATE-STATE-STORE-0000000049-changelog-2
> (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,286] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Producer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-1_1-producer,
> transactionalId=stream-soak-test-1_1] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,287] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task
> [1_1] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,287] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Consumer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer,
> groupId=null] Unsubscribed all topics or patterns and assigned partitions
> (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,290] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Producer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-3_2-producer,
> transactionalId=stream-soak-test-3_2] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,292] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task
> [3_2] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,293] ERROR
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] Unable
> to locate active task for received-record partition node-name-repartition-1.
> Current tasks: TaskManager
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) >
> MetadataState:
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) > Tasks:
> (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,293] ERROR
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> Encountered the following exception during processing and the thread is going
> to shut down: (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog)
> java.lang.NullPointerException: Task was unexpectedly missing for partition
> node-name-repartition-1
> at
> org.apache.kafka.streams.processor.internals.StreamThread.addRecordsToTasks(StreamThread.java:984)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:820)
> ```
> We should be more careful in this case by avoiding processing data when no
> corresponding task is found.
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:751)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:725)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,294] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] State
> transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.processor.internals.StreamThread)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
