[
https://issues.apache.org/jira/browse/KAFKA-9645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050887#comment-17050887
]
ASF GitHub Bot commented on KAFKA-9645:
---------------------------------------
abbccdda commented on pull request #8220: KAFKA-9645: Remove Illegal State
Check for Records Addition
URL: https://github.com/apache/kafka/pull/8220
After https://github.com/apache/kafka/pull/7312/, we could still return data
during the rebalance phase, which means it could be possible to find records
without corresponding tasks. Do not throw illegal state here and just ignore
should be safe.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Records could not find corresponding partition/task
> ---------------------------------------------------
>
> Key: KAFKA-9645
> URL: https://issues.apache.org/jira/browse/KAFKA-9645
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.6.0
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
>
> We could be hitting the illegal state when stream kicks off a rebalance with
> all tasks closed:
> ```
> [2020-03-03T18:36:09-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,105] WARN
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> Detected that the thread is being fenced. This implies that this thread
> missed a rebalance and dropped out of the consumer group. Will close out all
> assigned tasks and rejoin the consumer group.
> (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:09-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,105] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Consumer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer,
> groupId=null] Subscribed to partition(s):
> stream-soak-test-KSTREAM-AGGREGATE-STATE-STORE-0000000049-changelog-2
> (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,286] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Producer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-1_1-producer,
> transactionalId=stream-soak-test-1_1] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,287] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task
> [1_1] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,287] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Consumer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-restore-consumer,
> groupId=null] Unsubscribed all topics or patterns and assigned partitions
> (org.apache.kafka.clients.consumer.KafkaConsumer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,290] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> [Producer
> clientId=stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1-3_2-producer,
> transactionalId=stream-soak-test-3_2] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,292] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] task
> [3_2] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,293] ERROR
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] Unable
> to locate active task for received-record partition node-name-repartition-1.
> Current tasks: TaskManager
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) >
> MetadataState:
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) > Tasks:
> (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,293] ERROR
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> Encountered the following exception during processing and the thread is going
> to shut down: (org.apache.kafka.streams.processor.internals.StreamThread)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog)
> java.lang.NullPointerException: Task was unexpectedly missing for partition
> node-name-repartition-1
> at
> org.apache.kafka.streams.processor.internals.StreamThread.addRecordsToTasks(StreamThread.java:984)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:820)
> ```
> We should be more careful in this case by avoiding processing data when no
> corresponding task is found.
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:751)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:725)
> [2020-03-03T18:36:10-08:00]
> (streams-soak-trunk-eos_soak_i-050294ea2392cf355_streamslog) [2020-03-04
> 02:36:09,294] INFO
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1]
> stream-thread
> [stream-soak-test-e639a3a1-bd9d-49e0-896e-df9fe0cd63db-StreamThread-1] State
> transition from RUNNING to PENDING_SHUTDOWN
> (org.apache.kafka.streams.processor.internals.StreamThread)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
