[
https://issues.apache.org/jira/browse/KAFKA-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587475#comment-16587475
]
Seweryn Habdank-Wojewodzki commented on KAFKA-7214:
---------------------------------------------------
Hi,
I had updated Kafka client to 1.1.1. I have similar.
max.poll.interval.ms = 10 000 000 ~ 2,7 Hours
max.poll.records=500
Usual system message system processing is ~ 5000 Msg/s
2018-08-21 15:59:22 [] [ERROR] StreamTask:550 - task [0_0] Could not close
task due to the following error:
org.apache.kafka.streams.errors.TaskMigratedException: StreamsTask taskId: 0_0
ProcessorTopology:
KSTREAM-SOURCE-0000000000:
topics: [my_topic]
children: [KSTREAM-FILTER-0000000001]
KSTREAM-FILTER-0000000001:
children: [KSTREAM-MAP-0000000002]
KSTREAM-MAP-0000000002:
children: [KSTREAM-SINK-0000000003]
KSTREAM-SINK-0000000003:
topic: other_topic
Partitions [my_topic-0]
at org.apache.kafka.streams.processor.internals.StreamTask.commitOffsets
(StreamTask.java:380) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.access$000(St
reamTask.java:53) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask$1.run(StreamT
ask.java:316) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measu
reLatencyNs(StreamsMetricsImpl.java:211) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.commit(Stream
Task.java:307) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.suspend(Strea
mTask.java:440) ~[restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamTask.close(StreamT
ask.java:546) [restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.AssignedTasks.close(Assi
gnedTasks.java:405) [restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(Tas
kManager.java:260) [restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamThread.completeShu
tdown(StreamThread.java:1111) [restreamer.jar:?]
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamT
hread.java:730) [restreamer.jar:?]
Caused by: org.apache.kafka.clients.consumer.CommitFailedException: Commit canno
t be completed since the group has already rebalanced and assigned the partition
s to another member. This means that the time between subsequent calls to poll()
was longer than the configured max.poll.interval.ms, which typically implies th
at the poll loop is spending too much time message processing. You can address t
his either by increasing the session timeout or by reducing the maximum size of
batches returned in poll() with max.poll.records.
> Mystic FATAL error
> ------------------
>
> Key: KAFKA-7214
> URL: https://issues.apache.org/jira/browse/KAFKA-7214
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.11.0.3
> Reporter: Seweryn Habdank-Wojewodzki
> Priority: Critical
>
> Dears,
> Very often at startup of the streaming application I got exception:
> {code}
> Exception caught in process. taskId=0_1, processor=KSTREAM-SOURCE-0000000000,
> topic=my_instance_medium_topic, partition=1, offset=198900203;
> [org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:212),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks$2.apply(AssignedTasks.java:347),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks.applyToRunningTasks(AssignedTasks.java:420),
>
> org.apache.kafka.streams.processor.internals.AssignedTasks.process(AssignedTasks.java:339),
>
> org.apache.kafka.streams.processor.internals.StreamThread.processAndPunctuate(StreamThread.java:648),
>
> org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:513),
>
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:482),
>
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:459)]
> in thread
> my_application-my_instance-my_instance_medium-72ee1819-edeb-4d85-9d65-f67f7c321618-StreamThread-62
> {code}
> and then (without shutdown request from my side):
> {code}
> 2018-07-30 07:45:02 [ar313] [INFO ] StreamThread:912 - stream-thread
> [my_application-my_instance-my_instance-72ee1819-edeb-4d85-9d65-f67f7c321618-StreamThread-62]
> State transition from PENDING_SHUTDOWN to DEAD.
> {code}
> What is this?
> How to correctly handle it?
> Thanks in advance for help.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
