KUMAR created KAFKA-8737: ---------------------------- Summary: TaskMigrated Exception while rebalancing kafka streams Key: KAFKA-8737 URL: https://issues.apache.org/jira/browse/KAFKA-8737 Project: Kafka Issue Type: Bug Components: streams Affects Versions: 1.0.1, 1.0.0 Environment: 20 partitions 1 topic 8 Streamer service topic-region-1 9 7841726 8236017 394291 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer topic-region-1 15 7421710 7467666 45956 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer topic-region-1 19 7737360 8120611 383251 streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer-0276e83d-40b5-4b44-b764-7d29e0dab663/ streams-subscriberstopic-region-1-29d615ed-4243-4b9d-90b7-9c517aa0f2e3-StreamThread-1-consumer topic-region-1 Reporter: KUMAR
Kafka streams throws following exception while restart of a stream client service - o.a.k.s.p.internals.StreamThread.? - stream-thread [streams-subscriberstopic-region-1-32d968e3-f892-4772-a7a4-6f684d7e43c9-StreamThread-1] Detected a task that got migrated to another thread. This implies that this thread missed a rebalance and dropped out of the consumer group. Trying to rejoin the consumer group now. org.apache.kafka.streams.errors.TaskMigratedException: Log end offset of topic-region-1-12 should not change while restoring: old end offset 6286727, current offset 6380997 Kafka version is 1.0.0 and we have back merged the fix for KIP-6269- [https://github.com/apache/kafka/pull/4300/files#|https://github.com/apache/kafka/pull/4300/files] However we observe that there seems to be an issue in rebalance when "auto.offset.reset" is configured as "latest". Based on log analysis we see following behavior - # StreamThread starts a restore consumer # While Fetching it gets offset out of range o.a.k.c.consumer.internals.Fetcher.? - [Consumer clientId=streams-subscriberstopic-region-1-11b2d7fb-11ce-4b0b-a40a-388d3c7b6bc9-StreamThread-1-restore- consumer, groupId=] Fetch READ_UNCOMMITTED at offset 246431 for partition topic-region-1-12 returned fetch data (error=OFFSET_OUT_OF_RANGE, highWaterMark=-1, lastStableOffset = -1, logStartOffset = -1, abortedTransactions = null, recordsSizeInBytes=0) # Fetcher tries to reset the offset # While reset the offset it appears it is changing the offset position and causing TaskMigrated exception Above test repeated with "auto.offset.reset" is configured as "earliest" does not throw any TaskMigrated exception as in earliest case we are not reseting the restore consumer position. Please let us know if this is possible and if a fix would be needed for the offset reset piece when set to latest. -- This message was sent by Atlassian JIRA (v7.6.14#76016)