Tomasz Kaszuba created KAFKA-10420:
--------------------------------------
Summary: group instance id Optional.empty failed due to
UNKNOWN_MEMBER_ID, resetting generation when running kafka client 2.6 against
broker 2.3.1
Key: KAFKA-10420
URL: https://issues.apache.org/jira/browse/KAFKA-10420
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 2.6.0
Reporter: Tomasz Kaszuba
After upgrading our kafka clients to 2.6.0 and running it against broker
version 2.3.1 we got errors where the consumer groups are reset. We didn't
notice this happening with client 2.5.0.
{noformat}
020-08-17 04:35:27.787 INFO 1 --- [-StreamThread-1]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Attempt to
heartbeat with Generation{generationId=11,
memberId='ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer-3902e2a9-1755-466b-9255-d144be25876f',
protocol='stream'} and group instance id Optional.empty failed due to
UNKNOWN_MEMBER_ID, resetting generation2020-08-17 04:35:27.787 INFO 1 ---
[-StreamThread-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Giving away all
assigned partitions as lost since generation has been reset,indicating that
consumer is no longer part of the group2020-08-17 04:35:27.787 INFO 1 ---
[-StreamThread-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Lost previously
assigned partitions ieb.publish.baseline_pc.incurred_pattern-02020-08-17
04:35:27.787 INFO 1 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread
: stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1]
at state RUNNING: partitions [ieb.publish.baseline_pc.incurred_pattern-0] lost
due to missed rebalance. lost active tasks: [0_0] lost assigned
standby tasks: []2020-08-17 04:35:27.787 INFO 1 --- [-StreamThread-1]
o.a.k.s.processor.internals.StreamTask : stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1]
task [0_0] Suspended running 2020-08-17 04:35:27.788 INFO 1 ---
[-StreamThread-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-restore-consumer,
groupId=null] Unsubscribed all topics or patterns and assigned partitions
2020-08-17 04:35:27.789 INFO 1 --- [-StreamThread-1]
o.a.k.s.p.internals.RecordCollectorImpl : stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1]
task [0_0] Closing record collector dirty 2020-08-17 04:35:27.790 INFO 1 ---
[-StreamThread-1] o.a.k.s.processor.internals.StreamTask : stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1]
task [0_0] Closed dirty 2020-08-17 04:35:27.790 INFO 1 --- [-StreamThread-1]
o.a.k.clients.producer.KafkaProducer : [Producer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-0_0-producer,
transactionalId=ieb-x07-baseline-pc-data-storage-incurred-pattern-0_0] Closing
the Kafka producer with timeoutMillis = 9223372036854775807 ms. 2020-08-17
04:35:27.791 INFO 1 --- [-StreamThread-1] o.a.k.s.p.internals.StreamThread
: stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1]
partitions lost took 4 ms. 2020-08-17 04:35:27.791 INFO 1 ---
[-StreamThread-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] (Re-)joining group
2020-08-17 04:35:27.795 INFO 1 --- [-StreamThread-1]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Join group failed
with org.apache.kafka.common.errors.MemberIdRequiredException: The group member
needs to have a valid member id before actually entering a consumer group.
2020-08-17 04:35:27.795 INFO 1 --- [-StreamThread-1]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] (Re-)joining group
2020-08-17 04:35:30.881 INFO 1 --- [-StreamThread-1]
a.k.s.p.i.a.HighAvailabilityTaskAssignor : Decided on assignment:
{36fbee26-0c5f-4993-a203-f34c0cac7caf=[activeTasks: ([0_0]) standbyTasks: ([])
prevActiveTasks: ([]) prevStandbyTasks: ([]) prevOwnedPartitionsByConsumerId:
([]) changelogOffsetTotalsByTask: ([]) taskLagTotals: ([0_0=549]) capacity: 1
assigned: 1]} with no followup probing rebalance. 2020-08-17 04:35:30.882 INFO
1 --- [-StreamThread-1] o.a.k.s.p.i.StreamsPartitionAssignor :
stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer]
Assigned tasks to clients as
36fbee26-0c5f-4993-a203-f34c0cac7caf=[activeTasks: ([0_0]) standbyTasks: ([])
prevActiveTasks: ([]) prevStandbyTasks: ([]) prevOwnedPartitionsByConsumerId:
([]) changelogOffsetTotalsByTask: ([]) taskLagTotals: ([0_0=549]) capacity: 1
assigned: 1]. 2020-08-17 04:35:30.882 INFO 1 --- [-StreamThread-1]
o.a.k.s.p.i.StreamsPartitionAssignor : stream-thread
[ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer]
Finished stable assignment of tasks, no followup rebalances required.
2020-08-17 04:35:30.882 INFO 1 --- [-StreamThread-1]
o.a.k.c.c.internals.ConsumerCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Finished assignment
for group at generation 1:
{ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer-f1ddc3a3-7914-4a26-9d76-611acac71039=Assignment(partitions=[ieb.publish.baseline_pc.incurred_pattern-0],
userDataSize=48)} 2020-08-17 04:35:30.895 INFO 1 --- [-StreamThread-1]
o.a.k.c.c.internals.AbstractCoordinator : [Consumer
clientId=ieb-x07-baseline-pc-data-storage-incurred-pattern-36fbee26-0c5f-4993-a203-f34c0cac7caf-StreamThread-1-consumer,
groupId=ieb-x07-baseline-pc-data-storage-incurred-pattern] Successfully joined
group with generation 1 {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
