Sean Quah created KAFKA-20845:
---------------------------------
Summary: Consumer group downgrades can leave group in invalid
state when classic group metadata is very large
Key: KAFKA-20845
URL: https://issues.apache.org/jira/browse/KAFKA-20845
Project: Kafka
Issue Type: Bug
Components: group-coordinator
Affects Versions: 4.3.0, 4.2.0
Reporter: Sean Quah
Assignee: Sean Quah
Fix For: 4.4.0
When downgrading a consumer group, we apply changes to the group coordinator
state directly and do not replay the produced records. However, when appending
a large but compressible batch, we can allocate an empty batch, then flush,
then allocate another empty batch. Flushing an empty batch silently fails the
empty batch and rolls back coordinator state. We then write the records for the
downgrade, which leaves the in-memory state inconsistent with the state on disk.
After further writes, the state on disk becomes unloadable because the next
consumer group records cannot be applied to the classic group.
# We should not flush an empty batch when appending large batches.
# We could consider succeeding empty batches instead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
