[
https://issues.apache.org/jira/browse/KAFKA-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Cadonna updated KAFKA-16285:
----------------------------------
Description:
Currently, the new async Kafka consumer sends an event from the background
thread to the application thread when the group metadata is updated. Group
metadata is updated when the background thread receives a new assignment. More
specifically, the member epoch is updated each time a new assignment is
received and and the member ID is updated with the first assignment.
In contrast to the group metadata update, the assignment is directly set in the
subscription without sending an update event from the background thread to the
application thread. That means that there is a delay between the application
thread being aware of the update to the assignment and the application thread
being aware of the update to the group metadata. This behavior differs with
respect to the legacy consumer were the assignment and the group metadata is
updated at the same time.
We should make the update to the group metadata available to the application
thread when the update to the assignment is made available to the application
thread so that assignment an group metadata are in sync.
For example, {{producer.sendOffsetsToTransaction(offsetsToCommit,
groupMetadata);}} benefits from this improvement because if the offsets to
commit are consistent with the current assignment also the group metadata would
be. Currently, that is not guaranteed.
was:
Currently, the new async Kafka consumer sends an event from the background
thread to the application thread when the group metadata is updated. Group
metadata is updated when the background thread receives a new assignment. More
specifically, the member epoch is updated each time a new assignment is
received and and the member ID is updated with the first assignment.
In contrast to the group metadata update, the assignment is directly set in the
subscription without sending an update event from the background thread to the
application thread. That means that there is a delay between the application
thread being aware of the update to the assignment and the application thread
being aware of the update to the group metadata. This behavior differs with
respect to the legacy consumer were the assignment and the group metadata is
updated at the same time.
We should make the update to the group metadata available to the application
thread when the update to the assignment is made available to the application
thread so that assignment an group metadata are in sync.
> Make group metadata available when a new assignment is set in async Kafka
> consumer
> ----------------------------------------------------------------------------------
>
> Key: KAFKA-16285
> URL: https://issues.apache.org/jira/browse/KAFKA-16285
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer
> Reporter: Bruno Cadonna
> Priority: Major
> Labels: kip-848-client-support
>
> Currently, the new async Kafka consumer sends an event from the background
> thread to the application thread when the group metadata is updated. Group
> metadata is updated when the background thread receives a new assignment.
> More specifically, the member epoch is updated each time a new assignment is
> received and and the member ID is updated with the first assignment.
> In contrast to the group metadata update, the assignment is directly set in
> the subscription without sending an update event from the background thread
> to the application thread. That means that there is a delay between the
> application thread being aware of the update to the assignment and the
> application thread being aware of the update to the group metadata. This
> behavior differs with respect to the legacy consumer were the assignment and
> the group metadata is updated at the same time.
> We should make the update to the group metadata available to the application
> thread when the update to the assignment is made available to the application
> thread so that assignment an group metadata are in sync.
> For example, {{producer.sendOffsetsToTransaction(offsetsToCommit,
> groupMetadata);}} benefits from this improvement because if the offsets to
> commit are consistent with the current assignment also the group metadata
> would be. Currently, that is not guaranteed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
