[
https://issues.apache.org/jira/browse/KAFKA-17064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17862589#comment-17862589
]
Lianet Magrans commented on KAFKA-17064:
----------------------------------------
Hey [~yangpoan] , definitely, based on events that in the end will execute in
the background thread. I see 2 cases:
1. If the assign has empty partitions I would say we're covered: it relies on a
call to unsubscribe, and that performs all assignment updates in the background
already. We just need KAFKA-17017 to get it all right when the member is not in
a group, sure thing.
2. if the assign has non-empty partitions, it will end up updating the
assignment in the app thread, and we should fix that (this issue). We should
rely on a event indeed and we already have it actually, it's the
AssignmentChangeEvent. The issue is that this event currently does only part of
the job (it only triggers a commit
https://github.com/apache/kafka/blob/c97d4ce026fdb75f80760bcce00b239db951a481/clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java#L181).
Misleading name indeed.
We should consider having that event handle the full assignment change (commit
and update the subscription state with assignFromUser), all in the background.
> New consumer assign should update assignment in background thread
> -----------------------------------------------------------------
>
> Key: KAFKA-17064
> URL: https://issues.apache.org/jira/browse/KAFKA-17064
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.8.0
> Reporter: Lianet Magrans
> Priority: Major
> Labels: consumer-threading-refactor, kip-848-client-support
> Fix For: 3.9.0
>
>
> With the new async consumer, the subscriptionState object is shared between
> the app thread and the background thread, but in principle all updates to the
> assignment should happen in the background thread, to avoid race conditions.
> Note that it's in the background where most updates to the assignment happen,
> as result of app event processing like unsubscribe, reconciliations, etc.).
> We've faced such races in places like unsubscribe and close, fixed by
> ensuring that all assignment updates happen in the background, and this also
> needs to be reviewed for the consumer.assign. The current implementation
> triggers an AssignmentChange event that is processed in the background, but
> that event is not really changing the assignment. It only commits offsets,
> and the assignment is updated in the app thread by calling
> subscriptionState.assignFromUser
> We should consider moving the assignment update to the background thread, as
> part of the AssignmentChangeEvent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
