[
https://issues.apache.org/jira/browse/KAFKA-15305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850834#comment-17850834
]
Chia-Ping Tsai commented on KAFKA-15305:
----------------------------------------
{quote}
So seems we also need to make sure that the HB manager is actually polled when
closing the consumer (the HBManager.pollOnClose you had suggested at some point)
{quote}
yep, I prefer to return HB of leaving group by `pollOnClose` and that is the
description I written in KAFKA-16639. Also, I agree the solution of KAFKA-16639
PR as in the current flow the HB of leaving group can be generated.
thread of closing consumer
1. prepareShutdown
(https://github.com/apache/kafka/blob/32b2b73f673ecd41d17c03e99db3746c517990c4/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1268)
2. waiting for `LeaveOnCloseEvent`
(https://github.com/apache/kafka/blob/32b2b73f673ecd41d17c03e99db3746c517990c4/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1279)
3. call ConsumerNetworkThread#close to stop the loop of
ConsumerNetworkThread#run()
(https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java#L281)
network thread (ConsumerNetworkThread)
1. processApplicationEvents
(https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java#L138).
This method must be executed to handle `LeaveOnCloseEvent`
2. After `processApplicationEvents` (LeaveOnCloseEvent),
`membershipManager.leaveGroup` is executed so ConsumerNetworkThread is aware of
"leaving"
3. After `processApplicationEvents`, ConsumerNetworkThread will call HB manager
poll sequentially
(https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java#L144)
In short, ConsumerNetworkThread always call the HB manager poll after handling
LeaveOnCloseEvent.
{quote}
With that, at this point we would actually generate the HB to leave, add then
we would hit the logic you mentioned above with a request to send (not empty),
here. Makes sense?
{quote}
Both ways (poll/pollOnClose) can resolve KAFKA-16639. I feel your comment
(https://github.com/apache/kafka/pull/16017#discussion_r1612039771) is a simple
solution, and so I did not add comment to say "we must to create HB request in
pollOnClose)
> The background thread should try to process the remaining task until the
> shutdown timer is expired
> --------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15305
> URL: https://issues.apache.org/jira/browse/KAFKA-15305
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Philip Nee
> Assignee: Chia-Ping Tsai
> Priority: Major
> Labels: consumer-threading-refactor, timeout
> Fix For: 3.8.0
>
>
> While working on https://issues.apache.org/jira/browse/KAFKA-15304
> close() API supplies a timeout parameter so that the consumer can have a
> grace period to process things before shutting down. The background thread
> currently doesn't do that, when close() is initiated, it will immediately
> close all of its dependencies.
>
> This might not be desirable because there could be remaining tasks to be
> processed before closing. Maybe the correct things to do is to first stop
> accepting API request, second, let the runOnce() continue to run before the
> shutdown timer expires, then we can force closing all of its dependencies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
