Lianet Magrans created KAFKA-20321:
--------------------------------------
Summary: Mark partitions lost to avoid potential race with
buffered records
Key: KAFKA-20321
URL: https://issues.apache.org/jira/browse/KAFKA-20321
Project: Kafka
Issue Type: Bug
Components: clients, consumer
Reporter: Lianet Magrans
Assignee: Lianet Magrans
Fix For: 4.3.0
When partitions are revoked the consumer marks them as pendingRevocation while
the commit/callbacks run. This ensure that positions do not advance and no
records are returned from the buffer while the async operations run (note that
commit becomes particularly critical, runs async and does not block the app
thread as callbacks do, so consumer.poll could continue running). This path
ensures no more progress is made while the commit/callbacks run.
But in the case of onPartitionsLost we are not applying the same mark. It's
really not critical because we don't commit in that case, and callbacks block
the app thread, so no progress. But still, we should consider marking the
positions for consistency, and to be sure we don't ever get into a race if the
onPartitionsLost callbacks completes, and the app thread gets to return
buffered records before the assignment gets cleared in the backgound after the
callbacks completes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
