zheguang opened a new pull request, #20756: URL: https://github.com/apache/kafka/pull/20756
This patch clarifies `SinkTask.precommit`, which may see revoked partitions during rebalancing and stopping. As pointed out in [KAFKA-19685](https://issues.apache.org/jira/browse/KAFKA-19685), there is a difference in whether `SinkTask.precommit` sees revoked partitions during iteration or stopping. This difference appears to be intentional for correctness - During [iteration](https://github.com/apache/kafka/blob/2c444485123a5790170856284f613384b4b033c2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L237), partition ownership may change due to rebalanced, so revoked partitions should be [cleaned up](https://github.com/apache/kafka/blob/2c444485123a5790170856284f613384b4b033c2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L795) at each poll, to avoid committing offsets for partitions no longer owned - During [stopping](https://github.com/apache/kafka/blob/2c444485123a5790170856284f613384b4b033c2/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L223), all current offsets are committed regardless of partition ownership to record the final state. The task is ending its lifecycle and no new records will be processed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
