unknowntpo commented on code in PR #21584:
URL: https://github.com/apache/kafka/pull/21584#discussion_r2875328265
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java:
##########
@@ -289,23 +289,32 @@ private void process(final FetchCommittedOffsetsEvent
event) {
}
/**
- * Commit all consumed if auto-commit is enabled. Note this will trigger
an async commit,
- * that will not be retried if the commit request fails.
+ * Commit all consumed if auto-commit is enabled, then update the
assignment.
+ * The event future is completed only after the commit completes, ensuring
+ * assign() blocks until offsets are durably committed.
*/
private void process(final AssignmentChangeEvent event) {
+ CompletableFuture<Void> commitFuture = null;
if (requestManagers.commitRequestManager.isPresent()) {
CommitRequestManager manager =
requestManagers.commitRequestManager.get();
- manager.updateTimerAndMaybeCommit(event.currentTimeMs());
+ commitFuture =
manager.maybeAutoCommitSyncBeforeRebalance(event.deadlineMs());
Review Comment:
Thanks for the review. I agree invoking `maybeAutoCommitSyncBeforeRebalance`
for `assign()` is not suitable — that method is semantically tied to rebalance.
But I have a different view on what "best-effort" should mean here. This
PR intentionally introduced retry-with-deadline semantics in the classic
consumer's `assign()` via `maybeAutoCommitOffsetsSync`. The reason: if we do a
single commit and it
silently fails, the user has no way to know which partition wasn't
committed and no way to recover. Retrying within the deadline gives the user a
real best shot before partitions switch.
My proposal: refactor by extracting a new method (e.g.,
`maybeAutoCommitSyncBeforeAssign`) with the same retry logic as
`maybeAutoCommitOffsetsSync` in `ConsumerCoordinator.java`, so that classic
consumer and async consumer share the same behavior for `assign()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
