lianetm commented on PR #21495: URL: https://github.com/apache/kafka/pull/21495#issuecomment-3953300177
The `isValidatePositionsComplete` gates when there is no callbacks (the fix we have in the PR now). When there is callbacks, the gate is the callback itself that was already in place (blocks the app thread until it completes). But totally agree that we still have another gap on that callback path, as you describe (callback future completed in the background, which triggers 2 things that could race: assignment update and control back to the app thread that could exit poll and seek). I think the issue there is in how we process the `ConsumerRebalanceListenerCallbackNeededEvent` in the app thread (runs the callback, blocking, then tells the background to continue reconciling/updating assignment, but does not block here). It's conceptually wrong to assume that the app thread is done with the callbacks if it hasn't updated the assignment really, and that could lead to what you said, exit poll too soon + seek failure. I'll be away for a few days but will extend the fix to cover this bit too when I'm back, I think we need to ensure the update happens before releasing the poll from the callbacks too. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
