guozhangwang opened a new pull request #11057: URL: https://github.com/apache/kafka/pull/11057
This is an idea I had for attacking this on the consumer client level: 1. When listOffset result is retrieved inside Fetcher, check if the partitions are part of the subscriptions of the consumer; if yes update the corresponding LSO or HW based on the isolation level. 2. When partitionLag cannot return result since the log end offset (LSO/HW) is not known, send an async list offset which would be completed by other calls polling (also the hb thread may complete it as well), and hope the next partitionLag would get the result. Then on the streams side, the first partitionLag would still return empty, but soon enough the subsequent partitionLag should return data and we would not wait for the fetch response to update fetched state. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
