ChoMinGi commented on PR #23029: URL: https://github.com/apache/kafka/pull/23029#issuecomment-5169874191
Thanks for working on this. I tried the probe sequence with `EmbeddedKafkaCluster` and a real `read_committed` consumer using `pollTime = 100 ms`. I saw two cases worth considering. First, the retry still returned empty: ```text Committed transaction: 5 records + COMMIT marker, LSO=6 seek(5) -> 0 records, 2 ms seek(4) -> 0 records, 101 ms <- retry ... record returned on poll #6, 508 ms in Control: fresh consumer, seek(4) -> 1 record in 2 ms ``` Second, the `endOffset - 2` retry did not reach the latest visible record when the log ended with an aborted transaction: ```text Committed transaction followed by an aborted transaction, LSO=12 seek(11) -> 0 seek(10) -> 0 <- retry seek(9..5) -> 0 seek(4) -> 1 ``` Here, the latest visible record was eight offsets back because the aborted batch was filtered as a whole. Would it be better to start the probe from an earlier offset, clamped to `logStartOffset`, instead of retrying at `endOffset - 2`? I can share the reproduction test if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
