Hello, I understand that a consumer with Txn Isolation level set as "read_committed" will read till Last Stable Offset (LSO) and will not be given any messages for aborted Txn. LSO also includes non-transactional messages.
We have seen a behaviour where LSO didn't have any data and we had to seek (rewind by decrementing the offsets) to be able to poll data (or, poll just hangs forever). The reason for such seek() is that our consumer only needs to fetch the last successfully committed record to Kafka TopicPartition. Once we can retrieve it, we don't need to keep running the consumer (i.e. we close it successfully). This almost seems like a missing configuration or a bug, if the consumer isn't going to poll forever. For example, if a TopicPartition has last successfully committed records at offset 10, I should be able to call seek(TopicPartition, 10) and poll for data without hanging forever. But currently, with read_committed isolation level, we are getting offset as a result of endOffsets(), but the last committed record is available at offset 9. Is this an expected behaviour? If so, does it mean our consumer use case is unique? Regards,
