sollhui opened a new pull request, #67089: URL: https://github.com/apache/doris/pull/67089
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Routine load used `query_watermark_offsets()` when resolving `OFFSET_END` and refreshing latest offsets. The high watermark is the log end offset and includes records from open Kafka transactions, so `isolation.level=read_committed` did not affect routine load progress, lag, or scheduling decisions. This change uses Kafka’s `ListOffsets` Admin API and passes the job isolation level explicitly. `read_committed` now returns the last stable offset, while `read_uncommitted` continues to return the log end offset. Both periodic latest-offset refresh and `OFFSET_END` resolution share the same implementation. A transactional Kafka regression case covers the different initial offsets and lag values for `read_committed` and `read_uncommitted` jobs. ### Release note Kafka routine load offset queries now respect `property.isolation.level`. ### Check List (For Author) - Test: Not run (skipped at requester direction) - Behavior changed: Yes. `read_committed` offset and lag exclude records in open transactions. - Does this need documentation: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
