SEZ9 commented on issue #11433: URL: https://github.com/apache/seatunnel/issues/11433#issuecomment-5149901380
Thanks for the detailed report. This looks like a valid gap: the Doris 2PC commit endpoint returns once the transaction is COMMITTED, while data readability requires the VISIBLE state after the publish phase. A reasonable fix is for DorisCommitter to poll the transaction status (e.g., via `SHOW TRANSACTION WHERE ID = ...` or the FE transaction status API) until it reaches VISIBLE, with a configurable timeout. To keep backward compatibility and avoid blocking users who don't need read-after-commit guarantees, this could be gated behind a new sink option (e.g., `sink.wait-for-visible` / `sink.visible-timeout-ms`) that defaults to the current behavior or a sensible timeout. Could you share your Doris version? Publish/visibility behavior and available status-check APIs differ across Doris versions, which matters for implementing the fix. Also, contributions are welcome if you'd like to submit a PR. <!-- streview-comment:25 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
