chengwei977 commented on issue #11433: URL: https://github.com/apache/seatunnel/issues/11433#issuecomment-4976202612
> 感谢您清晰地阐述了 COMMITTED 和 VISIBLE 之间的区别。我追踪了 Doris 当前的 2PC 补全路径，这看起来确实是一个实际存在的问题，而不是使用错误。 > > 目前，SeaTunnel 会停止加载，获取事务 ID，然后一旦提交请求成功或 Doris 报告事务已提交/可见，Doris 提交器就会将 2PC 提交视为成功。没有额外的等待或轮询步骤来保持 SeaTunnel 作业处于打开状态，直到数据在 Doris 中实际可见。 > > 因此，SeaTunnel 暴露的完成边界目前早于下游 SQL 作业所依赖的可见性边界。这与您在使用调度器链式作业时观察到的行为一致。 > > 这个问题值得继续探讨。可能的解决方案是区分已提交的交易和可见的数据，并且只有在达到可见阶段后才将接收器报告为已完成，或者如果 Doris 端的行为强制执行不同的合约，则明确定义该边界。 > > 如果您碰巧有针对 COMMITTED -> VISIBLE 转换的具体 Doris FE 响应示例，那将有助于确定最终的轮询条件，但该问题本身已经可以作为错误报告提交。 There happens to be a case study, this is the configuration of a seatunnel job, where 2pc is enabled in the sink doris stage. <img width="736" height="734" alt="Image" src="https://github.com/user-attachments/assets/38a8725c-1ade-4831-9806-5a4d5acf214b"; /> The response process for the COMMITTED -> VISIBLE transformation can be found in Doris FE using sink.label-prefix. <img width="1902" height="343" alt="Image" src="https://github.com/user-attachments/assets/8b8d7a78-9adb-4171-873e-337a2fc752df"; /> Regarding your point about "helping to determine the final polling conditions," I think we can refer to the Doris FE API: GET /api/example_db/get_load_state?label=my_label { "msg": "success", "code": 0, "data": "VISIBLE", "count": 0 } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
