Adamyuanyuan commented on PR #9994: URL: https://github.com/apache/seatunnel/pull/9994#issuecomment-3479117448
This PR can run successfully in Flink, but the E2E run in Spark fails. The reason is that the Spark engine path loses options during the conversion between SeaTunnelRow and Spark Row, causing the Metadata Transform to fail to retrieve EventTime. As a result, the output event_time_ms is empty, leading to an assertion failure. I need to confirm whether to fix this issue in this PR or start a separate PR specifically to address the problem of metadata loss in the Spark translation layer. 现在这个PR，在Flink中可以成功运行，但是Spark的E2E运行失败，原因是 Spark 引擎路径在 SeaTunnelRow ↔ Spark Row 转换时丢失了 options，导致 Metadata Transform 取不到 EventTime，输出的 event_time_ms 为空，断言失败。 我需要确认是在这个PR中修改这个问题，还是单独启动一个PR，专门解决 Spark 翻译层丢 metadata 的问题。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
