zhengxiang378928908-code commented on PR #12787: URL: https://github.com/apache/seatunnel/pull/12787#issuecomment-6251552798
Thanks @DanielLeens for the review. The Spark implementation for #12814 is now open as #12862. The `Build` check here stays red on the Spark containers until that lands, since this PR alone still hits Spark's 38-digit limit. Testing both together showed one more Doris issue: the shared Doris test tables have `MAP<SMALLINT, LARGEINT>`, `MAP<LARGEINT, DOUBLE>` and `ARRAY<LARGEINT>` columns, which the agreed design rejects on Spark at job start. So `DorisIT` / `DorisMultiReadIT` would still fail on Spark by design. Pushed 73f52467c to keep the Doris Spark containers running: - On Spark the same jobs run from `spark/*.conf`, which leave out only those three columns with `doris.read.field` (the documented workaround). Top-level `LARGEINT` is still read and written on Spark. - Sink checks now compare columns by name and select explicit column lists, so they also work when the Spark sink table has fewer columns. - New Spark-only `DorisIT.testNestedLargeintFailsOnSpark` checks that each of the three columns fails with `SPARK-TRANSLATION-01` and names the table and column path (`.value`, `.key`, `.element`). I'm running the combined CI on my fork (this PR + #12862) and will post the result here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
