DanielLeens commented on issue #12814: URL: https://github.com/apache/seatunnel/issues/12814#issuecomment-6217834844
Classification: D / Spark translation wide-DECIMAL compatibility policy. I agree with the proposed direction: use the string carrier with explicit metadata for top-level fields and fields inside a ROW, and use a clear preflight failure for ARRAY and MAP elements in the first change. This preserves the SeaTunnel DecimalType(p, s) across the Spark boundary where StructField metadata exists, without silently narrowing values or changing the shared contract inside the Doris fix. The error for unsupported nested paths must identify the table, full column path, declared precision and Spark's limit of 38, and give a concrete safe workaround. The carrier conversion must use plain decimal text, preserve scale and null handling, and reject malformed carrier values explicitly. Do not mix the existing MySQL source-side DECIMAL clamp into this change; track it separately. Before opening code, please update this design with those exact boundaries and then include Spark 2.4 and 3.3 schema and row round trips for both signed 128-bit extrema, zero, null, and a nonzero scale. Keep the Doris Spark containers enabled, add EN/ZH documentation and the required incompatible-changes entry, and keep #12787 blocked on this policy rather than hiding its failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
