DanielLeens commented on PR #11493: URL: https://github.com/apache/seatunnel/pull/11493#issuecomment-5391451168
Thanks for the careful tracking, @SEZ9 -- agreed the `8ddb8ff420f0` -> `79103480df52` commit was test-only (resetting the mounted `json-type` paths in `LocalFileIT.java`) and doesn't move any of F1-F4. I won't call the test-only lane settled until the refreshed CI comes back on `79103480df52` too. On the four open findings, here is the plan for the next commit: - **F1 (HIGH):** add an opt-out option (default `true`, preserving the new mapping) so upgrading users can pin Snowflake `VARIANT` / Doris `JSON`/`JSONB` / StarRocks `JSON` columns back to legacy `STRING`. I'll add the `incompatible-changes.md` note either way, since the default behavior changes for existing pipelines regardless of the opt-out. - **F2 (MEDIUM):** document in `schema-feature.md` that `JSON` validation is deliberately deferred to sink-side boundaries (e.g. the StarRocks load path) rather than enforced generically, since `JSON` is backed by plain `String` end-to-end and there's no cheap place to validate structural correctness across every sink without a real cost on the hot path. - **F3 (MEDIUM):** add an init-time fail-fast check in the StarRocks sink so a native `JSON`/`JSONB` column combined with the default CSV Stream Load format is rejected immediately instead of failing at load time. - **F4 (MEDIUM):** for the Snowflake user-defined-query path, fail fast at startup when a JSON-typed field reaches a user query, rather than silently skipping the `PARSE_JSON` wrap -- silently changing what gets written is worse than an explicit config error. I'll push these together with the `incompatible-changes.md`, `schema-feature.md`, `StarRocks.md`, and `Snowflake.md` doc updates in one commit so it's reviewable as a set, and will ping you here once it's up and CI is green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
