SEZ9 commented on PR #11493: URL: https://github.com/apache/seatunnel/pull/11493#issuecomment-6594303162
Thanks for the follow-up in `ff53cb15bf5` and for syncing with `dev` @ `3244c9504c7`. **F1 (silent STRING → JSON change / opt-out).** Keeping `MAX_STRING_LENGTH` on the Doris `JSON`/`JSONB` and StarRocks `JSON` mappings sounds like the right direction so that `handle_json_as_string` restores exactly the previous column, and moving the entry into the `## dev` section of `incompatible-changes.md` (en/zh) makes sense since it is unreleased. I still need to verify this against the diff rather than the description. Two asks: (1) your comment appears to be cut off at the end — could you confirm which StarRocks test got its `getColumnLength()` assertions back? (2) Could you confirm that the Snowflake `VARIANT` mapping is covered by the same `handle_json_as_string` opt-out, and that the `## dev` entry documents the opt-out for all three connectors? **F3 (StarRocks fail-fast for native JSON + CSV).** Re-pointing `testNativeJsonRejectsCsvStreamLoad` at the sink-creation diagnostic is the shape I was hoping for, but the IT assertion alone does not show where the check lives. Could you point me to the validation code in the diff that runs at sink initialization (and the test that exercises it directly, if there is one) so I can confirm the rejection happens before any load is attempted? **F4 (Snowflake user-defined query / PARSE_JSON).** Generating `INSERT INTO t (id, payload) SELECT :id, PARSE_JSON(:payload)` only when the schema has a JSON field, with a no-JSON test pinning the unchanged `VALUES` form and identical bind order, sounds like it addresses the core concern; I will confirm against the diff. Leaving `UPDATE ... SET payload = PARSE_JSON(:payload)` unchanged is fine. I'm okay treating the "custom query is only checked for `PARSE_JSON` somewhere in the query" limitation as non-blocking as long as the docs state it plainly — could you point me at the exact wording in the Snowflake sink docs so I can confirm it reads as a limitation rather than a guarantee? **F2 (no validation of JSON text before native sinks).** I don't see anything in this update that speaks to F2. Could you either describe where non-JSON text is caught before it reaches Doris/Snowflake native JSON columns, or state explicitly that the sink-side database error is the intended behavior and document that? Either is acceptable, but it should be deliberate and written down. Once the F1 confirmations, the F3 pointer, the F4 doc wording, and an F2 answer are in, I'm happy to do a final pass. <!-- streview-comment:2244 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
