srielau opened a new pull request, #58087: URL: https://github.com/apache/spark/pull/58087
### What changes were proposed in this pull request? **Depends on #58080** (collated LCT / compare / parameterized lengths), which depends on #58033 (standardSemantics foundation). Please review/merge those first; rebase this onto `master` afterward. Delta vs follow-up: https://github.com/srielau/spark/compare/SPARK-58794-lct-compare...SPARK-58794-surfaces Language-surface hardening under `spark.sql.charVarchar.standardSemantics.enabled`: **Bug fixes (format write)** - **ORC:** `CharType`/`VarcharType` extend `StringType`, so write stamped `spark.sql.catalyst.type=string` and file-only inference lost the constraint. Match CHAR/VARCHAR first and stamp `char(n)` / `varchar(n)`. - **Avro:** same trap plus ser/de matched only the `StringType` singleton. Stamp the catalyst property on write, restore it on STRING read, and accept any `StringType` subclass in serializer/deserializer. **Coverage (already worked via gated replace / metastore; now pinned)** - CTAS / CREATE VIEW inherit CHAR/VARCHAR - ALTER COLUMN equal-length CHAR/VARCHAR - Session variables (`DECLARE` / `SET`) and SQL FUNCTION `RETURNS CHAR` - JSON / CSV with a user-specified CHAR/VARCHAR schema - ORC catalog + file-only round-trip; Avro schema conversion unit round-trip Clients (JDBC / HS2 metadata, Connect mapping) remain out of scope. ### Why are the changes needed? Without ORC/Avro catalyst-type stamping, Spark-written files cannot re-infer CHAR/VARCHAR under the flag. Language surfaces (CTAS/VIEW/vars/functions/user schemas) need explicit coverage so schema fidelity does not regress. ### Does this PR introduce _any_ user-facing change? Yes, when the flag is on: ORC (and Avro schema conversion) preserve CHAR/VARCHAR logical types across write/read instead of collapsing to STRING. CTAS/VIEW/vars/functions behavior matches the foundation rules and is now tested. ### How was this patch tested? - `BasicCharVarcharTestSuite` / `language surfaces` - Regenerated `charvarchar-standard-semantics.sql` goldens -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
