srielau opened a new pull request, #58132: URL: https://github.com/apache/spark/pull/58132
### What changes were proposed in this pull request? **Depends on #58130**, which depends on #58087 / #58080 / #58033. Please review and merge those first; rebase this onto `master` afterward. Delta vs follow-up: https://github.com/srielau/spark/compare/serge-rielau_data/SPARK-58794-r1-implicit...serge-rielau_data/SPARK-58794-clients Expose first-class `CHAR(n)` and `VARCHAR(n)` types and lengths through SQL client metadata when `spark.sql.charVarchar.standardSemantics.enabled` is on: - Spark Connect JDBC maps `CharType` / `VarcharType` to `java.sql.Types.CHAR` / `VARCHAR`, with Java string values and precision/display size `n`. - HiveServer2 `DatabaseMetaData.getColumns` reports `COLUMN_SIZE = n` for `VARCHAR(n)`, matching its existing `CHAR(n)` behavior. - HiveServer2 result metadata coverage verifies `Types.CHAR` / `Types.VARCHAR` and precision `n` for first-class CAST results. The existing Connect proto already carries CHAR/VARCHAR and their lengths; the Connect JDBC test exercises that schema round trip as well as value collection. ### Why are the changes needed? Spark Connect JDBC currently rejects first-class `CharType` and `VarcharType` as unsupported because its metadata mapping only recognizes the `StringType` singleton. HiveServer2 already identifies VARCHAR catalog columns correctly but reports their `COLUMN_SIZE` as unknown (`0`). Clients therefore cannot reliably describe or size CHAR/VARCHAR columns even though Catalyst retains the type and length. ### Does this PR introduce _any_ user-facing change? Yes, when first-class CHAR/VARCHAR types are present: JDBC and HiveServer2 metadata now report the corresponding JDBC type and declared character length instead of rejecting the type or reporting an unknown size. ### How was this patch tested? - Added `SparkConnectJdbcDataTypeSuite` coverage for CHAR/VARCHAR values, JDBC types, class names, signedness, precision, scale, and display size. - Added `SparkMetadataOperationSuite` result metadata coverage and updated `getColumns` expectations for `VARCHAR(n)`. - Both changed test suites compile. Local execution was blocked by unrelated build-environment failures: the HiveServer2 child process hit a stale `ConfigBuilder.withBindingPolicy` linkage error, and the Connect build could not resolve `com.google.errorprone.annotations.CheckReturnValue` while compiling `connect-common`. ### Was this patch authored or co-authored using generative AI tooling? Yes (Cursor). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
