shrirangmhalgi opened a new pull request, #57253: URL: https://github.com/apache/spark/pull/57253
### What changes were proposed in this pull request? Add `TimeType` support to `MsSqlServerDialect` with precision-preserving DDL: - `getCatalystType`: map `Types.TIME` to `TimeType(scale)` when `spark.sql.timeType.enabled` is true, gated by `legacyJdbcTimeMappingEnabled`. The Microsoft JDBC driver correctly reports fractional-second precision via `getScale()`. Includes defensive guard for missing metadata key with fallback to SQL Server's default precision of 7. - `getJDBCType`: map `TimeType(p)` to `TIME(p)` for p in [0,7]; fall back to `TIME(7)` (SQL Server's max 100ns precision) for out-of-range precisions (e.g. nanosecond TimeType(9)), which truncates to 100ns rather than losing all fractional seconds. ### Why are the changes needed? This is a followup to [#56653](https://github.com/apache/spark/pull/56653) which added core JDBC TIME support in `JdbcUtils`. The base `getCommonJDBCType` emits `TIME(${t.precision})` which produces `TIME(9)` for nanosecond TimeTypes - SQL Server rejects precisions > 7. The per-dialect `getCatalystType` runs before the generic mapper, so SQL Server needs its own TIME handling gated by the escape-hatch config. ### Does this PR introduce _any_ user-facing change? Yes. When `spark.sql.timeType.enabled` is true, SQL Server `TIME` and `TIME(n)` columns now correctly read as `TimeType` and write with precision-preserving `TIME(n)` DDL. Non-TIME columns are unaffected. ### How was this patch tested? Added integration tests in `MsSqlServerIntegrationSuite`: - **Scalar read**: verifies existing dates table bare `TIME` column is read as `TimeType(7)` (SQL Server default) with correct value - **Write round-trip**: writes `TimeType(0)` and `TimeType(7)` values and reads them back with full 100ns precision - **Precision preservation**: `TimeType(3)` -> `TIME(3)` -> read back as `TimeType(3)` - **Nanosecond write**: `TimeType(9)` -> `TIME(7)` -> read back as `TimeType(7)` with sub-100ns truncation - **Legacy escape hatch**: asserts `TIME` reads as non-TimeType when `legacyJdbcTimeMappingEnabled=true` Unit tests: `JDBCSuite` (all 127 tests pass). ### Was this patch authored or co-authored using generative AI tooling? CoAuthored using Claude Opus 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
