shrirangmhalgi opened a new pull request, #57198:
URL: https://github.com/apache/spark/pull/57198
### What changes were proposed in this pull request?
Add `TimeType` support to `MySQLDialect` and precision-preserving DDL for
writes:
- `getCatalystType`: map `Types.TIME` to `TimeType(scale)` when
`spark.sql.timeType.enabled` is true, gated by `legacyJdbcTimeMappingEnabled`
- `getJDBCType`: add `TimeType => TIME(p)` for p in [0,6], bare TIME for
out-of-range precisions (e.g. nanosecond TimeType)
### Why are the changes needed?
This is a followup to #56653 which added core JDBC TIME support in
`JdbcUtils`. That PR handled scalar read/write correctly via the generic path,
but:
- The write-side `getJDBCType` in the base JdbcUtils emits
`TIME(${t.precision})` which produces TIME(9) for nanosecond TimeTypes - MySQL
rejects precisions > 6
- The per-dialect `getCatalystType` runs before the generic `JdbcUtils`
mapper (confirmed in #56884), so MySQL needs its own TIME handling gated by the
escape-hatch config
### Does this PR introduce _any_ user-facing change?
Yes. When `spark.sql.timeType.enabled` is true, MySQL `TIME` and `TIME(p)`
columns now correctly read as `TimeType` and write with precision-preserving
`TIME(p)` DDL. Non-TIME columns are unaffected.
### How was this patch tested?
Added integration tests in `MySQLIntegrationSuite`:
- **Scalar read**: verifies existing dates table `TIME` column is read as
`TimeType(0)` and `TIME(3)` as `TimeType(3)` with correct values
- **Scalar write round-trip**: writes `TimeType(0)` and `TimeType(6)` values
and reads them back
- **Precision preservation**: `TimeType(3)` -> `TIME(3)` -> read back as
`TimeType(3)`
- **Nanosecond write**: `TimeType(9)` -> bare `TIME` -> read back as
`TimeType(6)` with microsecond truncation
- **Legacy mode**: asserts `TIME` reads as non-TimeType when
`legacyJdbcTimeMappingEnabled=true`
Unit tests: JDBCSuite (127 tests, all passing).
### Was this patch authored or co-authored using generative AI tooling?
CoAuthored using Claude Opus 4.6
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
