aleksandar-trajkovic-db opened a new pull request, #58809:
URL: https://github.com/apache/spark/pull/58809
### What changes were proposed in this pull request?
Follow-up to SPARK-58876. That change mapped Oracle `DATE`/`TIMESTAMP` to
`TimestampNTZType` directly in `OracleDialect.getCatalystType`, which
short-circuited
the shared `Types.TIMESTAMP` resolution in `JdbcUtils` that widens a
sub-microsecond
column to a nanosecond-capable type.
This PR extracts that micro-vs-nanos decision into
`JdbcUtils.resolveTimestampType(isTimestampNTZ, scale,
preferTimestampNanos)` (the
generic `Types.TIMESTAMP` branch now calls it) and routes Oracle's mapping
through it
with `isTimestampNTZ = true`. Oracle `DATE`/`TIMESTAMP` therefore stays NTZ,
but a
`TIMESTAMP(7-9)` now maps to `TimestampNTZNanosType(scale)` when the nanos
preview is
engaged.
### Why are the changes needed?
Before SPARK-58876, Oracle `DATE`/`TIMESTAMP` returned `None` from the
dialect and fell
through to the shared resolver, so `TIMESTAMP(7-9)` was mapped to a
nanosecond type when
the `preferTimestampNanos` read option and
`spark.sql.timestampNanosTypes.enabled` were
both on. SPARK-58876 accidentally regressed this, truncating such columns to
microsecond
`TimestampNTZType`. This restores the nanosecond precision while keeping the
new
zoneless-NTZ behavior.
### Does this PR introduce _any_ user-facing change?
Yes — relative to the unreleased SPARK-58876 change, not to any released
version. With
`preferTimestampNanos=true` and
`spark.sql.timestampNanosTypes.enabled=true`, an Oracle
`TIMESTAMP(7-9)` column now reads as `TimestampNTZNanos(scale)` instead of
microsecond
`TimestampNTZType`. `DATE`, `TIMESTAMP(0-6)`, and the default (preview off)
are
unchanged. The value is read time-zone-independently (wall-clock).
### How was this patch tested?
Added a `getSchema`-level unit test in `JDBCSuite` asserting that an Oracle
`TIMESTAMP(7-9)` resolves to `TimestampNTZNanosType(scale)` only when both
the
`preferTimestampNanos` option and the preview flag are on, and stays
microsecond
`TimestampNTZType` for coarser scales or with either flag off. This replaces
the prior
test that pinned tehavior.
build/sbt 'sql "SPARK-58876"'
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Anthropic Claude Opus 4.8)
This is my original work and I license it to the project under the project's
open source license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
