stevomitric opened a new pull request, #56735:
URL: https://github.com/apache/spark/pull/56735
### What changes were proposed in this pull request?
Migrate Parquet read/write for
`TimestampLTZNanosType`/`TimestampNTZNanosType` from inline `dt match` arms in
the `*Default` methods to the Phase-3a `ParquetTypeOps` framework, mirroring
how `TimeType` was integrated.
- Add `TimestampNanosParquetOps` (shared trait + LTZ/NTZ impls) covering
schema conversion, value write (INT64 epoch-nanos packing with overflow check),
and the row-based read converter.
- Register both nanos types in `ParquetTypeOps.apply`.
- Remove the now-dead nanos branches and private helpers from
`ParquetSchemaConverter`,
`ParquetWriteSupport`, `ParquetRowConverter`, and `ParquetUtils`
(framework dispatch runs first).
The Parquet→Spark read-schema inference stays inline since it keys on the
Parquet annotation, not a Spark type (same as `TimeType`).
### Why are the changes needed?
Two parallel Parquet code paths existed (the `ParquetTypeOps` registry for
`TimeType` and inline matches for nanos). This consolidates nanos onto the
single registration point, following the TimeType dead-branch cleanup in
SPARK-55444.
### Does this PR introduce _any_ user-facing change?
No. Behavior is unchanged; this is an internal refactor.
### How was this patch tested?
New tests in this PR.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
