yadavay-amzn opened a new pull request, #56942: URL: https://github.com/apache/spark/pull/56942
### What changes were proposed in this pull request? Adds vectorized Parquet reader support for nanosecond-precision timestamp columns (INT64 `TIMESTAMP(NANOS)`). A new `TimestampNanosUpdater` in `ParquetVectorUpdaterFactory` decomposes epoch-nanoseconds into the two-child column-vector representation (`epochMicros: Long`, `nanosWithinMicro: Short`), matching `DateTimeUtils.epochNanosToTimestampNanos` and the existing row-based `ParquetRowConverter` exactly (including the dictionary-decoding path). Previously these columns forced the row-based reader (`isBatchReadSupported=false`, otherwise `SchemaColumnConvertNotSupportedException`). ### Why are the changes needed? Part of nanosecond-precision timestamp support (SPARK-56822). The vectorized reader is the default fast path; nanosecond-precision timestamp columns should be readable through it - producing identical results to the row-based reader - rather than falling back to row-by-row reads. ### Does this PR introduce _any_ user-facing change? Yes - nanosecond-precision timestamp columns in Parquet are now read via the vectorized reader (a performance improvement); results are identical to the row-based reader. ### How was this patch tested? `ParquetTimestampNanosSuite` was upgraded to run under both readers (`withAllParquetReaders`), plus a new vectorized-vs-row-based parity test over edge-case nanosecond values (positive, pre-epoch, exact-second, nulls) at precisions 7/8/9 for both NTZ and LTZ, and a rebase-mode-invariance test. All 11 tests pass; `ParquetIOSuite` nanosecond tests pass; scalastyle + checkstyle clean. ### Was this patch authored or co-authored using generative AI tooling? Authored with assistance by Claude Opus 4.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
