zhangxinyao88 opened a new pull request, #59249:
URL: https://github.com/apache/spark/pull/59249
### What changes were proposed in this pull request?
Route `ColumnarRow.get` and `ColumnarBatchRow.get` through
`SpecializedGettersReader`, preserving each wrapper's null-handling contract.
Route framework-registered values in `ColumnVectorUtils.toBatch` through
`TypeOps.toCatalyst` and append according to their physical representation.
Preserve support for internal `TimestampNanosVal` inputs.
### Why are the changes needed?
The row getters maintain separate data-type lists and reject TIME despite
its supported long representation. The column-population path similarly
duplicates type-specific conversion logic. Using the shared operations lets the
existing framework registrations handle TIME and nanosecond timestamps
consistently.
### Does this PR introduce _any_ user-facing change?
Yes, in vectorized row utilities on the development branch:
- Generic row getters can read TIME. `ColumnarRow.get` also gains calendar
interval and geospatial binary-view support from the shared reader.
- `toBatch` accepts the framework's external nanosecond timestamp values
(`LocalDateTime` for NTZ and `Instant` for LTZ), honoring precision, while
continuing to accept internal nanos values unchanged.
- Unsupported getter types use `_LEGACY_ERROR_TEMP_3131` instead of
`_LEGACY_ERROR_TEMP_3152` or `_LEGACY_ERROR_TEMP_3155`.
### How was this patch tested?
The new generic TIME assertion failed at precisions 0, 6, 7, and 9 before
the production change. After the change, both `ColumnarBatchSuite` and
`ColumnVectorUtilsSuite` passed: 107 tests, zero failures. Added coverage
includes nested TIME reads, nullable struct fields, interval reads, external
timestamp precision, internal timestamp inputs, and null timestamps in both
memory modes. Existing UDT and randomized conversion tests also passed.
Local SQL Core test compilation has unrelated Parquet test-helper errors, so
the run selected only these suites and their `RandomDataGenerator` helper:
```sh
build/sbt 'project sql' \
'set Test / sources := (Test / sources).value.filter(f =>
Set("ColumnarBatchSuite.scala", "ColumnVectorUtilsSuite.scala",
"RandomDataGenerator.scala").contains(f.getName))' \
'testOnly *ColumnarBatchSuite *ColumnVectorUtilsSuite'
```
Catalyst and SQL Core Java Checkstyle passed. SQL Core test Scalastyle
passed with zero errors and warnings. `git diff --check` passed.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
