ump45nose opened a new pull request, #26905:
URL: https://github.com/apache/datafusion/pull/26905
## Which issue does this PR close?
- Closes #26336
## Rationale for this change
In Spark, `date_part('SECOND', ts)` and `EXTRACT(SECOND FROM ts)` return
`DECIMAL(8, 6)` that keeps the microsecond fraction (for example `45.123456`).
DataFusion's Spark `date_part` always returned `Int32` and rewrote seconds to
the core integer `date_part`, so the fraction was dropped. The standalone
`second(ts)` function correctly stays `INT` and is unchanged.
## What changes are included in this PR?
- Plan-time rewrite of Spark `date_part` / `datepart` / `EXTRACT` for the
SECOND aliases (`SECOND`, `S`, `SEC`, `SECONDS`, `SECS`) onto a dedicated
seconds UDF that returns `Decimal128(8, 6)` as `second + fraction`.
- The rewrite happens in `SparkFunctionPlanner` so the return type is fixed
before analysis (a user-written `'SECOND'::string` cast hides the literal from
`return_field_from_args`).
- Sqllogictest expectations updated for whole and fractional seconds,
`arrow_typeof`, `DESCRIBE`, and `EXTRACT(SECOND …)`.
## What is the testing strategy for this PR?
- `cargo test -p datafusion-sqllogictest --test sqllogictests --
spark/datetime/date_part`
- `cargo test -p datafusion-sqllogictest --test sqllogictests --
spark/datetime/second` (standalone `second` still Int32)
- `cargo test -p datafusion-spark --lib`
## Are there any user-facing changes?
Yes for the Spark dialect: `date_part('SECOND', …)` / `EXTRACT(SECOND FROM
…)` now return `DECIMAL(8, 6)` with fractional seconds, matching Spark.
Standalone `second(ts)` is unchanged.
## AI assistance
Used an AI coding assistant for the implementation and tests; I reviewed the
change and wrote this description myself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
