Max Gekk created SPARK-57817:
--------------------------------
Summary: Support nanosecond-precision timestamps in
from_utc_timestamp and to_utc_timestamp
Key: SPARK-57817
URL: https://issues.apache.org/jira/browse/SPARK-57817
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
The {{UTCTimestamp}} base (datetimeExpressions.scala ~L2205-2278) uses
{{inputTypes = Seq(TimestampType, StringType)}} and {{convertTz(Long, ...)}}.
Nanosecond input is cast to microsecond LTZ, truncating the sub-microsecond
part and imposing LTZ semantics.
h2. Goal
Shift a nanosecond timestamp between UTC and a target zone while preserving
{{nanosWithinMicro}} and the input type/precision.
h2. Scope
Accept {{AnyTimestampNanoType}}; add a {{TimestampNanosVal}} eval/codegen path
that applies the zone offset on the epoch micros and carries the nanosecond
remainder unchanged.
h2. Acceptance criteria
* {{from_utc_timestamp}} / {{to_utc_timestamp}} over nanosecond timestamps
preserve the nanosecond remainder and the input type/precision; existing
microsecond behavior unchanged.
h2. Testing
{{DateExpressionsSuite}}; nanos golden files.
h2. Dependencies
None - independent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
