Max Gekk created SPARK-57825:
--------------------------------
Summary: Support ANSI year-month interval +/- for
TIMESTAMP_NTZ(p)/TIMESTAMP_LTZ(p)
Key: SPARK-57825
URL: https://issues.apache.org/jira/browse/SPARK-57825
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{TimestampAddYMInterval}} (datetimeExpressions.scala ~L2378-2416) types its
timestamp arg as {{AnyTimestampType}} (microsecond only) and evaluates
{{timestampAddMonths(Long, Int, zone)}};
{{BinaryArithmeticWithDatetimeResolver}}
(analysis/BinaryArithmeticWithDatetimeResolver.scala ~L74-77 for {{Add}},
~L113-115 for {{Subtract}}) routes only {{TimestampType | TimestampNTZType}}
into it, so {{ts_nanos +/- INTERVAL '1' YEAR}} falls through to an unresolved
{{Add}} / {{Subtract}} and fails analysis. Verified NOT covered by SPARK-57501
(day-time interval, commit dea9c30), whose message explicitly scopes out
year-month and {{SubtractTimestamps}}.
h2. Goal
Feature parity for {{TIMESTAMP_NTZ(p)}} / {{TIMESTAMP_LTZ(p)}} (p in [7,9])
{{+/-}} ANSI year-month interval, preserving {{nanosWithinMicro}} and NTZ/LTZ
zone semantics (add months on epoch micros, carry the nanosecond remainder
unchanged).
h2. Scope
Extend {{TimestampAddYMInterval.inputTypes}} and its eval/codegen to accept
{{AnyTimestampNanoType}} with a {{TimestampNanosVal}} path; add the nanosecond
cases to the year-month-interval branches in
{{BinaryArithmeticWithDatetimeResolver}} (Add and Subtract); regenerate
{{interval.sql}} / nanos golden files.
h2. Non-goals
No change to {{SubtractTimestamps}} (timestamp - timestamp).
h2. Acceptance criteria
* {{ts_nanos +/- INTERVAL 'n' YEAR/MONTH}} resolves and returns the same
nanosecond type/precision with the remainder preserved; interpreted + codegen
tested.
h2. Testing
{{DateExpressionsSuite}}; {{timestamp-ntz-nanos.sql}},
{{timestamp-ltz-nanos.sql}}, {{interval.sql}} golden files.
h2. Dependencies
None hard (the day-time-interval sibling is resolved in SPARK-57501). BLOCKS
the window RANGE + interval-bounds sub-task; reused by the
window/session_window sub-task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
