[
https://issues.apache.org/jira/browse/SPARK-57825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57825:
-----------------------------
Shepherd: Max Gekk
> Support ANSI year-month interval +/- for TIMESTAMP_NTZ(p)/TIMESTAMP_LTZ(p)
> --------------------------------------------------------------------------
>
> Key: SPARK-57825
> URL: https://issues.apache.org/jira/browse/SPARK-57825
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
> precision).
> h2. Problem
> {{TimestampAddYMInterval}} (datetimeExpressions.scala ~L2378-2416) types its
> timestamp arg as {{AnyTimestampType}} (microsecond only) and evaluates
> {{timestampAddMonths(Long, Int, zone)}};
> {{BinaryArithmeticWithDatetimeResolver}}
> (analysis/BinaryArithmeticWithDatetimeResolver.scala ~L74-77 for {{Add}},
> ~L113-115 for {{Subtract}}) routes only {{TimestampType | TimestampNTZType}}
> into it, so {{ts_nanos +/- INTERVAL '1' YEAR}} falls through to an unresolved
> {{Add}} / {{Subtract}} and fails analysis. Verified NOT covered by
> SPARK-57501 (day-time interval, commit dea9c30), whose message explicitly
> scopes out year-month and {{SubtractTimestamps}}.
> h2. Goal
> Feature parity for {{TIMESTAMP_NTZ(p)}} / {{TIMESTAMP_LTZ(p)}} (p in [7,9])
> {{+/-}} ANSI year-month interval, preserving {{nanosWithinMicro}} and NTZ/LTZ
> zone semantics (add months on epoch micros, carry the nanosecond remainder
> unchanged).
> h2. Scope
> Extend {{TimestampAddYMInterval.inputTypes}} and its eval/codegen to accept
> {{AnyTimestampNanoType}} with a {{TimestampNanosVal}} path; add the
> nanosecond cases to the year-month-interval branches in
> {{BinaryArithmeticWithDatetimeResolver}} (Add and Subtract); regenerate
> {{interval.sql}} / nanos golden files.
> h2. Non-goals
> No change to {{SubtractTimestamps}} (timestamp - timestamp).
> h2. Acceptance criteria
> * {{ts_nanos +/- INTERVAL 'n' YEAR/MONTH}} resolves and returns the same
> nanosecond type/precision with the remainder preserved; interpreted + codegen
> tested.
> h2. Testing
> {{DateExpressionsSuite}}; {{timestamp-ntz-nanos.sql}},
> {{timestamp-ltz-nanos.sql}}, {{interval.sql}} golden files.
> h2. Dependencies
> None hard (the day-time-interval sibling is resolved in SPARK-57501). BLOCKS
> the window RANGE + interval-bounds sub-task; reused by the
> window/session_window sub-task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
