[
https://issues.apache.org/jira/browse/SPARK-57561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57561:
-----------------------------
Description:
h2. What
Add test coverage (and fix any gaps) so that datetime functions which do not
semantically
support {{TIME}} reject {{TimeType}} arguments with a clear "unsupported
argument type" error,
rather than silently coercing TIME through an unexpected path. This is
especially important
once the TIME <-> TIMESTAMP casts land (SPARK-52617, SPARK-57553, SPARK-57552,
SPARK-57554),
which could let a TIME value be implicitly cast to TIMESTAMP (via CURRENT_DATE)
and then be
accepted by a date/zone/epoch function.
h2. Background
TIME-relevant operations are already supported: {{hour}}/{{minute}}/{{second}},
{{extract}} of HOUR/MINUTE/SECOND, {{time_trunc}}, {{time_diff}}, {{TIME +/-
interval}},
{{TIME - TIME}}, and the {{make_timestamp*}} date+time overloads. The following
datetime
functions are date/zone/epoch-bound and must stay unsupported for TIME:
* Date arithmetic/navigation: {{date_add}}, {{date_sub}}, {{datediff}},
{{add_months}},
{{months_between}}, {{last_day}}, {{next_day}}.
* Calendar extractors: {{year}}, {{quarter}}, {{month}}, {{day}}/{{dayofmonth}},
{{dayofyear}}, {{dayofweek}}, {{weekday}}, {{weekofyear}}, {{monthname}},
{{dayname}}
(and {{extract}}/{{date_part}} of those fields on a TIME source).
* Date parsing/construction: {{to_date}}/{{try_to_date}}, {{make_date}},
{{trunc}} (date units).
* Epoch conversions: {{unix_timestamp}}/{{to_unix_timestamp}},
{{from_unixtime}},
{{timestamp_seconds}}/{{_millis}}/{{_micros}},
{{unix_seconds}}/{{_millis}}/{{_micros}},
{{unix_date}}/{{date_from_unix_date}}.
* Time-zone conversions: {{from_utc_timestamp}}, {{to_utc_timestamp}},
{{convert_timezone}}.
* Event-time windowing: {{window}}, {{session_window}}, {{window_time}}.
h2. Scope
* Audit the type-coercion rules (TypeCoercion / AnsiTypeCoercion,
AnsiGetDateFieldOperationsTypeCoercion, the implicit string/date/timestamp
promotions)
to confirm no rule casts {{TimeType}} into
{{DateType}}/{{TimestampType}}/{{TimestampNTZType}}
for the unsupported functions above.
* Add negative tests asserting a clear, stable error (e.g. DATATYPE_MISMATCH /
unsupported argument type) for TIME inputs to those functions, in both ANSI
and non-ANSI
modes.
* Where a gap is found (silent coercion/acceptance), add an explicit guard so
the function
rejects TIME.
h2. Acceptance criteria
* Negative tests cover the date/zone/epoch functions for TIME inputs and assert
a clear,
stable error class.
* No function silently accepts a TIME argument by coercion after the TIME <->
TIMESTAMP cast
tickets are merged.
> Verify datetime functions reject TIME inputs with a clear error (no silent
> coercion)
> ------------------------------------------------------------------------------------
>
> Key: SPARK-57561
> URL: https://issues.apache.org/jira/browse/SPARK-57561
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Max Gekk
> Priority: Major
>
> h2. What
> Add test coverage (and fix any gaps) so that datetime functions which do not
> semantically
> support {{TIME}} reject {{TimeType}} arguments with a clear "unsupported
> argument type" error,
> rather than silently coercing TIME through an unexpected path. This is
> especially important
> once the TIME <-> TIMESTAMP casts land (SPARK-52617, SPARK-57553,
> SPARK-57552, SPARK-57554),
> which could let a TIME value be implicitly cast to TIMESTAMP (via
> CURRENT_DATE) and then be
> accepted by a date/zone/epoch function.
> h2. Background
> TIME-relevant operations are already supported:
> {{hour}}/{{minute}}/{{second}},
> {{extract}} of HOUR/MINUTE/SECOND, {{time_trunc}}, {{time_diff}}, {{TIME +/-
> interval}},
> {{TIME - TIME}}, and the {{make_timestamp*}} date+time overloads. The
> following datetime
> functions are date/zone/epoch-bound and must stay unsupported for TIME:
> * Date arithmetic/navigation: {{date_add}}, {{date_sub}}, {{datediff}},
> {{add_months}},
> {{months_between}}, {{last_day}}, {{next_day}}.
> * Calendar extractors: {{year}}, {{quarter}}, {{month}},
> {{day}}/{{dayofmonth}},
> {{dayofyear}}, {{dayofweek}}, {{weekday}}, {{weekofyear}}, {{monthname}},
> {{dayname}}
> (and {{extract}}/{{date_part}} of those fields on a TIME source).
> * Date parsing/construction: {{to_date}}/{{try_to_date}}, {{make_date}},
> {{trunc}} (date units).
> * Epoch conversions: {{unix_timestamp}}/{{to_unix_timestamp}},
> {{from_unixtime}},
> {{timestamp_seconds}}/{{_millis}}/{{_micros}},
> {{unix_seconds}}/{{_millis}}/{{_micros}},
> {{unix_date}}/{{date_from_unix_date}}.
> * Time-zone conversions: {{from_utc_timestamp}}, {{to_utc_timestamp}},
> {{convert_timezone}}.
> * Event-time windowing: {{window}}, {{session_window}}, {{window_time}}.
> h2. Scope
> * Audit the type-coercion rules (TypeCoercion / AnsiTypeCoercion,
> AnsiGetDateFieldOperationsTypeCoercion, the implicit string/date/timestamp
> promotions)
> to confirm no rule casts {{TimeType}} into
> {{DateType}}/{{TimestampType}}/{{TimestampNTZType}}
> for the unsupported functions above.
> * Add negative tests asserting a clear, stable error (e.g. DATATYPE_MISMATCH /
> unsupported argument type) for TIME inputs to those functions, in both ANSI
> and non-ANSI
> modes.
> * Where a gap is found (silent coercion/acceptance), add an explicit guard so
> the function
> rejects TIME.
> h2. Acceptance criteria
> * Negative tests cover the date/zone/epoch functions for TIME inputs and
> assert a clear,
> stable error class.
> * No function silently accepts a TIME argument by coercion after the TIME <->
> TIMESTAMP cast
> tickets are merged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
