Max Gekk created SPARK-57660:
--------------------------------
Summary: Support casting between TIME(p) and TIMESTAMP_LTZ(q)
Key: SPARK-57660
URL: https://issues.apache.org/jira/browse/SPARK-57660
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
Assignee: Max Gekk
Add bidirectional casts between the TIME(p) data type (p in [0, 9]) and
TIMESTAMP_LTZ(q)
(q in [6, 9], where q=6 is the microsecond TimestampType and q in [7, 9] is the
nanosecond TimestampLTZNanosType).
This is the TIMESTAMP_LTZ counterpart of SPARK-57618 (TIME <-> TIMESTAMP_NTZ),
and a
sub-task of SPARK-56822 (timestamps with nanosecond precision).
Semantics follow the SQL standard (section 6.13 <cast specification>):
* CAST(TIMESTAMP_LTZ(q) AS TIME(p)) (rule 15.d): the LTZ value is an absolute
instant, so
its time-of-day is the local wall-clock time observed in the session time
zone, truncated
to precision p. Unlike TIMESTAMP_NTZ -> TIME, this direction depends on the
session
time zone.
* CAST(TIME(p) AS TIMESTAMP_LTZ(q)) (rule 17.c): the date fields come from
CURRENT_DATE and
the time fields from the value; the resulting local date-time is interpreted
in the
session time zone to produce the instant. Since CURRENT_DATE is stable within
a query, the
cast is stabilized via the existing ComputeCurrentTime optimizer rule, so it
shares the
same date literal as current_date().
Both directions of TIME <-> TIMESTAMP_LTZ therefore depend on the session time
zone (whereas
for TIMESTAMP_NTZ only TIME -> TIMESTAMP_NTZ does).
Fractional precision handling is pure truncation (floor toward the precision
step; no
rounding). Both directions always succeed, so no new nullability or error
condition is
introduced. TIME and TIMESTAMP_LTZ have no common type, so the conversion is
supported only
as an explicit CAST; implicit coercion (UNION / coalesce / CASE WHEN) is still
rejected.
Examples:
-- extract the time-of-day in the session time zone
SELECT CAST(TIMESTAMP'2020-05-17 12:34:56.789012' AS TIME(6));
-- 12:34:56.789012
-- attach the current date, interpreted in the session time zone
SELECT CAST(TIME'12:34:56.789012345' AS TIMESTAMP_LTZ(9));
-- <current_date> 12:34:56.789012345
Out of scope: Structured Streaming batch-timestamp parity for TIME ->
TIMESTAMP_LTZ (the
cast uses the optimizer-instant current date rather than the micro-batch
timestamp).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
