jiangxintong created SPARK-58217:
------------------------------------
Summary: CAST(DECIMAL AS TIMESTAMP) silently produces garbage
timestamps on overflow
Key: SPARK-58217
URL: https://issues.apache.org/jira/browse/SPARK-58217
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.3.0
Reporter: jiangxintong
When casting a large Decimal to Timestamp, the internal multiplication by
MICROS_PER_SECOND (1,000,000) can exceed Long range. {{BigDecimal.longValue()}}
silently truncates the low 64 bits per the Java specification, producing
nonsensical timestamps.
This affects both the interpreted and codegen paths.
h2. Reproduction
{code:sql}
-- Non-ANSI mode: should return NULL, but returns garbage
SET spark.sql.ansi.enabled=false;
SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
-- Returns: -265697-05-04 09:44:49.448384
-- Expected: NULL
SELECT CAST(CAST(99999999999999999999 AS DECIMAL(38,0)) AS TIMESTAMP);
-- Returns: -78449-06-15 02:58:03.596224
-- Expected: NULL
-- ANSI mode: should throw CAST_OVERFLOW, but also returns garbage
SET spark.sql.ansi.enabled=true;
SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
-- Returns: -265697-05-04 09:44:49.448384
-- Expected: CAST_OVERFLOW error
{code}
h2. Root cause
The cast implementation uses {{BigDecimal.longValue()}} without range checking.
Per the Java specification, when the value is too big to fit in a long, only
the low-order 64 bits are returned.
h2. Consistent with existing patterns
* The double-to-timestamp path already guards against {{NaN}}/{{Infinite}} by
returning null
* {{SecondsToTimestamp}} uses {{DATETIME_OVERFLOW}} for similar overflow
protection
* SPARK-45816 fixed the same class of bug for timestamp-to-integer overflow
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
