[
https://issues.apache.org/jira/browse/SPARK-58217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58217:
-----------------------------------
Labels: pull-request-available (was: )
> CAST(DECIMAL AS TIMESTAMP) silently produces garbage timestamps on overflow
> ---------------------------------------------------------------------------
>
> Key: SPARK-58217
> URL: https://issues.apache.org/jira/browse/SPARK-58217
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: jiangxintong
> Priority: Major
> Labels: pull-request-available
>
> When casting a large Decimal to Timestamp, the internal multiplication by
> MICROS_PER_SECOND (1,000,000) can exceed Long range.
> {{BigDecimal.longValue()}} silently truncates the low 64 bits per the Java
> specification, producing nonsensical timestamps.
> This affects both the interpreted and codegen paths.
> h2. Reproduction
> {code:sql}
> -- Non-ANSI mode: should return NULL, but returns garbage
> SET spark.sql.ansi.enabled=false;
> SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
> -- Returns: -265697-05-04 09:44:49.448384
> -- Expected: NULL
> SELECT CAST(CAST(99999999999999999999 AS DECIMAL(38,0)) AS TIMESTAMP);
> -- Returns: -78449-06-15 02:58:03.596224
> -- Expected: NULL
> -- ANSI mode: should throw CAST_OVERFLOW, but also returns garbage
> SET spark.sql.ansi.enabled=true;
> SELECT CAST(CAST(9999999999999 AS DECIMAL(20,0)) AS TIMESTAMP);
> -- Returns: -265697-05-04 09:44:49.448384
> -- Expected: CAST_OVERFLOW error
> {code}
> h2. Root cause
> The cast implementation uses {{BigDecimal.longValue()}} without range
> checking. Per the Java specification, when the value is too big to fit in a
> long, only the low-order 64 bits are returned.
> h2. Consistent with existing patterns
> * The double-to-timestamp path already guards against {{NaN}}/{{Infinite}} by
> returning null
> * {{SecondsToTimestamp}} uses {{DATETIME_OVERFLOW}} for similar overflow
> protection
> * SPARK-45816 fixed the same class of bug for timestamp-to-integer overflow
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
