[
https://issues.apache.org/jira/browse/SPARK-58235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58235:
-----------------------------------
Labels: pull-request-available (was: )
> [SQL] Fix Double/Float to Timestamp silent overflow clamping in non-ANSI mode
> -----------------------------------------------------------------------------
>
> Key: SPARK-58235
> URL: https://issues.apache.org/jira/browse/SPARK-58235
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: jiangxintong
> Priority: Major
> Labels: pull-request-available
>
> h2. Background
> In non-ANSI mode ({{spark.sql.ansi.enabled=false}}), casting a Double or
> Float value that overflows the Long range to Timestamp silently clamps to
> {{Long.MAX_VALUE}} or {{Long.MIN_VALUE}} instead of returning {{NULL}}.
> The interpreted path ({{Cast.doubleToTimestamp}}) directly computes {{(d *
> MICROS_PER_SECOND).toLong}} without overflow detection. The codegen path
> similarly casts without checking: {{(long)($c * $MICROS_PER_SECOND)}}. When
> the double multiplication exceeds Long range, the unchecked cast to long
> silently produces a clamped value instead of NULL.
> This is inconsistent with other overflow behavior in non-ANSI mode — for
> example, casting Timestamp to Byte/Short/Int returns {{NULL}} on overflow,
> and ANSI mode correctly throws {{CAST_OVERFLOW}} via
> {{DoubleExactNumeric.toLong}}.
> h2. Repro
> {code:sql}
> SET spark.sql.ansi.enabled = false;
> SELECT CAST(1e20 AS TIMESTAMP);
> -- Before fix: +294247-01-10 04:00:54.775807 (silently clamped to
> Long.MAX_VALUE)
> -- After fix: NULL
> SELECT CAST(-1e20 AS TIMESTAMP);
> -- Before fix: -290308-12-21 19:59:05.224192 (silently clamped to
> Long.MIN_VALUE)
> -- After fix: NULL
> {code}
> Note: NaN and Infinity are already handled by the existing guard ({{d.isNaN
> || d.isInfinite}} in the interpreted path, JDK methods in codegen) and are
> unaffected by this fix.
> h2. Fix
> Add overflow detection before converting to Long in both paths:
> * *Interpreted*: check if the multiplication result exceeds Long range before
> calling {{.toLong}}, then return NULL in non-ANSI mode (via {{errorOrNull}})
> or throw {{CAST_OVERFLOW}} in ANSI mode
> * *Codegen*: same range check for Double and Float branches in
> {{castToTimestampCode}}, then set {{$evNull = true}} in non-ANSI mode
> The overflow conditions use the same pattern as other Spark overflow guards:
> check the double result against Long range boundaries, then respect the ANSI
> mode setting.
> h2. Testing
> * {{CastWithAnsiOffSuite}} (138 tests): non-ANSI overflow
> (positive/negative), boundary values (9223372036854.0 safe / 9223372036855.0
> overflow), NaN, Infinity, normal values, zero
> * {{CastWithAnsiOnSuite}} (136 tests): ANSI overflow (CAST_OVERFLOW),
> boundary values, normal values
> * Both Double and Float paths covered
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
