Max Gekk created SPARK-60743:
--------------------------------
Summary: make_timestamp silently wraps an out-of-range seconds
value
Key: SPARK-60743
URL: https://issues.apache.org/jira/browse/SPARK-60743
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Max Gekk
make_timestamp and make_timestamp_ntz take their seconds argument as
DECIMAL(16, 6), but DateTimeExpressionUtils.makeTimestampMicros narrows its
unscaled micros to an int:
// 8 digits cannot overflow Int.
int totalMicros = (int) secAndMicros.toUnscaledLong();
The comment was true while the input was DECIMAL(8, 6). Since SPARK-36428
widened it to DECIMAL(16, 6), the cast drops the high bits, and an out-of-range
seconds value can wrap into a valid one. For example:
SELECT make_timestamp_ntz(2020, 1, 1, 0, 0, 4324.967296);
-- 2020-01-01 00:00:30
4324967296 micros wraps to 30000000, exactly 30 seconds, so the query returns a
timestamp instead of null (ANSI off) or DATETIME_FIELD_OUT_OF_BOUNDS (ANSI on),
which is what an out-of-range value such as 61 seconds gives. The same holds
for make_timestamp, for both the interpreted and the generated code, and in
both modes. Values that wrap to a negative number already fail, but their ANSI
error reports the wrapped seconds.
Affects 3.2.0 and later.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
