[
https://issues.apache.org/jira/browse/SPARK-60746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-60746:
-----------------------------
Description:
Under ANSI mode, dividing an ANSI interval by a number can overflow the
interval type. DivideYMInterval and DivideDTInterval report the overflow with
different errors depending on the evaluation mode and on the type of the
divisor:
- With an integral divisor, only the minimum interval divided by -1 overflows.
The interpreted path (IntervalDivide.checkDivideOverflow) raises
INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION, SQLSTATE 22015, "Use `try_divide`
to tolerate overflow and return NULL instead." The generated code (doGenCode)
raises ARITHMETIC_OVERFLOW, SQLSTATE 22003, "Overflow in integral divide."
SPARK-50226 changed the interpreted helper to the interval error class, and the
generated code still called QueryExecutionErrors.overflowInIntegralDivideError.
- With a decimal or a floating-point divisor, the quotient is rounded into the
interval type, and when it does not fit, both paths let the
java.lang.ArithmeticException of intValueExact/longValueExact or
DoubleMath.roundToInt/roundToLong escape, with no error condition and no
SQLSTATE.
Reproduction on 4.2.0 (also present on master), with
spark.sql.ansi.enabled=true:
{code:sql}
CREATE TABLE t USING parquet AS SELECT INTERVAL '-106751991 04:00:54.775808'
DAY TO SECOND AS i;
SELECT i / -1 FROM t;
-- whole-stage codegen on: ARITHMETIC_OVERFLOW (22003)
-- spark.sql.codegen.wholeStage=false, factoryMode=NO_CODEGEN:
INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION (22015)
SELECT (INTERVAL '-178956970-8' YEAR TO MONTH) / -1.0;
-- java.lang.ArithmeticException: Overflow
SELECT (INTERVAL '-178956970-8' YEAR TO MONTH) / -1.0D;
-- java.lang.ArithmeticException: rounded value is out of range for input
2.147483648E9 and rounding mode HALF_UP
SELECT INTERVAL 2147483647 MONTH / 0.5;
-- java.lang.ArithmeticException: Overflow
{code}
The same holds for a year-month interval at Int.MinValue months divided by -1.
An overflowing interval division should raise
INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION in every evaluation mode and for
every type of divisor.
was:
Under ANSI mode, dividing the minimum ANSI interval by -1 overflows. The
interpreted and code-generated paths of DivideDTInterval and DivideYMInterval
report the overflow with different error classes and SQLSTATEs:
- interpreted (IntervalDivide.checkDivideOverflow):
INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION, SQLSTATE 22015, "Use `try_divide`
to tolerate overflow and return NULL instead."
- generated code (doGenCode): ARITHMETIC_OVERFLOW, SQLSTATE 22003, "Overflow in
integral divide."
Reproduction on 4.2.0 (also present on master), with
spark.sql.ansi.enabled=true:
CREATE TABLE t USING parquet AS SELECT INTERVAL '-106751991 04:00:54.775808'
DAY TO SECOND AS i;
SELECT i / -1 FROM t; -- whole-stage codegen on: ARITHMETIC_OVERFLOW (22003)
-- spark.sql.codegen.wholeStage=false,
factoryMode=NO_CODEGEN:
-- INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION
(22015)
The same holds for a year-month interval at Int.MinValue months. SPARK-50226
changed the interpreted helper to the interval error class; the generated code
still calls QueryExecutionErrors.overflowInIntegralDivideError. The generated
code should raise the same error as the interpreted path.
Summary: Interval division overflow raises different errors depending
on the evaluation mode and the divisor type (was: Interval division by -1
overflow raises different errors in interpreted and generated code)
> Interval division overflow raises different errors depending on the
> evaluation mode and the divisor type
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60746
> URL: https://issues.apache.org/jira/browse/SPARK-60746
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.1.0, 4.0.0, 4.2.0, 4.3.0, 5.0.0
> Reporter: Max Gekk
> Priority: Major
> Labels: pull-request-available
>
> Under ANSI mode, dividing an ANSI interval by a number can overflow the
> interval type. DivideYMInterval and DivideDTInterval report the overflow with
> different errors depending on the evaluation mode and on the type of the
> divisor:
> - With an integral divisor, only the minimum interval divided by -1
> overflows. The interpreted path (IntervalDivide.checkDivideOverflow) raises
> INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION, SQLSTATE 22015, "Use
> `try_divide` to tolerate overflow and return NULL instead." The generated
> code (doGenCode) raises ARITHMETIC_OVERFLOW, SQLSTATE 22003, "Overflow in
> integral divide." SPARK-50226 changed the interpreted helper to the interval
> error class, and the generated code still called
> QueryExecutionErrors.overflowInIntegralDivideError.
> - With a decimal or a floating-point divisor, the quotient is rounded into
> the interval type, and when it does not fit, both paths let the
> java.lang.ArithmeticException of intValueExact/longValueExact or
> DoubleMath.roundToInt/roundToLong escape, with no error condition and no
> SQLSTATE.
> Reproduction on 4.2.0 (also present on master), with
> spark.sql.ansi.enabled=true:
> {code:sql}
> CREATE TABLE t USING parquet AS SELECT INTERVAL '-106751991 04:00:54.775808'
> DAY TO SECOND AS i;
> SELECT i / -1 FROM t;
> -- whole-stage codegen on: ARITHMETIC_OVERFLOW (22003)
> -- spark.sql.codegen.wholeStage=false, factoryMode=NO_CODEGEN:
> INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION (22015)
> SELECT (INTERVAL '-178956970-8' YEAR TO MONTH) / -1.0;
> -- java.lang.ArithmeticException: Overflow
> SELECT (INTERVAL '-178956970-8' YEAR TO MONTH) / -1.0D;
> -- java.lang.ArithmeticException: rounded value is out of range for input
> 2.147483648E9 and rounding mode HALF_UP
> SELECT INTERVAL 2147483647 MONTH / 0.5;
> -- java.lang.ArithmeticException: Overflow
> {code}
> The same holds for a year-month interval at Int.MinValue months divided by
> -1. An overflowing interval division should raise
> INTERVAL_ARITHMETIC_OVERFLOW.WITH_SUGGESTION in every evaluation mode and for
> every type of divisor.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
