David Mollitor created SPARK-59643:
--------------------------------------
Summary: Tighten CheckOverflow nullability to match MakeDecimal
Key: SPARK-59643
URL: https://issues.apache.org/jira/browse/SPARK-59643
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Environment: h2. Summary
{{CheckOverflow}} declares {{override def nullable: Boolean = true}}
unconditionally. Its sibling {{MakeDecimal}} (immediately above it in the same
file, {{decimalExpressions.scala}}) already declares the accurate form
{{override def nullable: Boolean = child.nullable || nullOnOverflow}} and
guards its generated {{isNull}} assignment accordingly. {{CheckOverflow}} was
simply never updated to match, so it over-declares nullability.
h2. Why it matters
{{CheckOverflow}} rounds a decimal to a target precision/scale; on overflow it
returns {{null}} when {{nullOnOverflow}} is true and *throws* when it is false
(ANSI). So the result is {{null}} only when the input is {{null}}, or when
{{nullOnOverflow}} is true and an overflow occurs -- i.e. {{child.nullable ||
nullOnOverflow}}. Declaring it unconditionally nullable is inaccurate: under
ANSI ({{nullOnOverflow = false}}), a decimal expression wrapped in
{{CheckOverflow}} over non-nullable inputs is reported as nullable when it can
never actually be null.
h2. Changes
In {{sql/catalyst/.../expressions/decimalExpressions.scala}}, aligning
{{CheckOverflow}} with {{MakeDecimal}}
Reporter: David Mollitor
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
