David Mollitor created SPARK-59802:
--------------------------------------
Summary: Multiply compact decimals without allocating BigDecimal
Key: SPARK-59802
URL: https://issues.apache.org/jira/browse/SPARK-59802
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: David Mollitor
h3. What
{{Decimal}} keeps small values in a compact {{long}} ({{decimalVal == null}})
and only expands to a {{java.math.BigDecimal}} beyond 18 digits, but
{{Decimal.*}} always went through {{toJavaBigDecimal.multiply(...,
MATH_CONTEXT)}} with no compact path -- a row-level {{a * b}} on two compact
operands allocated two operand {{BigDecimal}}s, a result {{BigDecimal}}, and
its {{BigInteger}}.
Add a compact {{long}} fast path for {{*}}, mirroring the existing same-scale
{{+}}/{{-}}: when both operands are compact and the exact product fits the
18-digit compact range (overflow-checked via {{Math.multiplyHigh}}), multiply
the unscaled longs directly; otherwise fall back to the existing BigDecimal
path.
h3. Why
Avoids per-row {{BigDecimal}}/{{BigInteger}} allocation for the common
small-decimal case (e.g. multiplying {{decimal(7,2)}} prices/amounts). On a
full TPC-DS v1.4 (SF10) JFR run, {{BigDecimal}} allocations attributed to
{{Decimal.*}} dropped ~93%. Correctness-neutral (byte-identical): the fast path
runs only where no rounding occurs, and high-precision operands (including the
SPARK-45786 case) keep the BigDecimal path; {{CheckOverflow}} re-normalizes
precision/scale exactly as before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
