nookcreed opened a new pull request, #12110:
URL: https://github.com/apache/gluten/pull/12110
## What changes were proposed in this pull request?
In Spark 4.1,
[SPARK-53968](https://issues.apache.org/jira/browse/SPARK-53968) introduced
`NumericEvalContext` which captures `allowPrecisionLoss` **at analysis time**
and embeds it in arithmetic expressions (`Add`,
`Subtract`, `Multiply`, `Divide`) via an `evalContext` field.
Previously, Gluten read `SQLConf.get.decimalOperationsAllowPrecisionLoss`
at plan-conversion time. These can diverge — for example, when querying a
persistent view whose expression was analyzed under a different session
config. This causes a decimal scale mismatch (e.g. `DECIMAL(38,18) +
DECIMAL(38,18)` producing scale 18 instead of 17 — a 10× error).
**Fix:** Add a shim method `decimalAllowPrecisionLoss(expr:
BinaryArithmetic): Boolean`:
- **`Spark41Shims`**: reads `expr.evalContext.allowDecimalPrecisionLoss`
directly from the expression
- **All other Spark versions**: fall back to `SQLConf.get` (preserving
existing behavior)
`DecimalArithmeticUtil.getResultType` and
`VeloxSparkPlanExecApi.getDecimalArithmeticExprName` now use the shim so the
correct result type and Velox native function name are selected.
## How was this patch tested?
Fixes the decimal precision failures in `GlutenSimpleSQLViewSuite` (1 of 2
failures tracked in #11917) and the excluded test `"should be able to resolve a
persistent view"` in `GlutenSQLQuerySuite`.
Closes #11917
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
