andygrove opened a new issue, #5576:
URL: https://github.com/apache/datafusion-comet/issues/5576
Part of #5572.
`round` on a float or double falls the whole projection back to Spark:
```scala
case _: FloatType | DoubleType =>
// ... Spark uses BigDecimal for rounding float/double ...
Unsupported(Some("Comet does not support Spark's BigDecimal rounding"))
```
`spark/src/main/scala/org/apache/comet/serde/arithmetic.scala:376-390`
The reasoning in that comment is sound — Spark rounds floating point through
`BigDecimal`, which goes via `java.lang.Double.toString()` and its rounding
algorithm, and the native path cannot reproduce it. But "we can't do this
natively" is precisely the case the dispatcher exists for: running Spark's own
`doGenCode` inside the Comet pipeline gives the `BigDecimal` behavior byte for
byte.
`bround` already does exactly this — `CometBRound` is a plain
`CometCodegenDispatch[BRound]` (`serde/math.scala:269`) — so the two sibling
functions behave completely differently today for no principled reason.
`round(double_col, n)` is everyday analytics SQL, which makes this probably
the highest-traffic single fallback in the sweep.
`docs/source/user-guide/latest/expressions.md` documents it as "Float/double
inputs fall back".
Fix: mix `CodegenDispatchFallback` into `CometRound`. `FloatType` and
`DoubleType` are both in `CometBatchKernelCodegen.isSupportedDataType`, so
`canHandle` admits it.
Leave the negative-scale decimal arm alone for now — it is only reachable
with `spark.sql.legacy.allowNegativeScaleOfDecimal=true` and is tracked as
deferred on the EPIC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
