andygrove opened a new pull request, #5033: URL: https://github.com/apache/datafusion-comet/pull/5033
## Which issue does this PR close? There is no dedicated tracking issue. The native path's incompatibility stems from the upstream DataFusion behavior in apache/datafusion#22598. Closes #. ## Rationale for this change Spark's `Pow` expression delegates to Java's `Math.pow`, which returns `Infinity` for a zero base raised to a negative exponent (`pow(0, -1)`). Comet's native path routed `pow` to DataFusion's `PowerFunc`, whose `float64_power_checked` instead **errors** on that input (apache/datafusion#22598). Because of that single edge case, `CometPow` was marked `Unsupported`, so any query using `pow`/`power` fell back to Spark for that part of the plan. Rust's `f64::powf` follows the same IEEE-754 pow semantics as Java's `Math.pow` (including `0^-1 = Infinity`, `(-0)^-1 = -Infinity`, and the `NaN`/`Infinity` cases), so a thin native kernel over `powf` matches Spark exactly and lets `pow` run natively by default instead of falling back. ## What changes are included in this PR? - Add a `spark_pow` native kernel (`native/spark-expr/src/math_funcs/pow.rs`) that computes `base.powf(exp)` element-wise with null propagation, covering all Array/Scalar argument combinations. - Register `"pow"` to `spark_pow` in `create_comet_physical_fun_with_eval_mode`, so it no longer falls through to DataFusion's checked `power`, plus the module/re-export wiring. - Simplify `CometPow` to a plain `Compatible` serde (drop the `Unsupported`/"Power has correctness issues" reporting). ## How are these changes tested? - Rust unit tests in `pow.rs` covering the basic cases, `pow(0, -1) == Infinity`, null propagation, and the scalar-base path. - The `expressions/math/pow.sql` SQL file test (run via `CometSqlFileTestSuite`) now asserts native execution matches Spark across zero-base-negative-exponent, `NaN`, `Infinity`, null, and column/literal argument combinations, replacing the previous `expect_fallback` assertions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
