Brijesh-Thakkar opened a new pull request, #22605: URL: https://github.com/apache/datafusion/pull/22605
## Which issue does this PR close? Closes #22598 ## Rationale for this change In Apache Spark, the `pow(base, exp)` function follows IEEE 754 semantics where raising `0` (or `-0.0`) to a negative exponent yields positive `Infinity`. Currently, DataFusion's default core `PowerFunc` mimics PostgreSQL behavior, throwing an explicit error (`"zero raised to a negative power is undefined"`). To support standard Spark compatibility without breaking core DataFusion expectations, this PR introduces a specialized `SparkPow` UDF inside the `datafusion-spark` crate. ## What changes are included in this PR? This PR introduces the following changes within the `datafusion-spark` integration crate: * **Added `SparkPow` UDF** (`datafusion/spark/src/function/math/pow.rs`): Overrides the `Float64` execution path to evaluate `base == 0.0 && exp < 0.0` as `f64::INFINITY` (safely catching both `0.0` and `-0.0` due to IEEE 754 equality rules). * **Decimal Delegation**: Preserves correctness by delegating non-float types (like decimals) back to the standard `PowerFunc`, as decimals cannot represent infinity. * **Function Registration** (`datafusion/spark/src/function/math/mod.rs`): Registers the new `pow` function and establishes `power` as a valid alias. * **SQL Integration Tests** (`datafusion/sqllogictest/test_files/spark/math/pow.slt`): Updates and adds test coverage ensuring `pow(0, -1)`, `power(0, -1)`, and `pow(0.0, -1.0)` successfully return `Infinity`. ## Are these changes tested? Yes, the changes are covered via both unit and integration tests: 1. **Unit Tests**: Added `test_spark_pow_zero_negative_returns_infinity` and `test_spark_pow_normal_cases` within `pow.rs` to validate the core scalar execution logic. 2. **Integration Tests**: Extended `datafusion/sqllogictest/test_files/spark/math/pow.slt` to verify the end-to-end SQL evaluation behavior. ## Are there any user-facing changes? Yes, but only for users utilizing the `datafusion-spark` compatibility features. When the Spark dialect/crate is active, evaluating `pow(0, <negative>)` will now return `Infinity` instead of throwing an evaluation error. Core DataFusion behavior remains completely unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
