dwsmith1983 opened a new issue, #7671:
URL: https://github.com/apache/datafusion-comet/issues/7671

   ### Describe the bug
   
   With ANSI off, `/`, `div` and `%` return wrong results when the divisor is a 
stateful nondeterministic expression, and so do `ln`, `log10` and `log2` when 
their argument is one. It is the same problem as #5781: Comet evaluates the 
operand twice, once in the guard that turns a zero divisor or a non-positive 
argument into NULL and once as the value, and the two copies see different rows.
   
   - `/` and `div`: `nullIfWhenPrimitive` in 
`spark/src/main/scala/org/apache/comet/serde/arithmetic.scala` wraps the 
divisor as `If(EqualTo(x, 0), null, x)`.
   - `%`: `null_if_zero_primitive` in 
`native/spark-expr/src/math_funcs/modulo_expr.rs` builds the same `IfExpr` 
natively.
   - `ln`, `log10`, `log2`: `nullIfNegative` in 
`spark/src/main/scala/org/apache/comet/serde/math.scala` wraps the argument as 
`If(LessThanOrEqual(x, 0), null, x)`. SQL `log(base, x)` goes through 
`CometLogarithm` and is not affected.
   
   None of these checks whether the operand is deterministic, and all of them 
run natively.
   
   ### Steps to reproduce
   
   ```sql
   SET spark.sql.ansi.enabled=false;
   CREATE TABLE t (id INT, v INT) USING parquet;
   -- one file, so monotonically_increasing_id() is the row index 0..15
   INSERT INTO t SELECT CAST(id AS INT), CAST(id + 1 AS INT) FROM range(0, 16, 
1, 1);
   
   SELECT id, v / IF(monotonically_increasing_id() % 2 = 0, 2, 0) FROM t;
   SELECT id, v div IF(monotonically_increasing_id() % 2 = 0, 2, 0) FROM t;
   SELECT id, v % IF(monotonically_increasing_id() % 2 = 0, 3, 0) FROM t;
   SELECT id, isnan(ln(IF(monotonically_increasing_id() % 2 = 0, CAST(v AS 
DOUBLE), CAST(-1 AS DOUBLE)))) FROM t;
   ```
   
   On main (ba16e1a53), Spark 4.1:
   
   - `/`: Comet returns `Infinity` for ids 2, 6, 10 and 14, where Spark returns 
1.5, 3.5, 5.5 and 7.5.
   - `div`: Comet returns 0 for the same ids, where Spark returns 1, 3, 5 and 7.
   - `%`: Comet fails with `CometNativeException: Divide by zero error`, where 
Spark returns NULL on the odd rows.
   - `ln`, `log10`, `log2`: Comet returns NaN for ids 2, 6, 10 and 14, where 
Spark returns a number.
   
   ### Expected behavior
   
   The same results as Spark. As on `branch-1.2` in #7670, these could fall 
back to Spark for a nondeterministic operand, or go through the codegen 
dispatcher as #5867 does for `size`, `arrays_zip` and `map_from_arrays`.
   
   ### Additional context
   
   Found while preparing #7670, the `branch-1.2` change for #5781.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to