tlfua commented on PR #7457:
URL: 
https://github.com/apache/datafusion-comet/pull/7457#issuecomment-6961187380

   Thanks @andygrove, all five points are addressed in 
ade66196f102207257bd53fa3bae3e6134cf207e and 
d4f087ea5f49567e9dbaa32f2eee4dad46c2295a .
   
   1. **Timestamp casts with ANSI off.** Removed the cast arm from 
`raisesInAnyMode`, so with ANSI off a timestamp-cast later key now runs in 
Comet. Under ANSI, `CometCast.cannotFail` still rejects these casts. 
@coderfender, this reverses the arm you added deliberately: Spark does not 
raise for these casts with ANSI off, and the chrono range failure is a Comet 
kernel bug that hits a projection the same way, so it belongs to 
https://github.com/apache/datafusion-comet/issues/7471 rather than to the TopK 
check.
   2. **Views.** With ANSI off, a later key that holds a node still in ANSI 
eval mode (arithmetic and casts by their `EvalMode`, plus 
`failOnError`/`ansiEnabled` on `UnaryMinus`, `Abs`, `Round`, `BRound`, 
`ElementAt`, `GetArrayItem` and `Elt`) and that `cannotFail` rejects now falls 
back. The test creates `SELECT a, b FROM topk ORDER BY a, b + 1 LIMIT 2` as a 
temporary view with ANSI on and queries it with ANSI off. Spark 4.1 keeps the 
arithmetic eval mode inside a `NumericEvalContext`, which the first version 
missed; fork CI caught it on the Spark 4.1 profile, and the follow-up commit 
reads `evalMode` by class instead. A view that only projects `b + 1 AS c` and 
is then sorted by `c` is not affected, because the projection evaluates `b + 1` 
for every row in Spark too.
   3. **`slice`, `array_insert`, `sequence`.** Added to `raisesInAnyMode`: 
`slice` unless the start is a nonzero constant and the length a non-negative 
constant, `array_insert` unless the position is a nonzero constant, and 
`sequence` unless all of its arguments are constants (invalid constant bounds 
already fail in constant folding).
   4. **Tests.**
      - Timestamp casts: since they no longer fall back with ANSI off, 
`date_add(ts, 1)` and `concat(s, ts)` are plain `query` tests that check they 
stay in Comet.
      - UDF: `raise_error` cannot reach this check in a SQL file test, because 
Comet has no serde for `RaiseError` and the TopK falls back for that first. So 
`CometCodegenSuite` has a ScalaUDF later-key test with ANSI off instead.
      - `element_at(arr, 1)`, `slice(arr, 1, 1)` and `array_insert(arr, 1, 0)` 
stay in Comet; column-driven `slice`, `array_insert` and `sequence` fall back.
   5. **Docs.** `operators.md` now says "runs in Comet", lists the new ANSI-off 
cases and the view case, says `allowIncompatible` applies in either mode, and 
links https://github.com/apache/datafusion-comet/issues/7471 for timestamp 
casts. I also replaced the same "native" wording in the code comments.
   
   Fork CI passed on d4f087ea5: 
https://github.com/tlfua/datafusion-comet/actions/runs/44155798081 . One Spark 
SQL job was rerun after a Maven Central download failure. Could you approve the 
workflows again when you have a moment?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to