andygrove opened a new pull request, #4637: URL: https://github.com/apache/datafusion-comet/pull/4637
## Which issue does this PR close? Part of #4596 (the `sort_array` candidate). ## Rationale for this change `CometSortArray` reports `Incompatible` in exactly one situation: when `spark.comet.exec.strictFloatingPoint=true` and the array element type contains a float or double (strict mode flags float ordering, for example NaN and signed zero, as not bit-identical to Spark). With `allowIncompatible` unset, that case falls the whole projection back to Spark. The issue flagged `sort_array` as needing an eligibility check because it might be `CodegenFallback`. It is not: Spark's `SortArray` has a real `doGenCode`, so the JVM codegen dispatcher accepts it. (And the dispatcher admits `CodegenFallback` expressions anyway.) ## What changes are included in this PR? * `CometSortArray` mixes in `CodegenDispatchFallback`, so its strict-floating-point `Incompatible` case routes through the JVM codegen dispatcher (Spark's own `doGenCode` inside the Comet pipeline) and matches Spark exactly instead of falling back. The `Unsupported` nested-struct/null element-type case is unchanged (still falls back), and default (non-strict) behavior is unchanged. ## How are these changes tested? New `sort_array_strict_fp.sql` runs with `spark.comet.exec.strictFloatingPoint=true` over double and float arrays containing NaN, +/-Infinity, +/-0.0, and nulls, asserting native execution that matches Spark via the dispatcher. The existing comprehensive `sort_array.sql` (default mode, all element types, including the nested-struct `expect_fallback` cases) still passes. Both run with `CometSqlFileTestSuite` on Spark 3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
