peterxcli opened a new pull request, #5176: URL: https://github.com/apache/datafusion-comet/pull/5176
## Which issue does this PR close? Closes #5101. ## Rationale for this change The nested fallback paths in `arrays_overlap` and `array_position` duplicate comparison behavior already provided by Arrow. `arrays_overlap` recursively compares nested elements and dispatches an equality kernel for each leaf pair, while `array_position` materializes a `ScalarValue` for every candidate element. Arrow's reusable `make_comparator` supports the same nested structural comparison without this per-element dispatch and allocation overhead. ## What changes are included in this PR? - Build and reuse an Arrow comparator for nested `arrays_overlap` probe/search pairs, while retaining the existing flat fast paths and outer null handling. - Build one comparator for the flattened values and search arrays in `array_position` instead of allocating `ScalarValue`s in the inner loop. - Remove the hand-written list/struct structural equality helpers. - Add focused coverage for nested NaN, signed-zero, and inner-null comparisons. The tests preserve the existing Arrow/`ScalarValue` total-order behavior. ## How are these changes tested? - `cargo test -p datafusion-comet-spark-expr --lib` (568 passed) - `cargo clippy -p datafusion-comet-spark-expr --lib --tests -- -D warnings` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
