jonasdedden opened a new pull request, #51611: URL: https://github.com/apache/arrow/pull/51611
### Rationale for this change There is no direct way to check whether each list in a list array contains a value. The workaround (flatten, compare, group by parent index) is slow and gets empty and null lists wrong. ### What changes are included in this PR? A binary scalar function `list_contains(lists, value)` returning a boolean per list, for list, large list, list view, large list view and fixed-size list inputs. `value` must be a scalar. The child values are compared once with `equal` (so implicit casts apply, e.g. an int8 list and an int64 value), then each list's range in the resulting bitmap is checked with a popcount. List and fixed-size list inputs only compare the child range they reference. Semantics: - Null lists give null. - Null list values don't match a non-null value. - A null value matches lists that hold a null, as in Polars. - A NaN value matches NaN list values, as in `is_in` and Polars (`equal` says NaN != NaN). ### Are these changes tested? Yes, in `scalar_nested_test.cc` and `test_compute.py`. Cases include sliced, chunked and out-of-order list-view inputs, null and NaN values, implicit casts, and several value types. ### Are there any user-facing changes? Yes, a new compute function, `pyarrow.compute.list_contains` in Python. ### Was AI used for this PR? **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [X] Human - [ ] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
