Hi Olo,
Typically you will evaluate some predicate in order to obtain a
BooleanArray, potentially using one of the comparison
<https://docs.rs/arrow/latest/arrow/compute/kernels/comparison/index.html>
kernels, and then pass this to the filter kernel to filter the actual
values. This separation not only allows both kernels to be implemented
more efficiently, but allows filtering an array with a predicate
evaluated on a different array.
I've created this PR <https://github.com/apache/arrow-rs/pull/3014> to
add an example doctest of this. Please let me know if anything isn't clear
Kind Regards,
Raphael Taylor-Davies
On 04/11/2022 06:48, Olo Sawyerr wrote:
Hi there,
Hope you're well.
I have a question about the *arrow
<https://docs.rs/arrow/latest/arrow/index.html>::compute
<https://docs.rs/arrow/latest/arrow/compute/index.html>::kernels
<https://docs.rs/arrow/latest/arrow/compute/kernels/index.html>::filter
<https://docs.rs/arrow/latest/arrow/compute/kernels/filter/fn.filter.html#>::filter
<https://docs.rs/arrow/latest/arrow/compute/kernels/filter/index.html>()***function*.* The
function takes a predicate which is a BooleanArray to do the
filtering. How should this be used in practice? One usually
doesn'tknow which item should be filtered out or not in advance before
you actually do the filter, so won't know what the BooleanArray should
contain.
I would have thought that the predicate would be a function instead to
test if each item should be filtered out or not.
How is arrow <https://docs.rs/arrow/latest/arrow/index.html>::compute
<https://docs.rs/arrow/latest/arrow/compute/index.html>::kernels
<https://docs.rs/arrow/latest/arrow/compute/kernels/index.html>::filter
<https://docs.rs/arrow/latest/arrow/compute/kernels/filter/fn.filter.html#>::filter()
meant to be used in practice?
Thanks.
