bharadwaj-pendyala commented on issue #11200: URL: https://github.com/apache/arrow-rs/issues/11200#issuecomment-5873160500
I've got a version of the first option working: zip the 64-bit words of the mask and the null buffer and stop at the first `keep & !valid`, falling back to probing the indices when the predicate is sparse `Indices` (same crossover `filter_bits` uses). On an M1 laptop the two denser only-valid filters drop from 73-222µs to 42-74µs at 64k rows, and the worst case I could build, a dense filter whose only selected null is in the last word, is about 7% slower. CONTRIBUTING asks for new benchmarks in their own PR, so the bench cases go up first and the kernel change right after. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
