felipecrv commented on issue #25822: URL: https://github.com/apache/arrow/issues/25822#issuecomment-2531621789
@alexeykudinkin take/filter on chunked arrays requires resolution of chunks which is more expensive than simple array offsetting [1]. Solutions that don't concatenate tend to be slower and are considered unacceptable. If it were up to me, I would leave the decision of concatenation to callers and never concatenate. [I'm going to compromise](https://github.com/apache/arrow/pull/41700#issuecomment-2531607070) and add conditional checks on sizes to decide if we concatenate or not. What I don't like about this solution is that it kinda doubles the amount of tests we need to run to cover both algorithms. [1] https://github.com/apache/arrow/pull/41700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
