alamb commented on issue #7762: URL: https://github.com/apache/arrow-rs/issues/7762#issuecomment-3174866005
> I experimented a POC PR: [#8103](https://github.com/apache/arrow-rs/pull/8103) > > And it shows it only improve performance for selective 0.001 case, about 2.x faster, but other cases have regression. Need further investigation. yeah, this is the case I expect will go much faster (very selective filter) I think the key game will be avoiding regressions for the others. I wonder if the regression is caused by trying to keep target batches exactly the same size (the same thing as https://github.com/apache/datafusion/pull/17105#issuecomment-3174858830 ) 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
