alamb commented on issue #6692: URL: https://github.com/apache/arrow-rs/issues/6692#issuecomment-2462539942
> Not sure about other type but for StringView, I can only think of iterating all the filtered row and append_value one by one. If there is no further optimization we can do, I think we can implement the append logic in datafusion @jayzhan211 yes I think this is effectively what would happen -- however the actual iteration over filtered values is quite optimized in the `filter` kernel (checkout what the [FilterBuilder](https://github.com/apache/arrow-rs/blob/e907bf8e3df5cd8cba8b5f761f9291f2ec0c2730/arrow-select/src/filter.rs#L175) does) based on how many values are filtered and other aspect The fact that `filter` is so fast in arrow means it is quite hard to get as good / faster :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
