sdf-jkl commented on issue #8844: URL: https://github.com/apache/arrow-rs/issues/8844#issuecomment-4155746969
Looking at where #9414 is heading, we will defer filters that introduce fragmentation in selectors until the final read. Currently, the deferral threshold is stricter than the threshold for switching to a bitmask. This means the iterative mask will effectively remain materialized as selectors, since it will never become fragmented enough to trigger a switch to a bitmask. As a result, conversion to a bitmask should only occur during the final read (when we merge deferred selections), allowing us to avoid unnecessary conversions. We can wait until #9414 and #8846 are closed and revisit closing this issue if the situation remains the same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
