alamb commented on issue #7582: URL: https://github.com/apache/arrow-rs/issues/7582#issuecomment-5766025366
> One thing I though of while working on this is that the Keep/Grid approach could also be used to selectively populate the metadata itself. We could keep the in-file indexing for row groups and columns, but only store the subset of metadata needed for a given query. This is phase three fanciness -- I love it Whereby phase three is my three level model of optimization (initial implementation --> rewrite to be good --> rewrite to be really good, but also can't be made any better without a rearchitecture) I think we are getting near phase three with some parts of this crate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
