tustvold commented on issue #6002: URL: https://github.com/apache/arrow-rs/issues/6002#issuecomment-2273480391
FWIW when I set out to write the MetadataLoader the initial goal was for it to be push-based, however, I struggled to come up with a suitable interface for this in the time I had available. One option might be to return a special Error that allows it to "request" a range be loaded, but it ends up pretty gnarly. IMO the trick is to share the sync decoding logic and expose it an ergonomic way, and accept that the IO piece will have to be different for async vs non-async. This is broadly the pattern that is used throughout the parquet crate, and I don't really see a way around it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
