Dandandan commented on PR #20916: URL: https://github.com/apache/datafusion/pull/20916#issuecomment-4063574790
> @Dandandan if this simplifies code and improves performance, should we move forward with it? My only concern is that results would be very different running against object storage... Yeah - we should add a benchmark first to see if this doesn't do horribly on object storage - I guess it might improve some things as it still prefetches the footer. @alamb mentioned we could use / adapt https://docs.rs/object_store/0.13.1/object_store/throttle/struct.ThrottledStore.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
