alamb commented on issue #17211: URL: https://github.com/apache/datafusion/issues/17211#issuecomment-3272260490
> These first queries really begin to show discrepancies. In spite of the GET requests being identical between the two queries, the partitioned table executes 22 LIST operations compared to the flat table's 1 LIST operation. Yes this seems like a quite compelling reason to: 1. Cache the listing results 2. Get the datafusion-cli listing feature polished up. > This data also suggests that if a ListFilesCache were to be implemented and enabled it could eliminate all the IO for simple queries that can be computed entirely from metadata. This sounds great and a worthy goal for sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
