yihua opened a new issue, #994: URL: https://github.com/apache/hudi-rs/issues/994
When a read has filters, the file system view loads each candidate base file's parquet footer and prunes the file on its column statistics. The footers are read one after another, so planning N files waits for N sequential round trips, and on object storage each of those is a HEAD plus the footer GETs. The files are independent, so their footers can be read concurrently, bounded by `hoodie.plan.listing.parallelism` like the partition listing that precedes them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
