I am looking for views on keeping the one flag true by default to use EagerInputFile in ManifestFiles read path (Major benefits in v4 Parquet Manifests). We have ready to run benchmark [1] and two independent sets of results showing similar benefits ~25 - 55% reduction in reading Parquet manifest depending on machine and file.
I would appreciate the community's look and feedback on this - PR having changes [2] and whether we can enable this path by default. [1] https://github.com/varun-lakhyani/iceberg/pull/1 [2] https://github.com/apache/iceberg/pull/17284 On Wed, Jul 29, 2026 at 6:29 AM Russell Spitzer <[email protected]> wrote: > I'm not sure if anyone else has checked this out yet, but I think it's a > really exciting improvement. We should seriously consider making this a > default feature in the next release. Or potentially just have it always be > on. > > On Tue, Jul 28, 2026 at 4:58 PM Varun Lakhyani <[email protected]> > wrote: > >> I think we can flag property to enable this as default true, It would be >> a great improvement for v4 parquet manifest. >> Tried to do benchmarkings as extensive as possible. >> I would love to hear thoughts on this. >> >> Please review the PR once and would appreciate feedback. >> >> On Mon, Jul 20, 2026 at 10:02 PM Varun Lakhyani < >> [email protected]> wrote: >> >>> Hello Everyone, >>> >>> I evaluated using EagerInputFile in v4 manifest reader path (parquet) >>> using ManifestBenchmark and S3 as object store and EC2 machine for jmh >>> benchmarking. >>> >>> This is to make sure we are doing a proper checklist while wiring this >>> EagerInputFile path here, >>> Detailed benchmarking setup and results are commented on in PR[1]. >>> Using an EC2 machine in the same region (ap-south-1) gives *~28.4%* to >>> *~42.7%* betterment while comparing against baseline default while >>> benchmarking. >>> >>> Please provide review and feedback. >>> Thanks >>> >>> [1] https://github.com/apache/iceberg/pull/17284 >>> -- >>> Lakhyani Varun >>> Indian Institute of Technology Roorkee >>> Contact: +91 96246 46174 >>> >>>
