iceberg-kitten opened a new pull request, #2644: URL: https://github.com/apache/iceberg-go/pull/2644
## Summary The Avro manifest reader allocates a fresh inflater and output buffer for every deflate block. Reuse both across blocks of a single manifest reader while honoring the existing decompressed block-size limit. ## Benchmark Planning all 100,000 files from 50 manifests with one planner worker on a local Hadoop warehouse. Median allocated bytes per operation across three fresh processes per build and fixture, with four warm samples per process. | Fixture | Upstream main | With buffer reuse | Reduction | | --- | ---: | ---: | ---: | | Java-written manifests | 262.4 MiB | 170.7 MiB | 35% | | Go-written manifests | 254.7 MiB | 168.2 MiB | 34% | Wall-clock timings varied on the shared host. ## Validation - `go test -p 4 ./...` - Lint passed for the changed package. - Tests cover retained strings and bytes across blocks, truncated data, and the decompressed size limit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
