iceberg-kitten opened a new pull request, #2644:
URL: https://github.com/apache/iceberg-go/pull/2644

   ## Summary
   
   The Avro manifest reader allocates a fresh inflater and output buffer for 
every deflate block. Reuse both across blocks of a single manifest reader while 
honoring the existing decompressed block-size limit.
   
   ## Benchmark
   
   Planning all 100,000 files from 50 manifests with one planner worker on a 
local Hadoop warehouse. Median allocated bytes per operation across three fresh 
processes per build and fixture, with four warm samples per process.
   
   | Fixture | Upstream main | With buffer reuse | Reduction |
   | --- | ---: | ---: | ---: |
   | Java-written manifests | 262.4 MiB | 170.7 MiB | 35% |
   | Go-written manifests | 254.7 MiB | 168.2 MiB | 34% |
   
   Wall-clock timings varied on the shared host.
   
   ## Validation
   
   - `go test -p 4 ./...`
   - Lint passed for the changed package.
   - Tests cover retained strings and bytes across blocks, truncated data, and 
the decompressed size limit.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to