fallintoplace opened a new pull request, #1906: URL: https://github.com/apache/iceberg-go/pull/1906
## What changed - Coalesce adjacent and overlapping deletion-vector blob ranges when `ReadDVs` reads one Puffin file. - Keep the output in the same order as the input files. - Keep metadata validation per blob and cap each coalesced range at 8 MiB. - Add a regression test for reversed input order and a benchmark for 2, 16, and 64 DVs. ## Why `ReadDVs` already opens a shared Puffin file once, but it still made one range read per DV blob. Puffin writers place blobs back-to-back, so reading adjacent payloads together cuts object-store requests. ## Benchmark Command: ```text go test ./table/dv -run '^$' -bench '^BenchmarkReadDVs$' -benchmem -benchtime=1s -count=5 ``` Before is `upstream/main` and after is this branch. These are medians over five runs. `range-reads/op` includes Puffin header and footer reads. | DVs in one Puffin | Before | After | | --- | --- | --- | | 2 | 24.9 µs/op, 4 range reads/op | 23.5 µs/op, 3 range reads/op | | 16 | 86.0 µs/op, 18 range reads/op | 68.2 µs/op, 3 range reads/op | | 64 | 272.6 µs/op, 72 range reads/op | 222.5 µs/op, 9 range reads/op | The coalesced buffer is capped at 8 MiB. The benchmark used about 5% more allocated bytes for the larger batches, while allocations dropped from 470 to 456 at 16 DVs and from 1,771 to 1,709 at 64 DVs. ## Testing - `go test ./... -count=1` - `go test ./table/dv -race -count=1` - `go vet ./table` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
