fallintoplace opened a new pull request, #1764: URL: https://github.com/apache/iceberg-go/pull/1764
## Summary - group deletion vector loads by physical Puffin path - open and decode each shared Puffin file footer once - keep a fast path for one deletion vector - keep concurrency across different Puffin files ## Why The DV writer stores one blob per data file in a shared Puffin file. The scanner was calling `ReadDV` once per blob, so 100 DVs in one Puffin opened and decoded the same file 100 times. ## Benchmarks `BenchmarkReadAllDeletionVectorsSharedPuffin` on Apple M1 Pro: | DVs | main | this PR | speedup | | ---: | ---: | ---: | ---: | | 1 | 49.98 us | 29.92 us | 1.67x | | 10 | 672.21 us | 97.03 us | 6.93x | | 100 | 19.35 ms | 558.17 us | 34.66x | At 100 DVs, allocations drop from 19.3 MB and 125,737 allocs to 278 KB and 2,793 allocs. The shared Puffin file is opened once instead of 100 times. ## Tests - `go test ./...` - `go test -race ./table/dv ./table -run "TestReadDV|TestReadDVs|TestReadAllDeletionVectors" -count=1` - `go vet ./...` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
