akritishakya opened a new pull request, #2352: URL: https://github.com/apache/iceberg-go/pull/2352
### Problem In v3 tables, snapshot summaries over-count the size of deletion vectors. `DVWriter` writes all DVs from one flush into a single Puffin file, and every DV's manifest entry carries that whole Puffin file's size in `file_size_in_bytes`. The summary added `file_size_in_bytes` for each DV, so a Puffin file holding N DVs was counted N times in `added-files-size`. When a single DV was removed, the whole Puffin file size was subtracted from `removed-files-size`, even though the other DVs in that file stay live. Because `total-files-size` is computed as previous + added − removed, the error carried forward into every later snapshot. Example: two DVs (300 and 400 bytes) in one 1000-byte Puffin file reported `added-files-size = 2000` instead of `700`. ### Fix Added a `contentSizeInBytes` helper in `table/snapshots.go` that returns `content_size_in_bytes` for a deletion vector and `file_size_in_bytes` for every other file. `updateMetrics.addDataFile` and `updateMetrics.removeFile` now use it. A DV without a content size falls back to the file size. This matches Java's `SnapshotSummary`, which uses `ScanTaskUtil.contentSizeInBytes` for the same purpose. ### Behaviour Change For commits that add or remove DVs, `added-files-size`, `removed-files-size` and `total-files-size` (and the per-partition size values) now count each DV's own blob size instead of its Puffin file's size. The commit report's `AddedFilesSizeBytes`, `RemovedFilesSizeBytes` and `TotalFilesSizeBytes` change the same way. Sizes for data files, Parquet position deletes and equality deletes are unchanged. ### Testing Extended `TestSnapshotSummaryCollectorDeletionVectors` to assert `added-files-size` and `removed-files-size`. The test adds two DVs that share one Puffin file plus a Parquet position delete, and removes one DV: - `added-files-size`: `1500` (300 + 400 + 800); previously `2800` - `removed-files-size`: `250`; previously `600` Both assertions fail without the fix and pass with it. `go test ./table/...` passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
