akritishakya commented on code in PR #2352:
URL: https://github.com/apache/iceberg-go/pull/2352#discussion_r4442137609
##########
table/snapshots.go:
##########
@@ -106,15 +110,29 @@ type updateMetrics struct {
removedEqDeletes int64
}
+// contentSizeInBytes returns the bytes a file contributes to the table. A DV
+// shares its Puffin file with other DVs, so it counts only its own blob.
+func contentSizeInBytes(df iceberg.DataFile) int64 {
+ if IsDeletionVector(df) && df.ContentSizeInBytes() != nil {
+ return *df.ContentSizeInBytes()
+ }
+
+ return df.FileSizeBytes()
+}
+
func (m *updateMetrics) addDataFile(df iceberg.DataFile) error {
- m.addedFileSize += df.FileSizeBytes()
+ m.addedFileSize += contentSizeInBytes(df)
switch df.ContentType() {
case iceberg.EntryContentData:
m.addedDataFiles++
m.addedRecords += df.Count()
case iceberg.EntryContentPosDeletes:
m.addedDeleteFiles++
Review Comment:
Added under **Unreleased compatibility notes** in `website/src/releases.md`.
Nuance: `total-files-size` isn't always lower than before, since removing a
DV written by an older iceberg-go subtracts only its content size, but the
commit that added it counted the whole Puffin file size. So the note says the
total "is updated from these values" rather than "goes down".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
