tanmayrauth commented on code in PR #1113:
URL: https://github.com/apache/iceberg-go/pull/1113#discussion_r3291926599
##########
table/properties.go:
##########
@@ -76,6 +76,11 @@ const (
WriteDeleteModeKey = "write.delete.mode"
WriteDeleteModeDefault = WriteModeCopyOnWrite
+ WriteDeleteFormatKey = "write.delete.format"
+
+ WriteDeleteFormatPosition = "position"
+ WriteDeleteFormatDV = "dv"
Review Comment:
Agreed, both the semantic conflict with Java's write.delete.format.default
and the "Java doesn't expose a knob" point. Dropped the property and its
constants entirely. Routing is now purely version + partitioning driven: v3+
unpartitioned uses DV, everything else uses Parquet position deletes.
##########
table/arrow_utils.go:
##########
@@ -1646,10 +1647,19 @@ func positionDeleteRecordsToDataFiles(ctx
context.Context, rootLocation string,
}
}
- // V3 and later prefer deletion vectors over Parquet position-delete
files;
- // warn so users migrate when DV-write support lands. The check is `>=
3`
- // rather than `== 3` so the warning carries forward to v4+ without
churn.
- // See apache/iceberg#12048.
+ deleteFormat := meta.props.Get(WriteDeleteFormatKey, "")
+ if deleteFormat == "" {
+ if latestMetadata.Version() >= 3 {
+ deleteFormat = WriteDeleteFormatDV
+ } else {
+ deleteFormat = WriteDeleteFormatPosition
+ }
+ }
+
+ if deleteFormat == WriteDeleteFormatDV {
+ return positionDeleteRecordsToDataFilesDV(ctx, rootLocation,
args)
+ }
Review Comment:
Good catch. Going to defer DV+partitioned support to a follow-up to keep
this PR scoped. Updated the routing so v3 partitioned writes fall through to
the existing Parquet position-delete writer (which already handles partition
fanout correctly) and continue to emit the prefer-DV warning. The DV branch is
now gated on v3+ AND unpartitioned, with a comment explaining why. Will open a
follow-up to thread the partition context map through DVWriter and add the
partitioned test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
