Hey iceberg:

  Following the earlier decision [1] to deprecate the position delete with
row (PDWR) in iceberg release 1.11, I am seeking community's feedback on
handling of existing PDWR in maintenance action.

  Two actions today read the existing position deletes and write them back:
1. rewrite-position-delete
2. rewrite-table-path
Once the row-schema is gone from the AVRO/ORC/Parquet position delete
builder, no future writes can produce PDWR anymore and it aligns with our
earlier decision. The only question is where the row of existing PDWR gets
dropped, either silently inside the maintenance action or deliberately by
the table owner.

For context, the row column itself is optional and does not affect the data
correctness. Today when table upgrades from v2 to v3 and undergoes position
delete rewrite, the new deletion vector does not carry the row in existing
PDWR.

Personally, I am leaning toward failing with explicit exceptions like in PR
#17706[2], which means table owners need to make explicit decisions with
existing PDWR. The workaround includes upgrading to v3 and rewriting as DV.
Or staying in v2 and allow PDWR to be rewritten into data files during data
compaction.

Thanks,
Hongyue Zhang

[1] vote: https://lists.apache.org/thread/tfy96bqmz1bmdxr73x17w3xxj3yzs606
[2] iceberg pull: https://github.com/apache/iceberg/pull/17706

Reply via email to