mxm opened a new pull request, #16948:
URL: https://github.com/apache/iceberg/pull/16948
Flink's upsert mode writes equality deletes, which are cheap to produce but
force every reader to join the data against the delete files (merge-on-read).
ConvertEqualityDeletes is a maintenance task that rewrites them into deletion
vectors on a target branch, so reads apply deletes by position. The writer
keeps appending equality deletes to a staging branch while the conversion runs
in the background.
Example:
```
TableMaintenance.forTable(env, tableLoader)
.add(
ConvertEqualityDeletes.builder()
.stagingBranch("staging")
.targetBranch("main")
.equalityFieldColumns(ImmutableList.of("id"))
.scheduleOnEqDeleteFileCount(1)
.append();
```
The table must be V3 for deletion vectors, the equality field columns must
match the writer's, and every partition column must be an equality field,
because the converter keys rows on equality values alone and would otherwise
resolve a delete against the wrong partition. The conversion task has checks in
place to ensure this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
