rdblue commented on a change in pull request #351: Extend Iceberg with a way to
overwrite files for eager updates/deletes
URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r313643529
##########
File path: api/src/main/java/org/apache/iceberg/OverwriteFiles.java
##########
@@ -50,6 +49,12 @@
* <p>
* Files that may contain some rows that match the expression and some rows
that do not will
* result in a {@link ValidationException}.
+ * <p>
+ * This has no requirements for the latest snapshot and will not fail based
on other snapshot
+ * changes.
+ * <p>
+ * Note that either this method or {@link
OverwriteFiles#deleteFile(DataFile)}
+ * should be used to mark deleted files, not both.
Review comment:
This update isn't quite true. There are methods to determine which files to
delete, and there are separate ways to configure validations.
When this operation is used to produce an idempotent overwrite, where it can
be completed regardless of what files are added after it starts, it doesn't
matter that the files were matched by a filter or directly by `deleteFile`. For
example, I might want to overwrite an old file with new values for a column
that was added. That overwrite is idempotent, but might use `deleteFile`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
