rdblue commented on a change in pull request #351: Extend Iceberg with a way to
overwrite files for eager updates/deletes
URL: https://github.com/apache/incubator-iceberg/pull/351#discussion_r315462125
##########
File path: api/src/main/java/org/apache/iceberg/OverwriteFiles.java
##########
@@ -74,5 +85,23 @@
*
* @return this for method chaining
*/
- OverwriteFiles validateAddedFiles();
+ OverwriteFiles validateAddedFilesMatchOverwriteFilter();
+
+ /**
+ * Enables validation that files added concurrently do not conflict with
this commit's operation.
+ * <p>
+ * This method should be called when the table is queried to determine which
files to delete/append.
+ * If a concurrent operation commits a new file after the data was read and
that file might
+ * contain rows matching the specified conflict detection filter, the
overwrite operation
+ * will detect this during retries and fail.
+ * <p>
+ * Calling this method with a correct conflict detection filter is required
to maintain
Review comment:
I agree with your take on it, though I'm not sure that I agree that
compaction isn't serializable. That operation takes effect when it is committed
and can be relocated in the order of commits without changing the work it did.
But once it is committed, it is present in the order of operations and that
won't change.
Anyway, I think that isolation level isn't a useful table config. Some
individual operations will probably want to relax it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
