aokolnychyi commented on a change in pull request #3195:
URL: https://github.com/apache/iceberg/pull/3195#discussion_r717180228
##########
File path: api/src/main/java/org/apache/iceberg/RowDelta.java
##########
@@ -109,6 +109,54 @@
*
* @param conflictDetectionFilter an expression on rows in the table
* @return this for method chaining
+ * @deprecated since 0.13.0, will be removed in 0.14.0; use {@link
#conflictDetectionFilter(Expression)} and
+ * {@link #validateNoConflictingDataFiles()} instead.
*/
- RowDelta validateNoConflictingAppends(Expression conflictDetectionFilter);
+ @Deprecated
+ default RowDelta validateNoConflictingAppends(Expression
conflictDetectionFilter) {
+ conflictDetectionFilter(conflictDetectionFilter);
+ return validateNoConflictingDataFiles();
+ }
+
+ /**
+ * Sets a conflict detection filter used to validate concurrently added data
and delete files.
+ * <p>
+ * If not called, a true literal will be used as the conflict detection
filter.
+ *
+ * @param conflictDetectionFilter an expression on rows in the table
+ * @return this for method chaining
+ */
+ RowDelta conflictDetectionFilter(Expression conflictDetectionFilter);
Review comment:
This is actually different compared to the implementation in PR #3069. I
think this one is slightly better as we set the conflict detection filter only
once and then enable data/delete file validation.
##########
File path: api/src/main/java/org/apache/iceberg/RowDelta.java
##########
@@ -109,6 +109,54 @@
*
* @param conflictDetectionFilter an expression on rows in the table
* @return this for method chaining
+ * @deprecated since 0.13.0, will be removed in 0.14.0; use {@link
#conflictDetectionFilter(Expression)} and
+ * {@link #validateNoConflictingDataFiles()} instead.
*/
- RowDelta validateNoConflictingAppends(Expression conflictDetectionFilter);
+ @Deprecated
+ default RowDelta validateNoConflictingAppends(Expression
conflictDetectionFilter) {
+ conflictDetectionFilter(conflictDetectionFilter);
+ return validateNoConflictingDataFiles();
+ }
+
+ /**
+ * Sets a conflict detection filter used to validate concurrently added data
and delete files.
+ * <p>
+ * If not called, a true literal will be used as the conflict detection
filter.
+ *
+ * @param conflictDetectionFilter an expression on rows in the table
+ * @return this for method chaining
+ */
+ RowDelta conflictDetectionFilter(Expression conflictDetectionFilter);
Review comment:
Otherwise, we would need to reason what happens if they are different,
which one to use for validating referenced data files, etc.
##########
File path: api/src/main/java/org/apache/iceberg/RowDelta.java
##########
@@ -109,6 +109,54 @@
*
* @param conflictDetectionFilter an expression on rows in the table
* @return this for method chaining
+ * @deprecated since 0.13.0, will be removed in 0.14.0; use {@link
#conflictDetectionFilter(Expression)} and
+ * {@link #validateNoConflictingDataFiles()} instead.
*/
- RowDelta validateNoConflictingAppends(Expression conflictDetectionFilter);
+ @Deprecated
+ default RowDelta validateNoConflictingAppends(Expression
conflictDetectionFilter) {
+ conflictDetectionFilter(conflictDetectionFilter);
+ return validateNoConflictingDataFiles();
+ }
+
+ /**
+ * Sets a conflict detection filter used to validate concurrently added data
and delete files.
+ * <p>
+ * If not called, a true literal will be used as the conflict detection
filter.
+ *
+ * @param conflictDetectionFilter an expression on rows in the table
+ * @return this for method chaining
+ */
+ RowDelta conflictDetectionFilter(Expression conflictDetectionFilter);
Review comment:
This is actually different from the implementation in PR #3069. I think
this one is slightly better as we set the conflict detection filter only once
and then enable data/delete file validation.
##########
File path: api/src/main/java/org/apache/iceberg/RowDelta.java
##########
@@ -109,6 +109,54 @@
*
* @param conflictDetectionFilter an expression on rows in the table
* @return this for method chaining
+ * @deprecated since 0.13.0, will be removed in 0.14.0; use {@link
#conflictDetectionFilter(Expression)} and
+ * {@link #validateNoConflictingDataFiles()} instead.
*/
- RowDelta validateNoConflictingAppends(Expression conflictDetectionFilter);
+ @Deprecated
+ default RowDelta validateNoConflictingAppends(Expression
conflictDetectionFilter) {
+ conflictDetectionFilter(conflictDetectionFilter);
+ return validateNoConflictingDataFiles();
+ }
+
+ /**
+ * Sets a conflict detection filter used to validate concurrently added data
and delete files.
+ * <p>
+ * If not called, a true literal will be used as the conflict detection
filter.
+ *
+ * @param conflictDetectionFilter an expression on rows in the table
+ * @return this for method chaining
+ */
+ RowDelta conflictDetectionFilter(Expression conflictDetectionFilter);
Review comment:
Otherwise, we would need to reason about what happens if the filters are
different, which one to use for validating referenced data files, etc.
##########
File path: core/src/main/java/org/apache/iceberg/DeleteFileIndex.java
##########
@@ -86,6 +86,20 @@ public boolean isEmpty() {
return (globalDeletes == null || globalDeletes.length == 0) &&
sortedDeletesByPartition.isEmpty();
}
+ public List<DeleteFile> referencedDeleteFiles() {
Review comment:
Updated.
##########
File path: core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java
##########
@@ -320,6 +333,56 @@ protected void
validateNoNewDeletesForDataFiles(TableMetadata base, Long startin
}
}
+ /**
+ * Validates that no delete files matching a filter have been added to the
table since a starting snapshot.
+ *
+ * @param base table metadata to validate
+ * @param startingSnapshotId id of the snapshot current at the start of the
operation
+ * @param dataFilter an expression used to find new conflicting delete files
+ * @param caseSensitive whether expression evaluation should be
case-sensitive
+ */
+ protected void validateNoNewDeleteFiles(TableMetadata base, Long
startingSnapshotId,
+ Expression dataFilter, boolean
caseSensitive) {
+ // if there is no current table state, no files have been added
+ if (base.currentSnapshot() == null) {
Review comment:
Added to both methods. Could you check, @rdblue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
