RussellSpitzer commented on a change in pull request #3480:
URL: https://github.com/apache/iceberg/pull/3480#discussion_r751658945
##########
File path: core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java
##########
@@ -313,6 +316,23 @@ protected void
validateNoNewDeletesForDataFiles(TableMetadata base, Long startin
protected void validateNoNewDeletesForDataFiles(TableMetadata base, Long
startingSnapshotId,
Expression dataFilter,
Iterable<DataFile> dataFiles,
boolean caseSensitive) {
+ validateNoNewDeletesForDataFiles(base, startingSnapshotId, dataFilter,
dataFiles, caseSensitive, false);
+ }
+
+ /**
+ * Validates that no new delete files that must be applied to the given data
files have been added to the table since
+ * a starting snapshot, with the option to ignore equality deletes during
the validation.
Review comment:
Nbd, but I would add a note here about why we want to ignore equality
deletes, just so future readers could understand.
##########
File path: api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java
##########
@@ -77,6 +77,17 @@
*/
String TARGET_FILE_SIZE_BYTES = "target-file-size-bytes";
+ /**
+ * If the compaction should use the sequence number of the snapshot at
compaction start time for new data files,
+ * instead of using the sequence number of the newly produced snapshot.
+ * <p>
+ * This avoids commit conflicts with updates that add newer equality deletes
at a higher sequence number.
+ * <p>
+ * Defaults to true.
+ */
+ String USE_STARTING_SEQUENCE_NUMBER = "use-starting-sequence-number";
+ boolean USE_STARTING_SEQUENCE_NUMBER_DEFAULT = true;
Review comment:
Now that this is true, do we have to ignore it with V1 Tables?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
