rdblue commented on a change in pull request #1939:
URL: https://github.com/apache/iceberg/pull/1939#discussion_r545291361
##########
File path:
flink/src/main/java/org/apache/iceberg/flink/sink/FlinkManifestUtil.java
##########
@@ -63,4 +66,53 @@ static ManifestOutputFileFactory
createOutputFileFactory(Table table, String fli
TableOperations ops = ((HasTableOperations) table).operations();
return new ManifestOutputFileFactory(ops, table.io(), table.properties(),
flinkJobId, subTaskId, attemptNumber);
}
+
+ static DeltaManifests writeCompletedFiles(WriteResult result,
Review comment:
We should serialize it and add it to the commit. This is the set of
files that is referenced by any positional delete, which identifies deleted
rows by file and row position. The commit will validate that all of the files
still exist in the table.
This isn't strictly needed for this use case because we know that the
position deletes only refer to files that are created in this commit. Since the
files are being added in the commit, it isn't possible for some other process
to delete some of them from metadata. But it is still good to configure the
commit properly in case this gets reused later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
