ahmedabu98 commented on code in PR #33509:
URL: https://github.com/apache/beam/pull/33509#discussion_r1907727748
##########
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/AppendFilesToTables.java:
##########
@@ -201,14 +200,15 @@ private void appendManifestFiles(Table table,
Iterable<FileWriteResult> fileWrit
}
private ManifestWriter<DataFile> createManifestWriter(
- String tableLocation, String uuid, PartitionSpec spec, FileIO io) {
+ String tableLocation, String uuid, PartitionSpec spec, Table table) {
Review Comment:
Thx for the catch. I refactored this a lil to make it more clean.
btw `spec` here isn't a simple `table.spec()`. This part of the code deals
with the edge case where we have a batch of data files that were written with
different specs (e.g. user updates spec during pipeline runtime). We group data
files by spec and create/append one ManifestFile per spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
