openinx commented on a change in pull request #2886:
URL: https://github.com/apache/iceberg/pull/2886#discussion_r679097460
##########
File path: flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java
##########
@@ -246,52 +248,71 @@ public Builder uidPrefix(String newPrefix) {
}
}
- // Find out the equality field id list based on the user-provided
equality field column names.
- List<Integer> equalityFieldIds = Lists.newArrayList();
- if (equalityFieldColumns != null && equalityFieldColumns.size() > 0) {
- for (String column : equalityFieldColumns) {
- org.apache.iceberg.types.Types.NestedField field =
table.schema().findField(column);
- Preconditions.checkNotNull(field, "Missing required equality field
column '%s' in table schema %s",
- column, table.schema());
- equalityFieldIds.add(field.fieldId());
- }
- }
-
// Convert the requested flink table schema to flink row type.
RowType flinkRowType = toFlinkRowType(table.schema(), tableSchema);
// Distribute the records from input data stream based on the
write.distribution-mode.
rowDataInput = distributeDataStream(rowDataInput, table.properties(),
table.spec(), table.schema(), flinkRowType);
- // Chain the iceberg stream writer and committer operator.
- IcebergStreamWriter<RowData> streamWriter = createStreamWriter(table,
flinkRowType, equalityFieldIds);
- IcebergFilesCommitter filesCommitter = new
IcebergFilesCommitter(tableLoader, overwrite);
+ // Add parallel writers that append rows to files
+ SingleOutputStreamOperator<WriteResult> writerStream =
appendWriter(rowDataInput, flinkRowType);
Review comment:
The
[unchecked](https://github.com/apache/iceberg/pull/2886/files#diff-a26cb8b3f0f5b0e6a9f110bb0b3fdd0ed625bfc9faf27e248f6446818cad4d69L234)
warning can be removed now ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
