stevenzwu commented on a change in pull request #2217:
URL: https://github.com/apache/iceberg/pull/2217#discussion_r572194593
##########
File path:
flink/src/main/java/org/apache/iceberg/flink/actions/MigrateAction.java
##########
@@ -294,27 +292,26 @@ private Namespace toNamespace() {
return buildManifest(conf, spec, metadataLocation, files);
}
- private List<ManifestFile> buildManifest(Configuration conf, PartitionSpec
partitionSpec,
- String basePath, List<DataFile>
dataFiles) {
+ private ManifestFile buildManifest(Configuration conf, PartitionSpec
partitionSpec,
+ String basePath, List<DataFile>
dataFiles) {
if (dataFiles.size() > 0) {
FileIO io = new HadoopFileIO(conf);
int subTaskId = getRuntimeContext().getIndexOfThisSubtask();
int attemptId = getRuntimeContext().getAttemptNumber();
- String suffix = String.format("%d-task-%d-manifest", subTaskId,
attemptId);
+ String suffix = String.format("manifest-%d-%d", subTaskId, attemptId);
Path location = new Path(basePath, suffix);
String outputPath = FileFormat.AVRO.addExtension(location.toString());
OutputFile outputFile = io.newOutputFile(outputPath);
ManifestWriter<DataFile> writer = ManifestFiles.write(partitionSpec,
outputFile);
- try (ManifestWriter<DataFile> writerRef = writer) {
- dataFiles.forEach(writerRef::add);
+ try (ManifestWriter<DataFile> manifestWriter = writer) {
+ dataFiles.forEach(manifestWriter::add);
} catch (IOException e) {
throw new UncheckedIOException("Unable to close the manifest
writer", e);
}
- ManifestFile manifestFile = writer.toManifestFile();
- return Lists.newArrayList(manifestFile);
+ return writer.toManifestFile();
} else {
- return Lists.newArrayList();
+ return null;
Review comment:
now we got null value. `AppendFiles#appendManifest` doesn't handle null
values. is `flatMap` better?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
