rdblue commented on a change in pull request #1818:
URL: https://github.com/apache/iceberg/pull/1818#discussion_r529998358
##########
File path: core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java
##########
@@ -51,39 +60,149 @@ protected BaseTaskWriter(PartitionSpec spec, FileFormat
format, FileAppenderFact
this.targetFileSize = targetFileSize;
}
+ protected PartitionSpec spec() {
+ return spec;
+ }
+
+ protected FileAppenderFactory<T> appenderFactory() {
+ return appenderFactory;
+ }
+
@Override
public void abort() throws IOException {
close();
// clean up files created by this writer
- Tasks.foreach(completedFiles)
+ Tasks.foreach(Iterables.concat(completedFiles, completedDeletes))
.throwFailureWhenFinished()
.noRetry()
.run(file -> io.deleteFile(file.path().toString()));
}
@Override
- public DataFile[] complete() throws IOException {
+ public WriterResult complete() throws IOException {
close();
- return completedFiles.toArray(new DataFile[0]);
+ return WriterResult.builder()
+ .addDataFiles(completedFiles)
+ .addDeleteFiles(completedDeletes)
+ .build();
+ }
+
+ protected abstract class BaseDeltaWriter implements Closeable {
+ private final RollingFileWriter dataWriter;
+
+ private final boolean enableEqDelete;
+ private RollingEqDeleteWriter eqDeleteWriter = null;
+ private SortedPosDeleteWriter<T> posDeleteWriter = null;
+ private StructLikeMap<FilePos> insertedRowMap = null;
Review comment:
Minor: Do we need to use `StructLikeMap` in declarations or can we use
`Map<StructLike, FilePos>`? The latter seems easier to read, even if the map
implementation is `StructLikeMap`.
I think the risk is someone accidentally replacing the implementation with a
hash map, but we have that problem in places where we use `Maps.newTreeMap()`
as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
