openinx commented on a change in pull request #1213:
URL: https://github.com/apache/iceberg/pull/1213#discussion_r460684652
##########
File path: core/src/main/java/org/apache/iceberg/io/PartitionedWriter.java
##########
@@ -63,10 +66,29 @@ public void write(InternalRow row) throws IOException {
throw new IllegalStateException("Already closed files for partition: "
+ key.toPath());
}
- setCurrentKey(key.copy());
- openCurrent();
+ currentKey = key.copy();
+ }
+
+ if (currentAppender == null) {
+ currentAppender = new RollingFileAppender(currentKey);
Review comment:
Now we've changed to maintain the partitionKey inside the
`RollingFileWriter` (we've discussed this before , this is because for fanout
writer we may have multiple writers append records), so the
RollingFileAppender creation is actually doing the partition key setting. I
did not open the appender here because we only need to open an appender when
there's a real record to write (in case of opening an appender without writing
a record) , all those logic have been hidden inside the RollingFileAppender.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
