danielcweeks commented on a change in pull request #3813:
URL: https://github.com/apache/iceberg/pull/3813#discussion_r789144125
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java
##########
@@ -172,15 +187,23 @@ public void write(byte[] b, int off, int len) throws
IOException {
private void newStream() throws IOException {
if (stream != null) {
- stream.close();
+ closeStream();
}
createStagingDirectoryIfNotExists();
currentStagingFile = File.createTempFile("s3fileio-", ".tmp",
stagingDirectory);
currentStagingFile.deleteOnExit();
- stagingFiles.add(currentStagingFile);
- stream = new CountingOutputStream(new BufferedOutputStream(new
FileOutputStream(currentStagingFile)));
+ stagingFiles.add(new FileAndDigest(currentStagingFile));
+
+ if (isEtagCheckEnabled) {
+ currentPartMessageDigest.reset();
Review comment:
This overlaps with Ryan's comment below, but if we just create a message
digest and associate it here, we don't need to update it in the close method
below.
Something like:
```java
stagingFiles.add(new FileAndDigest(currentStagingFile, newMessageDigest));
```
It means that we have to create a new digest object per file but also means
they're standalone and don't require as much coordination.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
