singhpk234 commented on PR #5317: URL: https://github.com/apache/iceberg/pull/5317#issuecomment-1198885093
+1, I also observed this, have a possible RC, presently in case of any failure of completable future in `uploadParts` we call abortUpload() which further deletes all the staging files https://github.com/apache/iceberg/blob/3d00780f5f508ac7a0f475fd0e1ddfd736000c5b/aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java#L377-L389 Now when another completable future starts to read file for creating a request https://github.com/apache/iceberg/blob/3d00780f5f508ac7a0f475fd0e1ddfd736000c5b/aws/src/main/java/org/apache/iceberg/aws/s3/S3OutputStream.java#L323-L328 it fails with a FileNotFoundException (as staging files have been deleted from earlier future) Now when we do join of all the completable future we can get a failure due to `FileNotFound`. here is a gist for repro consistently & complete stack trace : https://gist.github.com/singhpk234/4257ea980017db5704857c3c7cc2fd0b I think this PR of mine can fix this flakyness as well : https://github.com/apache/iceberg/pull/5366 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
