link3280 commented on code in PR #16927:
URL: https://github.com/apache/flink/pull/16927#discussion_r1068856775
##########
flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java:
##########
@@ -360,4 +391,37 @@ private static boolean waitUntilLeaseIsRevoked(final
FileSystem fs, final Path p
}
return isClosed;
}
+
+ /**
+ * If the last block of the file that the previous execution was writing
to is not in COMPLETE
+ * state, HDFS will perform block recovery which blocks truncate. Thus we
have to wait for block
+ * recovery to ensure the truncate is successful.
+ */
+ private static boolean waitForBlockRecovery(final FileSystem fs, final
Path path)
Review Comment:
I'm not particularly familiar with this API, but I think it may work and is
worth a try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
