According to FileSystem.rename() we should be catching java.io.FileNotFoundException instead of java.nio.file.FileAlreadyExistsException.
Also, I'm not 100% sure if GCS will throw the proper exception here (java.io.FileNotFoundException). Unfortunately we cannot properly test this in unittests unless we manually introduce failures and connect to GCS service. Can you try executing rename() operation for a batch of files including and make sure that rename() operation will succeed in following cases for GCS (and other file-systems but probably you already tested this). Otherwise, this could end up being a major regression for Dataflow since bundle re-execution will not operate properly. (1) FileSystem.rename() with MoveOptions.StandardMoveOptions.REPLACE_EXISTING for a set of files where some of the files in the destination set are already available. (2) FileSystem.rename() with MoveOptions.StandardMoveOptions. IGNORE_MISSING_FILES for a set of files where some of the source files are not available. (3) FileSystem.rename() with both MoveOptions.StandardMoveOptions.REPLACE_EXISTING and MoveOptions.StandardMoveOptions. IGNORE_MISSING_FILES for a set of files where both above conditions are true. (As before, please let me know if you need help with GCS/Dataflow). [ Full content available at: https://github.com/apache/beam/pull/6289 ] This message was relayed via gitbox.apache.org for [email protected]
