GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/3219
[FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows
This PR hardens `FileBaseUtils#deleteFileOrDirectory` when the method is
called concurrently on Windows.
`AccessDeniedException`s can be thrown when attempting to delete a file
while a deletion is in progress. In order to delete a file it has to be opened,
and as we all know you can't (easily) delete open files on Windows.
For this case we catch the exception, double-check whether the file is
deleted by now, and throw the exception if it isn't, assuming a lack of
permission prevented the deletion.
`DirectoryNotEmptyException`s can be thrown for directories for which a
delete call for every file has succeeded, since the OS may be slow in updating
the actual FileSystem.
For this case we wait for a bit (50 milliseconds, yeah it's that slow) and
check whether the directory is empty by now. If it isn't we throw the exception
assuming a concurrent creation of a new file in the given directory.
Note that neither changes are perfect; they are not guaranteed to prevent
the issue. However, it no longer categorically fails for me ð, but only once
every few hundred runs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 5659_fileutils_win
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3219.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3219
----
commit f372899c7cd8f5102421b986be1f1b87e32cf15f
Author: zentol <[email protected]>
Date: 2017-01-26T14:47:11Z
[FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
