[
https://issues.apache.org/jira/browse/FLINK-5659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839808#comment-15839808
]
ASF GitHub Bot commented on FLINK-5659:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/3219
[FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows
This PR hardens `FileBaseUtils#deleteFileOrDirectory` when the method is
called concurrently on Windows.
`AccessDeniedException`s can be thrown when attempting to delete a file
while a deletion is in progress. In order to delete a file it has to be opened,
and as we all know you can't (easily) delete open files on Windows.
For this case we catch the exception, double-check whether the file is
deleted by now, and throw the exception if it isn't, assuming a lack of
permission prevented the deletion.
`DirectoryNotEmptyException`s can be thrown for directories for which a
delete call for every file has succeeded, since the OS may be slow in updating
the actual FileSystem.
For this case we wait for a bit (50 milliseconds, yeah it's that slow) and
check whether the directory is empty by now. If it isn't we throw the exception
assuming a concurrent creation of a new file in the given directory.
Note that neither changes are perfect; they are not guaranteed to prevent
the issue. However, it no longer categorically fails for me 🎉, but only once
every few hundred runs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 5659_fileutils_win
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3219.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3219
----
commit f372899c7cd8f5102421b986be1f1b87e32cf15f
Author: zentol <[email protected]>
Date: 2017-01-26T14:47:11Z
[FLINK-5659] Harden FileBaseUtils#deleteFileOrDirectory on WIndows
----
> FileBaseUtils#deleteFileOrDirectory not thread-safe on Windows
> --------------------------------------------------------------
>
> Key: FLINK-5659
> URL: https://issues.apache.org/jira/browse/FLINK-5659
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.2.0, 1.3.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Trivial
>
> The {code}FileBaseUtils#deleteFileOrDirectory{code} is not thread-safe on
> Windows.
> First you will run into AccessDeniedExceptions since one thread tried to
> delete a file while another thread was already doing that, for which the file
> has to be opened.
> Once you resolve those exceptions (by catching them double checking whether
> the file still exists), you run into DirectoryNotEmptyExceptions since there
> is some wacky timing/visibility issue when deleting files concurrently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
