GitHub user sarutak opened a pull request:
https://github.com/apache/spark/pull/3186
[SPARK-4323] Utils#fetchFile method should close lock file certainly
In Utils#fetchFile method, lock file is created like as follows.
val raf = new RandomAccessFile(lockFile, "rw")
// Only one executor entry.
// The FileLock is only used to control synchronization for
executors download file,
// it's always safe regardless of lock type (mandatory or
advisory).
val lock = raf.getChannel().lock()
val cachedFile = new File(localDir, cachedFileName)
try {
if (!cachedFile.exists()) {
doFetchFile(url, localDir, cachedFileName, conf, securityMgr,
hadoopConf)
}
} finally {
lock.release()
}
If some error occurs between opening RandomAccessFile and getting lock,
lock file can be not closed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sarutak/spark SPARK-4323
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3186.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3186
----
commit a2b30bc8afbc20a91ca7b145029168e0e02a5835
Author: Kousuke Saruta <[email protected]>
Date: 2014-11-10T19:33:48Z
Fixed Utils#fetchFile to close lock file certainly
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
