[
https://issues.apache.org/jira/browse/SPARK-4323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen resolved SPARK-4323.
-------------------------------
Resolution: Not a Problem
Resolving this as "Not a Problem" since the pull request was closed after some
discussion.
> Utils#fetchFile method should close lock file certainly
> -------------------------------------------------------
>
> Key: SPARK-4323
> URL: https://issues.apache.org/jira/browse/SPARK-4323
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.3.0
> Reporter: Kousuke Saruta
>
> In Utils#fetchFile method, lock file is created like as follows.
> {code}
> val raf = new RandomAccessFile(lockFile, "rw")
>
>
> // Only one executor entry.
>
>
> // The FileLock is only used to control synchronization for executors
> download file,
>
> // it's always safe regardless of lock type (mandatory or advisory).
>
>
> val lock = raf.getChannel().lock()
>
>
> val cachedFile = new File(localDir, cachedFileName)
>
>
> try {
>
>
> if (!cachedFile.exists()) {
>
>
> doFetchFile(url, localDir, cachedFileName, conf, securityMgr,
> hadoopConf)
>
> }
>
>
> } finally {
>
>
> lock.release()
>
>
> }
> {code}
> If some error occurs between opening RandomAccessFile and getting lock, lock
> file can be not closed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
