[
https://issues.apache.org/jira/browse/SPARK-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Davidson resolved SPARK-2324.
-----------------------------------
Resolution: Fixed
Resolved by https://github.com/apache/spark/pull/1274
> SparkContext should not exit directly when spark.local.dir is a list of
> multiple paths and one of them has error
> ----------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-2324
> URL: https://issues.apache.org/jira/browse/SPARK-2324
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: YanTang Zhai
>
> The spark.local.dir is configured as a list of multiple paths as follows
> /data1/sparkenv/local,/data2/sparkenv/local. If the disk data2 of the driver
> node has error, the application will exit since DiskBlockManager exits
> directly at createLocalDirs. If the disk data2 of the worker node has error,
> the executor will exit either.
> DiskBlockManager should not exit directly at createLocalDirs if one of
> spark.local.dir has error. Since spark.local.dir has multiple paths, a
> problem should not affect the overall situation.
> I think DiskBlockManager could ignore the bad directory at createLocalDirs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
