[
https://issues.apache.org/jira/browse/SPARK-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954304#comment-14954304
]
Marcelo Vanzin commented on SPARK-11074:
----------------------------------------
The shuffle files are created under an application-specific root directory that
is created by the cluster manager. That directory belongs to the application,
and Spark is free to do whatever it wants with it. No other application depends
on it, unlike the event log directory.
> Attempt to automatically create event log directory if it doesn't already
> exist
> -------------------------------------------------------------------------------
>
> Key: SPARK-11074
> URL: https://issues.apache.org/jira/browse/SPARK-11074
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.5.1
> Reporter: Kay Ousterhout
> Assignee: Kay Ousterhout
> Priority: Minor
>
> Right now, if the log where event logs are to be placed (spark.eventLog.dir)
> doesn't already exist, the SparkContext fails to start. We should try to
> automatically create this directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
