Weizhong created SPARK-5663:
-------------------------------
Summary: Delete appStagingDir on local file system
Key: SPARK-5663
URL: https://issues.apache.org/jira/browse/SPARK-5663
Project: Spark
Issue Type: Improvement
Components: YARN
Reporter: Weizhong
Priority: Minor
As we know, in yarn mode Client will create appStagingDir on file system, and
AppMaster will delete this appStagingDir when it exit. If file system is HDFS
then it will work OK.
But if we don't add HADOOP_CONF_DIR to classpath, then default file system is
local file system(Use FileSystem.get(conf) to get fs). So in yarn mode Client
will create appStagingDir on local file system, and if Client and AppMaster are
not in the same node, then the appStagingDir will not be deleted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
