Timothy Chen created SPARK-9708:
-----------------------------------
Summary: Spark should create local temporary directories in Mesos
sandbox when launched with Mesos
Key: SPARK-9708
URL: https://issues.apache.org/jira/browse/SPARK-9708
Project: Spark
Issue Type: Bug
Components: Mesos
Reporter: Timothy Chen
Currently Spark creates temporary directories with
Utils.getConfiguredLocalDirs, and it writes to YARN directories if YARN is
detected, otherwise just writes in a temporary directory in the host.
However, Mesos does create a directory per task and ideally Spark should use
that directory to create its local temporary directories since it then can be
cleaned up when the task is gone and not left on the host or cleaned until
reboot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
