Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136785979
I doubt this works with dynamic allocation under Mesos. It also completely
shuns the previous configuration parameters. Here's what docs say:
> spark.local.dir Directory to use for "scratch" space in Spark,
including map output files and RDDs that get stored on disk. This should be on
a fast, local disk in your system. It can also be a comma-separated list of
multiple directories on different disks. NOTE: In Spark 1.0 and later this will
be overriden by SPARK_LOCAL_DIRS (Standalone, Mesos) or LOCAL_DIRS (YARN)
environment variables set by the cluster manager.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
