Hi I am trying to change the spark.local.dir property. I am running spark on yarn and have already tried the following properties:
export LOCAL_DIRS=<my-tmp> spark.yarn.appMasterEnv.LOCAL_DIRS=<my-tmp> spark.yarn.appMasterEnv.SPARK_LOCAL_DIRS=<my-tmp> spark.yarn.nodemanager.local-dirs=/<my-tmp> spark.local.dir=<my-tmp> But still it does not change the executors local directory. Is there a way to change this value without changing yarn-site.xml ? Thanks in advance ! --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
