Hi, Currently my application jars and logs are stored in $SPARK_HOME/work, I would like to change it to somewhere having more space. Could anyone advise me on this? Changing the log dir is straightforward which just to export SPARK_LOG_DIR, however, there is no environment variable for SPARK_WORK_DIR.
Thanks a lot, -chen