Hi, How can I override the default java.io.tmpdir and spark.local.dir in YARN. I had tried to set SPARK_YARN_USER_ENV with SPARK_JAVA_OPTS. It seems has no effect. The position is still from YarnConfiguration.DEFAULT_CONTAINER_TEMP_DIR, and it is a very small disk for me. Any suggestion?
Thanks, Jiacheng Guo
