HeartSaVioR commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local. URL: https://github.com/apache/spark/pull/23764#issuecomment-468063737 I'm not 100% sure which is right, but if we have to force creating temp checkpoint directory in local filesystem, we may need to know available temp directory of local filesystem instead of HDFS path what YARN provides us, because I'm not sure below directory can be created in local filesystem by any account running Spark (this is picked up from tmp dir in cluster mode): ``` /yarn/nm/usercache/root/appcache/application_1549064555573_0029/container_1549064555573_0029_01_000001/tmp/ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
