HeartSaVioR edited a comment on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local. URL: https://github.com/apache/spark/pull/23764#issuecomment-470441915 My bad. Now I realized the temporary directory YARN provides for container is local filesystem, not HDFS. If my understanding is right, then `creating the path in HDFS as it is` sounds me as a bug and the patch fixes it. LGTM on the patch. > If the suggestion is to create a temp dir on default FS (HDFS for instance) then a mechanism should be suggested to find a proper temp directory which I'm not aware of. Same understanding as of now.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
