HeartSaVioR commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local. URL: https://github.com/apache/spark/pull/23764#issuecomment-471792205 We're only dealing with temporary checkpoint here, so while I can see what @jose-torres mentioned, it is unlikely to happen in happy case and I haven't seen cache miss from running query. That could still happen in bad case like if some executors are becoming bad. Btw, may be better to document somewhere that the checkpoint should be accessible from executor if query is stateful? I guess that's only thing where non-driver reads and writes into checkpoint directory, but I might be missing something.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
