httfighter commented on issue #23437: [SPARK-26524] If the application directory fails to be created on the SPARK_WORKER_… URL: https://github.com/apache/spark/pull/23437#issuecomment-453896511 @srowen There are two main considerations. First, considering that as long as the user fixes the worker directory, you do not need to restart the worker process, you can continue to use the resources on the worker, which can simplify user operations and make full use of computing resources. Second, before the worker directory is damaged, There are already applications running on the worker, stopping the worker will affect the running applications. For applications that are already running, damage to the worker directory only affects the writing of the log and does not affect the computational execution of the application. Therefore, I didn't let the worker quit directly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
