AngersZhuuuu commented on issue #26643: [SPARK-29998][CORE] Retry getFile() until all folder failed then exit URL: https://github.com/apache/spark/pull/26643#issuecomment-557847261 > Hi, @AngersZhuuuu . > This is possible, but I'm not sure it's a good idea to live with the node. > We already handle executor failure in the upper layers, don't we? I know we will handle executor, but when this situation happened. Executor won't failed. By current` getFile`'s logical. This task will always retry on this executor's same localdir and subdir. Since the hash value is same. And will failed all task retries. Finally job failed. And this kind problem won't make stage retry.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
