clownxc commented on PR #40707: URL: https://github.com/apache/spark/pull/40707#issuecomment-1513933182
> OK, I'm on the fence now. On one hand, the number of transient errors should be much smaller than the number of user-triggered errors, so it's better to find out these transient errors and mark them. On the other hand, not retrying the task can be a regression that leads to job failure, so we should make sure we only skip task retry when the error is definitely user-triggered. > > To be conservative, now I'm leaning towards picking some errors and marking them as "can skip task retry". I like the idea from @aokolnychyi that we can add a JSON field for it. I'm trying to change the code now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
