yaooqinn commented on pull request #34735:
URL: https://github.com/apache/spark/pull/34735#issuecomment-986503970
> Are we trying to preserve skipped stages when they are retried? If yes,
why ?
Yes. When the skipped stages are retried,
1. the skipped info on UI get lost
2. the retry info the first retry for those stages get lost
3. the total tasks of those stages get lost when partially retry
4. If it gets retried more than once, the total tasks/inputs/outputs of
each attempt may be inconsistent, for example, you have a skipped stage with
200 tasks indeed, but you won't see this. In fact, you might see `Stage X -
Tasks: (Succeeded/Total) (3/3)` and `Stage X (retry 1) - Tasks:
(Succeeded/Total) (15/15)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
