Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1384#issuecomment-49111190
@tsudukim @kayousterhout so I think in general here, our handling of stage
re-submissions is broken in the UI. For instance, I looked in the
`JobProgressListener` and we index many things on `StageId` that might better
be indexed on `StageId, AttemptId`. Also, we should probably give the
`AttemptId` when start a task so that we understand which stage attempt to
associated it with. I also don't understand exactly what happens when a stage
gets re-attempted, do we send a "stage completed" event? It might be good to
fix the way we deal with stage re-submissions to make this work better overall.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
