Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/9051#issuecomment-148842285
It seems that there are two separate issues here: (1) the stage duration is
misleading, and (2) there's no way to see from the "All Stages Page" which
stages took longer than others. So can't we just fix the stage duration
semantics for (1) and add that as a column in the "All Stages Page" to fix (2)?
I still don't fully see why we need to introduce a completely new metric (e.g.
max task time). Is it only to avoid changing the semantics of the existing
stage duration?
> It seems like as long as its labelled properly it shouldn't be more
confusing.
My concern is not so much that it will confuse the user as it will make the
UI less user friendly. If we keep the existing stage `Duration` and add this
`Max task time`, then we might end up with two disagreeing the user has to
reconcile. We can add tooltips but these don't make the UI any less cluttered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
