Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/9051#issuecomment-147498589
So one thing I would I want from this is if I have 100's of stages I want
to be able to quickly (I don't want to click on each individual stage) look and
see which ones "took the longest" to see if there were problems or
optimizations that could be made for those stages. "took the longest" can
obviously have different meanings and different things could be useful.
I don't want the wait time but the active time might not be what I want
either as if its only running a few tasks out of thousands at a time the
"active time" might be huge when each task only took very little time. That
is why we were talking about the max task time because it could be an indicator
that a certain node or task was having issues. I think the "active time" is
better then duration and can tell you certain things but I don't think it tells
me what I'm looking for here.
How is the max task time here misleading? Its basically the same thing you
get if you click on the stage and can see the min/25th/med/75th/max, just in an
easier to view across stages.
Note I don't think we should redefine duration since its more a backwards
compatibility thing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
