GitHub user sarutak opened a pull request:
https://github.com/apache/spark/pull/2342
[WIP][SPARK-3468] WebUI Timeline-View feature
I sometimes trouble-shoot and analyse the cause of long time spending job.
At the time, I find the stages which spends long time or fails, then I find
the tasks which spends long time or fails, next I analyse the proportion of
each phase in a task.
Another case, I find executors which spends long time for running a task
and analyse the details of a task.
In such situation, I think it's helpful to visualize timeline view of
stages / tasks / executors and visualize details of proportion of activity for
each task.
Now I'm developing prototypes like captures I attached.
I'll integrate these viewer into WebUI.
[Capture of prototype for Stages timeline view]
Each brown bar represents submission time to finish time of a stage.
[Capture of prototype for Task timeline view]
Each brown bar represents launch time to finish time of a task.
[Capture of prorotype for Task details view]
This shows the proportion of activities in a task.
[Capture of prorotype for Executor timeline view]
Each brown bar represents start time to finish time of a task which a
executor executes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sarutak/spark timeline-viewer-feature
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2342.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2342
----
commit 4569a8e3ff703c17d125a6a80fff7f6baeb9e1f1
Author: Kousuke Saruta <[email protected]>
Date: 2014-09-10T05:09:35Z
Added d3.min.js
Modified .rat-exclude for d3.min.js
Modified UIUtils.scala for d3.min.js
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
