GitHub user shroffpradyumn opened a pull request:
https://github.com/apache/spark/pull/5547
[SPARK-6418] Add simple per-stage visualization to the UI [WIP]
I'm working on adding a graph-visualization to the Spark jobs page to
analyze the data visually. This is a working version, and it would be great if
I could get some feedback on what changes you'd like to see.
NOTE: This is a work in progress, and I'm currently working on adding the
following changes:
1. Changing the unit of the x-axis (from ms to s to minutes) depending on
the time taken by the longest job.
2. Reducing the tick count for the y-axis when the total # of tasks is ~50
or more.
3. Eliminate the shading for the launch time (keeping it transparent) as it
can potentially confuse users.
4. Breaking down the duration phase into shuffle read, write time, etc.
Any feedback is greatly appreciated.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shroffpradyumn/spark graph3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5547.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5547
----
commit a2141c484c51f068eb1f5c67d3f8984dd49633e0
Author: Pradyumn Shroff
<shroffprady...@airbears2-10-142-132-205.airbears2.1918.berkeley.edu>
Date: 2015-04-14T06:34:20Z
basic framework with only scheduler delays
commit a6f92e2711c8f654229ecae58d9a01df032cc4b5
Author: Pradyumn Shroff <[email protected]>
Date: 2015-04-15T05:12:43Z
working version
commit 1035f121173d9f8cd888f4e0064184cba4074cd0
Author: Pradyumn Shroff <[email protected]>
Date: 2015-04-15T06:38:06Z
good looking version (without hover)
commit a4d52079a510e02e029cdd284e305a4ab1f10ef4
Author: Pradyumn Shroff <[email protected]>
Date: 2015-04-15T09:07:34Z
final version
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
