GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/6171
[SPARK-7627][SPARK-7472] DAG visualization: style skipped stages
This patch fixes two things:
**SPARK-7627.** Cached RDDs no longer light up on the job page. This is a
simple fix.
**SPARK-7472.** Display skipped stages differently from normal stages.
The latter is a major UX issue. Because we link the job viz to the stage
viz even for skipped stages, the user may inadvertently click into the stage
page of a skipped stage, which is empty.
-------------------
<img
src="https://cloud.githubusercontent.com/assets/2133137/7646940/8ad40602-fa80-11e4-9d7f-0e1e01fe9d40.png";
width="700px" />
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark dag-viz-skipped
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6171.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6171
----
commit c121fa2eec7d4e3d8da746bdf149b1cd701419d3
Author: Andrew Or <[email protected]>
Date: 2015-05-14T04:36:03Z
Fix cache color
commit 7cc34ce98154000d3e2db436bc75eec84bf8170f
Author: Andrew Or <[email protected]>
Date: 2015-05-15T01:55:05Z
Merge branch 'master' of github.com:apache/spark into dag-viz-skipped
commit 7c4c36408a23cfa17729b1d5fb674875e86f9aa2
Author: Andrew Or <[email protected]>
Date: 2015-05-15T02:56:53Z
Show skipped stages differently
This requires storing more state in the listener and making sure
we clean them up later. Tests are coming in the next commit.
commit b928cd4b0602d413b60e989832288436b80d802a
Author: Andrew Or <[email protected]>
Date: 2015-05-15T04:21:44Z
Fix potential leak + write tests for it
Previously there was an opportunity to leak in `completedStageIds`.
This commit fixes that and includes tests that ensures that it is
fixed.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://cloud.githubusercontent.com/assets/2133137/7646940/8ad40602-fa80-11e4-9d7f-0e1e01fe9d40.png"](https://cloud.githubusercontent.com/assets/2133137/7646940/8ad40602-fa80-11e4-9d7f-0e1e01fe9d40.png"){kind=link}