GitHub user kayousterhout opened a pull request:
https://github.com/apache/spark/pull/13646
[SPARK-15927] Eliminate redundant DAGScheduler code.
## What changes were proposed in this pull request?
To try to eliminate redundant code to traverse the RDD dependency graph,
this PR creates a new function getShuffleDependencies that returns shuffle
dependencies that are immediate parents of a given RDD. This new function is
used by getParentStages and getAncestorShuffleDependencies.
cc @squito @markhamstra
FYI @rxin
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kayousterhout/spark-1 SPARK-15927
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13646.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13646
----
commit 5078bb2037201cbcd51223d2acf971a7a7fcc0cb
Author: Kay Ousterhout <[email protected]>
Date: 2016-06-10T23:12:02Z
[SPARK-15927] Eliminate redundant DAGScheduler code.
commit 42a8d16ed0b7e8175a58d1d6fa21685cc36c85c2
Author: Kay Ousterhout <[email protected]>
Date: 2016-06-13T19:43:32Z
Improved method comment
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
