Takuya Ueshin created SPARK-13902:
-------------------------------------
Summary: Make DAGScheduler.getAncestorShuffleDependencies() return
in topological order to ensure building ancestor stages first.
Key: SPARK-13902
URL: https://issues.apache.org/jira/browse/SPARK-13902
Project: Spark
Issue Type: Bug
Components: Scheduler
Reporter: Takuya Ueshin
{{DAGScheduler}} sometimes generate correct stage graph.
Some stages are generated for the same shuffleId twice or more and they are
referenced by the child stages because the building order of the graph is not
correct.
I added the sample RDD graph to show the illegal stage graph to
{{DAGSchedulerSuite}} and then fixed it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
