Takuya Ueshin created SPARK-13902: ------------------------------------- Summary: Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first. Key: SPARK-13902 URL: https://issues.apache.org/jira/browse/SPARK-13902 Project: Spark Issue Type: Bug Components: Scheduler Reporter: Takuya Ueshin
{{DAGScheduler}} sometimes generate correct stage graph. Some stages are generated for the same shuffleId twice or more and they are referenced by the child stages because the building order of the graph is not correct. I added the sample RDD graph to show the illegal stage graph to {{DAGSchedulerSuite}} and then fixed it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org