Takuya Ueshin created SPARK-14269: ------------------------------------- Summary: Eliminate unnecessary submitStage() call. Key: SPARK-14269 URL: https://issues.apache.org/jira/browse/SPARK-14269 Project: Spark Issue Type: Improvement Components: Scheduler Reporter: Takuya Ueshin
Currently a method {{submitStage()}} for waiting stages is called on every iteration of the event loop in {{DAGScheduler}} to submit all waiting stages, but most of them are not necessary because they are not related to Stage status. The case we should try to submit waiting stages is only when their parent stages are successfully completed. This elimination can improve {{DAGScheduler}} performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org