Imran Rashid created SPARK-10193:
------------------------------------
Summary: Eliminate "skipped" stages for shared shuffle
dependencies by reusing associated Stage
Key: SPARK-10193
URL: https://issues.apache.org/jira/browse/SPARK-10193
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.5.0
Reporter: Imran Rashid
Assignee: Imran Rashid
While looking into writing some tests for skipped stages (SPARK-10192), I
realized that we might be able to eliminate skipped stages completely. This
would be great, since they are a source of confusion for end users, and are
also confusing even for developers and lead to bugs like SPARK-9809.
I'm not totally sure yet this will even be possible -- perhaps there will be
some unintended consequences I haven't thought of yet, but I will give it a
shot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
