Stefano Pettini created SPARK-23777:
---------------------------------------
Summary: Missing DAG arrows between stages
Key: SPARK-23777
URL: https://issues.apache.org/jira/browse/SPARK-23777
Project: Spark
Issue Type: Bug
Components: Web UI
Affects Versions: 2.3.0, 1.3.0
Reporter: Stefano Pettini
Attachments: Screenshot-2018-3-23 RDDTestApp - Details for Job 0.png
In the Spark UI DAGs, sometimes there are missing arrows between stages. It
seems to happen when the same RDD is shuffled twice.
For example in this case:
{{val a = context.parallelize(List(10, 20, 30, 40, 50)).keyBy(_ / 10)}}
{{val b = a join a}}
{{b.collect()}}
There's a missing arrow from stage 1 to 2.
_This is an old one, since 1.3.0 at least, still reproducible in 2.3.0._
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
