GitHub user yunjzhang opened a pull request:
https://github.com/apache/spark/pull/22177
stages in wrong order within job page DAG chart
if spark job contains multiple tasks , the order in DAG chart might be
incorrect.
sample and screen snapshot can be found in jira ticket
https://issues.apache.org/jira/browse/SPARK-25119
to fix this issue, just sort RDDOperationGraph Array on "task id" before
UIUtils.showDagVizForJob()
## How was this patch tested?
packaging with all existing UT passed.
run complex query in Spark-sql console with the new jars
check the DAG chart in job page.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yunjzhang/spark SPARK-25119
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22177.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22177
----
commit f75b3fa1a09b63617d940e779a24614b82d22905
Author: Zhang <yunjzhang@...>
Date: 2018-08-21T23:06:08Z
stages in wrong order within job page DAG chart
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
