GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/19689
[SPARK-22462][SQL] Make rdd-based actions in Dataset trackable in SQL UI
## What changes were proposed in this pull request?
For the few Dataset actions such as `foreach`, currently no SQL metrics are
visible in the SQL tab of SparkUI. It is because it binds wrongly to Dataset's
`QueryExecution`. As the actions directly evaluate on the RDD which has
individual `QueryExecution`, to show correct SQL metrics on UI, we should bind
to RDD's `QueryExecution`.
## How was this patch tested?
Manually test. Screenshot is attached in the PR.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-22462
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19689.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19689
----
commit ac539cd0e761193d9a665d8ccb19a8fba5dd504b
Author: Liang-Chi Hsieh <[email protected]>
Date: 2017-11-07T10:54:14Z
Make rdd-based actions trackable in UI.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
