GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/19833
[SPARK-22605][SQL] SQL write job should also set Spark task output metrics
## What changes were proposed in this pull request?
For SQL write jobs, we only set metrics for the SQL listener and display
them in the SQL plan UI. We should also set metrics for Spark task output
metrics, which will be shown in spark job UI.
## How was this patch tested?
test it manually. For a simple write job
```
spark.range(1000).write.parquet("/tmp/p1")
```
now the spark job UI looks like
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark ui
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19833.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19833
----
commit e135bceaefade62ca8354f15f2900e8db7783eaa
Author: Wenchen Fan <[email protected]>
Date: 2017-11-28T14:39:30Z
SQL write job should also set Spark task output metrics
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
