GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/20955
[SPARK-23838][WEBUI] Running SQL query is displayed as "completed" in SQL
tab
## What changes were proposed in this pull request?
A running SQL query would appear as completed in the Spark UI:
We can see the query in "Completed queries", while in in the job page we
see it's still running Job 132.
After some time in the query still appears in "Completed queries" (while
it's still running), but the "Duration" gets increased.
To reproduce, we can run a query with multiple jobs. E.g. Run TPCDS q6.
The reason is that updates from executions are written into kvstore
periodically, and the job start event may be missing.
## How was this patch tested?
Manually run the job again and check the SQL Tab. The fix is pretty simple.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark jobCompleted
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20955.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20955
----
commit b418feb6ac4bb7523c50ea27f74d3b02c310956a
Author: Wang Gengliang <gengliang.wang@...>
Date: 2018-04-01T07:08:06Z
Fix sql tab
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
