GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/20481
[SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with the completed
timestamp before cleaning up them
## What changes were proposed in this pull request?
Sort jobs/stages/tasks/queries with the completed timestamp before cleaning
up them to make the behavior consistent with 2.2.
## How was this patch tested?
- Jenkins.
- Manually ran the following codes with `spark.ui.retainedJobs=10` and
confirmed job 0 was kept in the UI
```
new Thread() {
override def run() {
// job 0
sc.makeRDD(1 to 1, 1).foreach { i =>
Thread.sleep(10000)
}
}
}.start()
Thread.sleep(1000)
for (_ <- 1 to 20) {
new Thread() {
override def run() {
sc.makeRDD(1 to 1, 1).foreach { i =>
}
}
}.start()
}
Thread.sleep(15000)
sc.makeRDD(1 to 1, 1).foreach { i =>
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark SPARK-23307
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20481.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20481
----
commit 761f1ee1f3ef98a1c9d5f3d7e5c4ecbb71755656
Author: Shixiong Zhu <zsxwing@...>
Date: 2018-02-01T21:59:43Z
Sort jobs/stages/tasks/queries with the completed timestamp before cleaning
up them
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
