GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/1102
[SPARK-2147 / 2161] Show removed executors on the UI
This PR includes two changes
- When an application finishes cleanly (i.e. `sc.stop()` is called), all of
its executors disappear from the Master UI. It might be useful to keep them
around so the user can click on the executor logs even after the application
finishes.
- This adds a "Removed Executors" table to Master UI, so the user can find
out why their executors died, for instance. The equivalent table already exists
in the Worker UI, but was hidden because of a bug. (The bug is that the comment
`//scalastyle:off` disconnected the `Seq[Node]` that represents the HTML for
table)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark remember-removed-executors
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1102.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1102
----
commit fe47402c83232f5db64ddfb496a542f3efe145ad
Author: Andrew Or <[email protected]>
Date: 2014-06-17T01:26:56Z
Show exited executors on the Master UI
Previously executors disappear if the application exits cleanly
(i.e. if sc.stop() is called).
commit c89bb6e90af3ceb9e1332b76fd416e0b543c2a2e
Author: Andrew Or <[email protected]>
Date: 2014-06-17T02:20:46Z
Add table for removed executors in MasterWebUI
commit fbb65b8df0165b84e1a42a7191139eeb8cf1a14c
Author: Andrew Or <[email protected]>
Date: 2014-06-17T02:24:46Z
Removed unused method
commit 161f8a2a574f89a49b90ee2eb0a12b86a489a0a8
Author: Andrew Or <[email protected]>
Date: 2014-06-17T02:59:46Z
Display finished executors table (fix bug)
The table was previously hidden by a comment, which disconnected the
HTML from the rest of the content. Yay Scala DSL.
commit 3390b49b2ac4c810ff5f2d58f7ed228b156acd25
Author: Andrew Or <[email protected]>
Date: 2014-06-17T03:04:15Z
Add executor state column to WorkerPage
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
