GitHub user jsoltren opened a pull request:
https://github.com/apache/spark/pull/16346
[SPARK-16654][CORE] Add UI coverage for Application Level Blacklisting
Builds on top of work in SPARK-8425 to update Application Level
Blacklisting in the scheduler.
## What changes were proposed in this pull request?
Adds a UI to these patches by:
- defining new listener events for blacklisting and unblacklisting, nodes
and executors;
- sending said events at the relevant points in BlacklistTracker;
- adding JSON (de)serialization code for these events;
- augmenting the Executors UI page to show which, and how many, executors
are blacklisted;
- adding a unit test to make sure events are being fired;
- adding HistoryServerSuite coverage to verify that the SHS reads these
events correctly.
- updates the Executor UI to show Blacklisted/Active/Dead as a tri-state in
Executors Status
Updates .rat-excludes to pass tests.
@username squito
(Please fill in changes proposed in this fix)
## How was this patch tested?
./dev/run-tests
testOnly org.apache.spark.util.JsonProtocolSuite
testOnly org.apache.spark.scheduler.BlacklistTrackerSuite
testOnly org.apache.spark.deploy.history.HistoryServerSuite
https://github.com/jsoltren/jose-utils/blob/master/blacklist/test-blacklist.sh
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jsoltren/spark SPARK-16654-submit
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16346.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16346
----
commit 20ff7dddea72bf8fc9330f464992b19e1bf1c59e
Author: José Hiram Soltren <[email protected]>
Date: 2016-10-14T21:09:44Z
[SPARK-16654][CORE] Add UI coverage for Application Level Blacklisting
Builds on top of work in SPARK-8425 to update
Application Level Blacklisting in the scheduler.
Adds a UI to these patches by:
- defining new listener events for blacklisting and unblacklisting,
nodes and executors;
- sending said events at the relevant points in BlacklistTracker;
- adding JSON (de)serialization code for these events;
- augmenting the Executors UI page to show which, and how many,
executors are blacklisted;
- adding a unit test to make sure events are being fired;
- adding HistoryServerSuite coverage to verify that the SHS reads
these events correctly.
- updates the Executor UI to show Blacklisted/Active/Dead
as a tri-state in Executors Status
Updates .rat-excludes to pass tests.
@username squito
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
