This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e2441c41de4 [SPARK-46292][CORE][UI] Show a summary of workers in
MasterPage
e2441c41de4 is described below
commit e2441c41de476b09542db60836d7d853d47f6158
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Dec 6 17:49:37 2023 -0800
[SPARK-46292][CORE][UI] Show a summary of workers in MasterPage
### What changes were proposed in this pull request?
This PR aims to show a summary of workers in MasterPage.
### Why are the changes needed?
Although `Alive Workers` is a useful information, it's insufficient to
analyze the whole cluster status because we don't know how many workers are in
other status. Especially, this is useful during the recovery process of Spark
Master HA setting.
In short, this helps the users identify the issues intuitively.
```
- Alive Workers: 1
+ Workers: 1 Alive, 1 Dead, 0 Decommissioned, 0 Unknown
```
Here is a screenshot.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual test.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #44218 from dongjoon-hyun/SPARK-46292.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git
a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
index b2f35984d37..f25e3495d79 100644
--- a/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala
@@ -144,7 +144,11 @@ private[ui] class MasterPage(parent: MasterWebUI) extends
WebUIPage("") {
</li>
}.getOrElse { Seq.empty }
}
- <li><strong>Alive Workers:</strong> {aliveWorkers.length}</li>
+ <li><strong>Workers:</strong> {aliveWorkers.length} Alive,
+ {workers.count(_.state == WorkerState.DEAD)} Dead,
+ {workers.count(_.state == WorkerState.DECOMMISSIONED)}
Decommissioned,
+ {workers.count(_.state == WorkerState.UNKNOWN)} Unknown
+ </li>
<li><strong>Cores in use:</strong>
{aliveWorkers.map(_.cores).sum} Total,
{aliveWorkers.map(_.coresUsed).sum} Used</li>
<li><strong>Memory in use:</strong>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
