This is an automated email from the ASF dual-hosted git repository.
nicholasjiang pushed a commit to branch branch-0.5
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/branch-0.5 by this push:
new 754934509 [CELEBORN-1475] Fix unknownExcludedWorkers filter for
/exclude request
754934509 is described below
commit 75493450949fbd75fdac4cce37b03025dc5e58c6
Author: Fei Wang <[email protected]>
AuthorDate: Sun Jun 23 03:43:18 2024 +0800
[CELEBORN-1475] Fix unknownExcludedWorkers filter for /exclude request
### What changes were proposed in this pull request?
Now for unknown excluded workers filter,
```
val unknownExcludedWorkers =
(workersToAdd ++
workersToRemove).filter(!statusSystem.workers.contains(_))
```
The `workersToAdd` and `workersToRemove` are in `Array[String]` type, and
`statusSystem.workers` is in `Set<WorkerInfo>` type.
In this pr, `workersToAdd` and `workersToRemove` is in `List[WorkerInfo]`
type.
### Why are the changes needed?
As title.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Closes #2586 from turboFei/fix_work_filter.
Authored-by: Fei Wang <[email protected]>
Signed-off-by: SteNicholas <[email protected]>
(cherry picked from commit 5dfbfd28406557036e2ba640ce1b2bab532f4be7)
Signed-off-by: SteNicholas <[email protected]>
---
.../scala/org/apache/celeborn/service/deploy/master/Master.scala | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git
a/master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala
b/master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala
index 6c74606fc..cfbb98fc2 100644
---
a/master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala
+++
b/master/src/main/scala/org/apache/celeborn/service/deploy/master/Master.scala
@@ -1263,11 +1263,12 @@ private[celeborn] class Master(
override def exclude(addWorkers: String, removeWorkers: String): String = {
val sb = new StringBuilder
sb.append("============================ Add/Remove Excluded Workers
Manually =============================\n")
- val workersToAdd = addWorkers.split(",").filter(_.nonEmpty)
- val workersToRemove = removeWorkers.split(",").filter(_.nonEmpty)
+ val workersToAdd =
addWorkers.split(",").filter(_.nonEmpty).map(WorkerInfo.fromUniqueId).toList
+ val workersToRemove =
+
removeWorkers.split(",").filter(_.nonEmpty).map(WorkerInfo.fromUniqueId).toList
val workerExcludeResponse =
self.askSync[PbWorkerExcludeResponse](WorkerExclude(
- workersToAdd.map(WorkerInfo.fromUniqueId).toList.asJava,
- workersToRemove.map(WorkerInfo.fromUniqueId).toList.asJava,
+ workersToAdd.asJava,
+ workersToRemove.asJava,
MasterClient.genRequestId()))
if (workerExcludeResponse.getSuccess) {
sb.append(
