Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/20640#discussion_r195731123
--- Diff:
resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala
---
@@ -108,6 +108,28 @@ class MesosCoarseGrainedSchedulerBackendSuite extends
SparkFunSuite
verifyTaskLaunched(driver, "o2")
}
+ test("mesos declines offers from blacklisted slave") {
+ setBackend()
+
+ // launches a task on a valid offer on slave s1
+ val minMem = backend.executorMemory(sc) + 1024
+ val minCpu = 4
+ val offer1 = Resources(minMem, minCpu)
+ offerResources(List(offer1))
+ verifyTaskLaunched(driver, "o1")
+
+ // for any reason executor(aka mesos task) failed on s1
+ val status = createTaskStatus("0", "s1", TaskState.TASK_FAILED)
+ backend.statusUpdate(driver, status)
+ when(taskScheduler.nodeBlacklist()).thenReturn(Set("hosts1"))
--- End diff --
@squito reading the code here:
https://github.com/apache/spark/pull/21068/files
is there an option to update the info about blacklisted node when there is
a mesos task failure.
It is a bit inconvenient to lose such events and wait for spark tasks to
fail which may never launch since you dont have any executors running anyway.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
