Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/15249#discussion_r82007301
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -809,32 +816,65 @@ class TaskSetManagerSuite extends SparkFunSuite with
LocalSparkContext with Logg
test("Kill other task attempts when one attempt belonging to the same
task succeeds") {
sc = new SparkContext("local", "test")
sched = new FakeTaskScheduler(sc, ("exec1", "host1"), ("exec2",
"host2"))
- val taskSet = FakeTask.createTaskSet(4)
+ val taskSet = FakeTask.createTaskSet(5)
// Set the speculation multiplier to be 0 so speculative tasks are
launched immediately
sc.conf.set("spark.speculation.multiplier", "0.0")
+ sc.conf.set("spark.speculation.quantile", "0.6")
val clock = new ManualClock()
val manager = new TaskSetManager(sched, taskSet, MAX_TASK_FAILURES,
clock)
val accumUpdatesByTask: Array[Seq[AccumulatorV2[_, _]]] =
taskSet.tasks.map { task =>
task.metrics.internalAccums
}
// Offer resources for 4 tasks to start
+ val tasks = new ArrayBuffer[TaskDescription]()
for ((k, v) <- List(
"exec1" -> "host1",
"exec1" -> "host1",
+ "exec1" -> "host1",
"exec2" -> "host2",
"exec2" -> "host2")) {
val taskOption = manager.resourceOffer(k, v, NO_PREF)
assert(taskOption.isDefined)
val task = taskOption.get
assert(task.executorId === k)
+ tasks += task
}
- assert(sched.startedTasks.toSet === Set(0, 1, 2, 3))
- // Complete the 3 tasks and leave 1 task in running
+ assert(sched.startedTasks.toSet === (0 until 5).toSet)
+ // Complete 3 tasks and leave 2 task in running
for (id <- Set(0, 1, 2)) {
manager.handleSuccessfulTask(id, createTaskResult(id,
accumUpdatesByTask(id)))
assert(sched.endedTasks(id) === Success)
}
+ def runningTaskForIndex(index: Int): TaskDescription = {
+ val t = tasks.find { task => task.index == index &&
!sched.endedTasks.contains(task.taskId) }
+ t match {
+ case Some(x) => x
+ case None =>
+ throw new RuntimeException(s"couldn't find index $index in " +
+ s"tasks: ${tasks.map{t => t.index -> t.taskId}} with
endedTasks:" +
+ s" ${sched.endedTasks.keys}")
+ }
+ }
+
+ // have each of the running tasks fail 3 times (not enough to abort
the stage)
+ (3 until 6).foreach { attempt =>
--- End diff --
after sleeping on this, I actually think its worth updating the original
test. It used to only check that the task was killed when there was a
lingering task for the *final* task. That's a rather special case -- it seems
like its worth adding a check for the other tasks as well. That adds a most of
the complexity here (with two speculative tasks, you need the extra
book-keeping to figure out what your'e dealing with.)
I could put it back in one test, and change the test name to include both
aspects being tested, maybe "Kill other task attempts when one attempt
belonging to the same task succeeds, and ensure killed tasks do not count
towards failing task set".
Or I could make two different tests with virtually the same setup (with
shared code) and just different asserts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
