squito commented on a change in pull request #23677: [SPARK-26755][SCHEDULER] :
Optimize Spark Scheduler to dequeue speculative tasks…
URL: https://github.com/apache/spark/pull/23677#discussion_r308389710
##########
File path:
core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorSuite.scala
##########
@@ -107,7 +107,7 @@ class OutputCommitCoordinatorSuite extends SparkFunSuite
with BeforeAndAfter {
val taskSet = invoke.getArguments()(0).asInstanceOf[TaskSet]
new TaskSetManager(mockTaskScheduler, taskSet, 4) {
private var hasDequeuedSpeculatedTask = false
- override def dequeueSpeculativeTask(execId: String,
+ def dequeueSpeculativeTask(execId: String,
Review comment:
sorry I hadn't noticed this before -- this change is wrong as now this whole
setup isn't doing anything. To keep the same behavior as before, I think you
need to change `dequeueTaskHelper` to be `protected`, and then override it here.
https://github.com/squito/spark/commit/366e916f4e7499d477d6fafda538d5fe7d65d6a4
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
