Ngone51 commented on a change in pull request #28746:
URL: https://github.com/apache/spark/pull/28746#discussion_r436446619
##########
File path:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
##########
@@ -557,7 +557,7 @@ class CoarseGrainedSchedulerBackend(scheduler:
TaskSchedulerImpl, val rpcEnv: Rp
}
}
- override def reviveOffers(): Unit = {
+ override def reviveOffers(): Unit = Utils.tryLogNonFatalError {
Review comment:
This change fixes the failure of test
`org.apache.spark.launcher.LauncherBackendSuite.standalone/client: launcher
handle`. After sleeping one more second, the application launched by the
`SparkLauncher` now has a chance to submit tasks to TaskScheduler and call
`reviveOffers` on the SchedulerBackend. At the same time, the `SparkLauncher`
will ask the application to stop. Therefore, the SchedulerBackend could have
been already stopped when it receives `ReviveOffers` messages, which would fail
the entire application at the end.
So, I use ` Utils.tryLogNonFatalError` to fix it and I think this should be
fine since we've already use it at:
https://github.com/apache/spark/blob/c560428fe0113f17362bae2b369910049914696f/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala#L137-L139
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
