jerryshao commented on a change in pull request #24848: [SPARK-28014][core] All
waiting apps will be changed to the wrong state of Running after master changed.
URL: https://github.com/apache/spark/pull/24848#discussion_r295161118
##########
File path: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
##########
@@ -561,7 +561,7 @@ private[deploy] class Master(
apps.filter(_.state == ApplicationState.UNKNOWN).foreach(finishApplication)
// Update the state of recovered apps to RUNNING
- apps.filter(_.state == ApplicationState.WAITING).foreach(_.state =
ApplicationState.RUNNING)
+ apps.filter(_.coresGranted > 0).foreach(_.state = ApplicationState.RUNNING)
Review comment:
From the code, it looks like only when executor is registered, then the
state will change to `RUNNING` in master UI (the code there is quite old). But
I think it is not valid to use registered executor numbers to judge the state
of application:
1. When dynamic allocation is enabled, executors can be 0, we cannot say
this application is waiting because it will ramp up soon.
2. Some driver only tasks can be run without executors, we cannot say this
application is waiting.
So as a thorough fix, I would suggest to revisit all the changes related to
application state, to see whether it is meaningful as for now, rather than just
a point fix here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
