Github user witgo commented on the issue:
https://github.com/apache/spark/pull/17480
The ExecutorAllocationManager.reset method is called when re-registering
AM, which sets the ExecutorAllocationManager.initializing field true. When this
field is true, the Driver does not start a new executor from the AM request.
The following two cases will cause the field to False
1. executor idle for some time.
2. There are new stages to be submitted
If the stage after the submission, AM was killed and restart, the above two
cases will not appear.
1. When AM is killed, the yarn will kill all running containers. All
execuotr will be lost and no executor will be idle.
2. No surviving executor, resulting in the current stage will never be
completed, DAG will not submit a new stage.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
