attilapiros commented on pull request #31513: URL: https://github.com/apache/spark/pull/31513#issuecomment-781427702
@holdenk Even if they do not reach any known state they will be taken care by the `ExecutorPodsLifecycleManager`: https://github.com/apache/spark/blob/a316243c7b7c01b0cd95136b236daf30dd65becf/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala#L118-L147 I think `ExecutorPodsAllocator` responsibility are: 1) allocating more resources for scaling up 2) mitigating the risk of losing something useful during downscaling: by killing prefereby newly created or (if there is not enough newly created then the second best) pending requests. This PR improves on downscaling. As by checking the scheduler backend we have some more information than checking only the POD snapshot source. With this extra information those pending pods known by the scheduler backend **here** are treated equally with the running pods (as they might run a task). Back to the eviction: so from `ExecutorPodsLifecycleManager` we know what will happen with the newly created request not reaching even the pending state. Let's see what will happen with a pending POD which never reaches the running state but known by the scheduler backend: they will die in the same way as any running POD. Either `spark.dynamicAllocation.executorIdleTimeout` is elapsed and the driver kills it (in case of DA) or finishes unexpectedly or if we are lucky it reaches the end of the application. Those cases cannot be tested here. But I agree with you we need more tests: even for `onNewSnapshots` as it is getting more complex. I will look into this. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
