venkata91 commented on issue #28287: URL: https://github.com/apache/spark/pull/28287#issuecomment-617973718
> so took a quick look and I don't follow. You send a message that says all blacklisted, but all you do is call the onSchedulerBacklogged, how does that add another executor if they are all blacklisted? that will update when it needs to add more, but it doesn't change the number if calculated. So for instance say I needed 3 executors. All of them blacklisted, updating the time to get a new one won't change that it thinks it needs 3. > If we change that to tell it to get one more, then my question is how do you do the proper accounting on that. This is why in the jira and other PR we said the allocation manager needs to be closely tied with the blacklist manager. Ok. Got it, makes sense now. Let me think more about it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
