sameeragarwal commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-450444016 From a pure feature perspective, I think it makes sense to support this for non-yarn modes as well. While Facebook uses a custom scheduler, we rely on this leveldb state to be robust against external shuffle service failures/restarts (for certain deployment types). Although admittedly, I lack full context behind as to why this was only done for Yarn to begin with. cc @dafrista @squito who might've more details there.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
