Ngone51 commented on issue #23377: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/23377#issuecomment-450374072 Hi, @felixcheung , thanks for your comment. Hmmm....I'm not pretty sure what you mean here, so, just given my understanding here. As far as I know, schedulerbackend is not responsible for resource management. It just request executors to cluster manager, but do not know clearly about the available resources. And for Barrier TaskSet, if app's maximum config resources is less than Barrier TaskSet's demand, we'll fail the related stage, or even job at the end. You can see `DAGScheduler.checkBarrierStageWithNumSlots` for details.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
