bmarcott commented on issue #28188: [SPARK-18886][CORE][FOLLOWUP] remove requirement to launch a task to reset locality wait timer URL: https://github.com/apache/spark/pull/28188#issuecomment-613175971 I believe this is the reason we added it. This is your comment from other PR: @tgravescs > One thing I don't think I like is that if you are fully scheduled, we keep trying to schedule "all resources" but if there are no resources, then we continue to reset the timer. This means that it takes a long time to fall back in case where you may have multiple tasksets and the first task set rejects it and the second one takes it and the tasks are finishing such that you get an all resources offer in between the task finishes. In this scenario the first taskset can get starved. We would need to perhaps track this separately. I will also spend some more time thinking on this. @cloud-fan Do we need to revert the other commit and later land this all as one commit?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
