tgravescs commented on issue #28188: [SPARK-18886][CORE][FOLLOWUP] allow follow up locality resets even if no task was launched URL: https://github.com/apache/spark/pull/28188#issuecomment-613441362 I don't think we need to revert the original, this is 3.1 which won't go out for a while. I'd rather have us error towards allowing scheduling more then reset it to aggressively. > One thing I don't think I like is that if you are fully scheduled, we keep trying to schedule "all resources" but if there are no resources, then we continue to reset the timer. This means that it takes a long time to fall back in case where you may have multiple tasksets and the first task set rejects it and the second one takes it and the tasks are finishing such that you get an all resources offer in between the task finishes. In this scenario the first taskset can get starved. We would need to perhaps track this separately. Ah right, I think originally I was wondering if we could track this separately. Somehow have a kind of starvation flag. I'll think about this some more as well.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
