zentol opened a new pull request #14469: URL: https://github.com/apache/flink/pull/14469
When a slot was freed the slot pool was trying to find another requirement it could potentially fulfill, using the same code paths as for newly offered slots, which also includes an increase in acquired resources. As a result we were potentially counting a single slot as multiple resources, causing an inconsistent state. The slot pool hence believed it had enough slots to fulfill the requirements, causing it to reject slots, while the Scheduler was waiting for enough slots to arrive and the RM repeatedly trying to provide the actually required slots. While it is not necessarily a bad thing to try an find another matching requirement, it is currently not necessary to do so, since the scheduler will use slots however it sees fit anyway. In the worst case we go through another slot allocation cycle. As such freed slots now remain assigned to the originally matched requirement, and this mapping will only change when the slot is explicitly reserved for a different requirement. In the long-term it would be desirable for the slot pool to automatically adjust the matching if a slot was freed. This could prevent cases where we request new slots although we still have idling slots around that we could use, or release slots as idle although there are still outstanding requirements. However this needs a more sophisticated approach, it has to account for cases where: - a freed slot could not be matched to another requirement (i.e., it is truly an excess resource) - a freed slot was matched against another profile, which should go through the same codepath as for slot reservations for a different profile - also try to re-assign currently excess resources if the requirements are increased ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
