craigcondit commented on PR #567: URL: https://github.com/apache/yunikorn-core/pull/567#issuecomment-1591372317
I have some concerns about this. Queue sorting has never really showed up as a hot spot for several reasons. For one thing, there are generally far fewer queues than outstanding requests (if not, the cluster is relatively idle and performance isn't an issue). Sorting of queues also happens only once per scheduling cycle, and if we are able to schedule a task, the sorting needs to be re-done anyway, so the only thing caching the sorted queues gets us is a slight speedup when there is nothing available to schedule anyway. Some of the changes needed here to make this work (including potentially a new FSM for Queue?) strike me as a bridge too far, given the likely negligible impact on performance. Additionally, changes this invasive are going to make implementing additional features more difficult -- things like multiple partition support come to mind. In short, I think we should table this JIRA. If at some point in the future this becomes a bottleneck, we can readdress it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
