Weiwei, Thanks for the feedback.
The dynamic priority is based on the fact that every pod on K8s has a priority. K8s will use that priority to decide on node evictions and scheduling preemptions. If no priority is set the default priority will be set by K8s (default == 0, can be changed cluster wide). If you want all pods in the application to behave the same then they should have the same priority. If you do not do that the scheduler, or the node, could decide that a pod of what you think belongs to a high priority application gets preempted or evicted as it has a real much lower priority. I would consider that a far bigger issue. You cannot say a spark driver has a higher priority than the executors. Preventing evictions and pushing drivers to the back of the list in preemptions Mixing priority adherence is dangerous. Example: an app's priority is 0, the pod's priority is 1000. The app gets sorted in the back of the queue. The high priority pod part of the low priority app will not get looked at. That is more confusing then changing the app priority to 1000 until that high priority pod is scheduled and then lower the app priority again. Like state changes we can track the dynamic priority change and expose it. I have dropped the same comment in the document. WIlfred On Mon, 5 Dec 2022 at 15:36, Weiwei Yang <[email protected]> wrote: > Hi Wilfred > > Thanks for putting things together, this looks really good. > I really like the concepts introduced in the proposal, such as priority > fencing and offset brilliant ideas! > I have one major doubt here, which is more related to how users can consume > this feature. I have added some comments in the doc, pls check. > The key idea is I want to make sure this is something users can easily > understand and use these concepts to achieve what they cannot do in the > past. And at the same time, they won't get confused by lots of > implications. > > Thanks > Weiwei > > On Thu, Dec 1, 2022 at 5:42 PM Wilfred Spiegelenburg <[email protected]> > wrote: > > > Hi, > > > > Priorities are a standard feature in K8s. YuniKorn has only had a very > > limited use of priorities in Daemon Set preemption and request sorting. > > Extending this into the rest of the scheduling cycle has been on the > cards > > for a long time. > > > > This design does not change the sorting of requests, i.e. AllocationAsk, > > within an application. Priority based sorting has been used since our > > earliest releases. Adding a different sorting for requests falls outside > of > > the design. > > > > Implementation will be tracked under YUNIKORN-1 > > > > Please provide feedback on the design > > > > Wilfred > > > > [1] > > > > > https://docs.google.com/document/d/1P_xlWSj65BiwAglvDvFyrnh8wZ7M72FRtIg199fyHNo/edit?usp=sharing > > >
