[
https://issues.apache.org/jira/browse/YUNIKORN-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YUNIKORN-1665:
-----------------------------------
Description:
In core, we track a bunch of things inside a queue. Sometimes it's tracked
automatically with the parent, sometimes not:
||Tracked item||Type||Updated in the parent||
|Applications|map[string]*Application|no|
|Allocation|*resource.Resource|yes|
|Reserved Apps|map[string]int|no|
|Pending resource|*resource.Resource|yes|
|Preempted resource|*resource.Resource|yes|
|Reserved apps|map[string]int|no|
|Running apps|uint64|yes|
|AllocatingAcceptedApps|map[string]bool|yes|
When we move an application to one leaf to another, all of these data
structures need to be updated.
For example, if an application is moved from "root.users.qa.alice" to
"root.dev", then its "pending" resources tracked in "root.users.qa.alice" must
be subtracted from the two parents + leaf except root and it need to be added
to "root.dev".
Whenever we add something new to track, it's easy to forget that it affects the
app moving logic. So we can do two things:
1) We re-organize the current source code a bit better to make it clear that
code in "queue_moveapp.go" needs to be updated if we add something new to track.
2) We extract everything to a separate Tracker type and this becomes even more
obvious. The Tracker can be similar to {{ugm.QueueTracker}}, where the trackers
are built in a hierarchy which accurately reflects the queue hierarchy, ie. all
trackers have child trackers.
was:
In core, we track a bunch of things inside a queue. Sometimes it's tracked
automatically with the parent, sometimes not:
||Tracked item||Type||Updated in the parent||
|Applications|map[string]*Application|no|
|Allocation|*resource.Resource|yes|
|Reserved Apps|map[string]int|no|
|Pending resource|*resource.Resource|yes|
|Preempted resource|*resource.Resource|yes|
|Reserved apps|map[string]int|no|
|Running apps|uint64|yes|
|AllocatingAcceptedApps|map[string]bool|yes|
When we move an application to one leaf to another, all of these data
structures need to be updated.
For example, if an application is moved from "root.users.qa.alice" to
"root.dev", then its "pending" resources tracked in "root.users.qa.alice" must
be subtracted from the two parents + leaf except root and it need to be added
to "root.dev".
Whenever we add something new to track, it's easy to forget that it affects the
app moving logic. So we can do two things:
1) We re-organize the current source code a bit better to make it clear that
code in "queue_moveapp.go" needs to be updated if we add something new to track.
2) We extract everything to a separate Tracker type and this becomes even more
obvious. The Tracker can be similar to the {{QueueTracker}}, where the trackers
are built in a hierarchy which accurately reflects the queue hierarchy, ie. all
trackers have child trackers.
> Group tracked resources/counters to a separate type from Queue
> --------------------------------------------------------------
>
> Key: YUNIKORN-1665
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1665
> Project: Apache YuniKorn
> Issue Type: Sub-task
> Components: core - scheduler
> Reporter: Peter Bacsko
> Priority: Major
>
> In core, we track a bunch of things inside a queue. Sometimes it's tracked
> automatically with the parent, sometimes not:
>
> ||Tracked item||Type||Updated in the parent||
> |Applications|map[string]*Application|no|
> |Allocation|*resource.Resource|yes|
> |Reserved Apps|map[string]int|no|
> |Pending resource|*resource.Resource|yes|
> |Preempted resource|*resource.Resource|yes|
> |Reserved apps|map[string]int|no|
> |Running apps|uint64|yes|
> |AllocatingAcceptedApps|map[string]bool|yes|
> When we move an application to one leaf to another, all of these data
> structures need to be updated.
> For example, if an application is moved from "root.users.qa.alice" to
> "root.dev", then its "pending" resources tracked in "root.users.qa.alice"
> must be subtracted from the two parents + leaf except root and it need to be
> added to "root.dev".
> Whenever we add something new to track, it's easy to forget that it affects
> the app moving logic. So we can do two things:
> 1) We re-organize the current source code a bit better to make it clear that
> code in "queue_moveapp.go" needs to be updated if we add something new to
> track.
> 2) We extract everything to a separate Tracker type and this becomes even
> more obvious. The Tracker can be similar to {{ugm.QueueTracker}}, where the
> trackers are built in a hierarchy which accurately reflects the queue
> hierarchy, ie. all trackers have child trackers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
