[
https://issues.apache.org/jira/browse/YUNIKORN-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
weichen lai reassigned YUNIKORN-3413:
-------------------------------------
Assignee: weichen lai
> Wildcard limit config read without the manager lock on scheduling paths
> -----------------------------------------------------------------------
>
> Key: YUNIKORN-3413
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3413
> Project: Apache YuniKorn
> Issue Type: Sub-task
> Components: core - scheduler
> Reporter: Dale Richardson
> Assignee: weichen lai
> Priority: Major
>
> {{Manager.getUserWildCardLimitsConfig}} reads {{m.userWildCardLimitsConfig}}
> with no lock. It is reached from {{newQueueTracker}} through the
> package-global manager, which means from {{IncreaseTrackedResource}},
> {{Headroom}} and {{CanRunApp}} on the scheduling paths, holding only a user
> or group tracker lock. {{replaceLimitConfigs}} and {{ClearConfigLimits}}
> replace the whole map under the manager write lock on every config reload.
> Taking the manager lock in place does not work: {{setUserLimits}} reaches the
> same constructor while already holding {{m.Lock()}}, and the tracker-locked
> callers would take the manager lock inside a tracker lock, the wrong way
> round. Confirmed with the race detector on exactly that chain (test on the
> fork fix branch). Beyond the race, each child tracker re-reads the global
> during one hierarchy walk, so a reload landing mid-walk builds
> {{root.parent}} with the old wildcard limit and {{root.parent.child}} with
> the new one.
> Fix: resolve the wildcard config once at each operation's entry and pass it
> through tracker construction. Manager-locked callers read the field directly;
> tracker-locked callers take one {{RLock}} snapshot at entry; {{UpdateConfig}}
> passes the map it is applying. The map is only ever replaced wholesale, never
> written element by element, so the reference is a valid snapshot. A fix
> exists on the tigerquoll fork ({{fix/ugm-wildcard-config-threading}}, PR #16
> there) and will be filed once this has a number.
> Marker: {{getUserWildCardLimitsConfig}} in {{manager.go}} carries this JIRA;
> the fix removes it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
