[
https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592307#comment-16592307
]
Zian Chen commented on YARN-8509:
---------------------------------
Hi [~eepayne], sorry for getting back late. I took some time to run SLS test in
order to evaluate if this change introduced unnecessary preemption, however,
there is no suitable dataset which can be used to test preemption behavior
since almost all the dataset submits applications at the same time, which let
scheduler considered all the resource request in allocation stage. This leaves
no chance for preemption to come into play. I'll work on generate a dataset for
preemption SLS test offline and may take several weeks. I'll comment on the
updates once I have some progress.
> Total pending resource calculation in preemption should use user-limit factor
> instead of minimum-user-limit-percent
> -------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-8509
> URL: https://issues.apache.org/jira/browse/YARN-8509
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Zian Chen
> Assignee: Zian Chen
> Priority: Major
> Labels: capacityscheduler
> Attachments: YARN-8509.001.patch, YARN-8509.002.patch,
> YARN-8509.003.patch, YARN-8509.004.patch, YARN-8509.005.patch
>
>
> In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total
> pending resource based on user-limit percent and user-limit factor which will
> cap pending resource for each user to the minimum of user-limit pending and
> actual pending. This will prevent queue from taking more pending resource to
> achieve queue balance after all queue satisfied with its ideal allocation.
>
> We need to change the logic to let queue pending can go beyond userlimit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
