[
https://issues.apache.org/jira/browse/YARN-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075232#comment-15075232
]
Sunil G commented on YARN-4304:
-------------------------------
Hi [~leftnoteasy]
Thank you very much for sharing the comments. I have few doubts in same.
bq.Could you update user am-limit (in User.resourceUsage) as well when
computing user am-limit at activateApplications?
We already doing this in activateApplications as below.
{noformat}
user.getResourceUsage().incAMUsed(partitionName,
application.getAMResource(partitionName));
user.getResourceUsage().setAMLimit(partitionName, userAMLimit);
{noformat}
Now if we need to do this in {{getUserAMResourceLimitPerPartition}}, we need to
pass the username as well. This user name comes from {{application}}
{code}
// Check user am resource limit
User user = getUser(application.getUser());
{code}
Hence,
1. We cannot pre-compute user am-limit like we have done for am-limit before
pendingOrderingPolicy loop in {{activateApplications}}
2. So as we compute this limit every time, we store in
{{user.getResourceUsage()}}. I thought of resusing this. However there can be
cases where 0 user/1 users/Multiple users for one queue. So getting correct
user is not really predictable for all getters (now we do not supply any user
name in getAMUserLimit/Partition, even though we take first user, there can be
cases where 0 users).
Thoughts?
bq. CapacitySchedulerPage, Instead of getAMResourceLimit() , Shouldn't you use
getAMResourceLimit(partition)?
{code}
PartitionResourcesInfo resourceUsages =
lqinfo.getResources().getPartitionResourceUsageInfo(label);
// Get UserInfo from first user to calculate AM Resource Limit per user.
ResourceInfo userAMResourceLimit = null;
ArrayList<UserInfo> usersList = lqinfo.getUsers().getUsersList();
if (usersList.isEmpty()) {
// If no users are present, consider AM Limit for that queue.
userAMResourceLimit = resourceUsages.getAMResourceLimit();
}
....
{code}
Here {{resourceUsages}} is already taken for specific label. Hence i think we
do not need per label am-limit.
> AM max resource configuration per partition to be displayed/updated correctly
> in UI and in various partition related metrics
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-4304
> URL: https://issues.apache.org/jira/browse/YARN-4304
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: webapp
> Affects Versions: 2.7.1
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-YARN-4304.patch, 0002-YARN-4304.patch,
> 0003-YARN-4304.patch, 0004-YARN-4304.patch, 0005-YARN-4304.patch,
> 0005-YARN-4304.patch, 0006-YARN-4304.patch, 0007-YARN-4304.patch,
> REST_and_UI.zip
>
>
> As we are supporting per-partition level max AM resource percentage
> configuration, UI and various metrics also need to display correct
> configurations related to same.
> For eg: Current UI still shows am-resource percentage per queue level. This
> is to be updated correctly when label config is used.
> - Display max-am-percentage per-partition in Scheduler UI (label also) and in
> ClusterMetrics page
> - Update queue/partition related metrics w.r.t per-partition
> am-resource-percentage
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
