[
https://issues.apache.org/jira/browse/YARN-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266791#comment-16266791
]
Wilfred Spiegelenburg commented on YARN-7535:
---------------------------------------------
The demand for an application is not limited and is we do not stop adding up
demand at any limit for an application.
This is purely about the maximum demand a queue is assigned for the purpose of
scheduling. Showing a demand in a queue that is higher than what could ever be
allocated in the queue also does not make sense from a scheduling perspective.
I agree that it is one of the possible factors you look at from a tuning
perspective.
Demand should not be the one and only number to base a queue size on. An
applications demand is just the requested resources for the Application Master
until the application runs. When the AM is not started there is no request for
any other resources beside that one AM container. There is thus no demand
beside that one container. This means that the demand in a queue is of limited
value for tuning the size of a queue. Non runnable applications always skew the
demand number. If you run into the maximum size of a queue demand will be
skewed unless all applications in all queues are running. Most likely it will
give the wrong info and your tuning would be off. If you are not running into
the maximum queue size and still have unsatisfied demand something else is
going on. Again demand will not be the value you are tuning on in that case.
> We should display origin value of demand in fair scheduler page
> ---------------------------------------------------------------
>
> Key: YARN-7535
> URL: https://issues.apache.org/jira/browse/YARN-7535
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Reporter: YunFan Zhou
> Assignee: YunFan Zhou
>
> The value of *demand* of leaf queue that we now view on the fair scheduler
> page shows only the value of *maxResources* when the demand value is greater
> than *maxResources*. It doesn't reflect the real situation. Most of the time,
> when we expand the queue, we often rely on seeing the current demand real
> value.
> {code:java}
> private void updateDemandForApp(FSAppAttempt sched, Resource maxRes) {
> sched.updateDemand();
> Resource toAdd = sched.getDemand();
> if (LOG.isDebugEnabled()) {
> LOG.debug("Counting resource from " + sched.getName() + " " + toAdd
> + "; Total resource consumption for " + getName() + " now "
> + demand);
> }
> demand = Resources.add(demand, toAdd);
> demand = Resources.componentwiseMin(demand, maxRes);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
