[
https://issues.apache.org/jira/browse/YARN-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu updated YARN-5774:
---------------------------
Description:
MR Job stuck in ACCEPTED status without any progress in Fair Scheduler because
there is no resource request for the AM.
The problem is in the code used by both Capacity Scheduler and Fair Scheduler.
scheduler.increment-allocation-mb is a concept in FS, but not CS. So the common
code in class RMAppManager passes the yarn.scheduler.minimum-allocation-mb as
incremental one because there is no incremental one for CS when it tried to
normalize the resource requests.
{code}
SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
scheduler.getClusterResource(),
scheduler.getMinimumResourceCapability(),
scheduler.getMaximumResourceCapability(),
scheduler.getMinimumResourceCapability()); --> incrementResource
should be passed here.
{code}
was:
The problem is in the code used by both Capacity Scheduler and Fair Scheduler.
scheduler.increment-allocation-mb is a concept in FS, but not CS. So the common
code in class RMAppManager passes the yarn.scheduler.minimum-allocation-mb as
incremental one because there is no incremental one for CS when it tried to
normalize the resource requests.
{code}
SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
scheduler.getClusterResource(),
scheduler.getMinimumResourceCapability(),
scheduler.getMaximumResourceCapability(),
scheduler.getMinimumResourceCapability()); --> incrementResource
should be passed here.
{code}
> A wrong parameter is passed when normalizing resource requests
> --------------------------------------------------------------
>
> Key: YARN-5774
> URL: https://issues.apache.org/jira/browse/YARN-5774
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 3.0.0-alpha1
> Reporter: Yufei Gu
> Assignee: Yufei Gu
>
> MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
> because there is no resource request for the AM.
> The problem is in the code used by both Capacity Scheduler and Fair
> Scheduler. scheduler.increment-allocation-mb is a concept in FS, but not CS.
> So the common code in class RMAppManager passes the
> yarn.scheduler.minimum-allocation-mb as incremental one because there is no
> incremental one for CS when it tried to normalize the resource requests.
> {code}
> SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
> scheduler.getClusterResource(),
> scheduler.getMinimumResourceCapability(),
> scheduler.getMaximumResourceCapability(),
> scheduler.getMinimumResourceCapability()); --> incrementResource
> should be passed here.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
