[jira] [Commented] (YARN-5774) MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set yarn.scheduler.minimum-allocation-mb to 0.

[Miklos Szegedi (JIRA)]

    [ 
https://issues.apache.org/jira/browse/YARN-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623924#comment-15623924
 ] 

Miklos Szegedi commented on YARN-5774:
--------------------------------------

Thank you, [~yufeigu]!

I am wondering, if we really need to compare stepFactor to Resources.none() 
here rather than stepFactor.getMemorySize() to 0. It is an edge case but what 
if stepFactor.getGetVirtualCores() is non-zero but stepFactor.getMemorySize() 
is 0? We avoid throwing an exception in this case.

{code}
  public Resource normalize(Resource r, Resource minimumResource,
      Resource maximumResource, Resource stepFactor) {
    if (Resources.equals(stepFactor, Resources.none())) {
      throw new YarnRuntimeException("StepFactor resource cannot be zero!");
    }

    long normalizedMemory = Math.min(
        roundUp(
            Math.max(r.getMemorySize(), minimumResource.getMemorySize()),
            stepFactor.getMemorySize()),
            maximumResource.getMemorySize());
    return Resources.createResource(normalizedMemory);
  }
{code}


> MR Job stuck in ACCEPTED status without any progress in Fair Scheduler if set 
> yarn.scheduler.minimum-allocation-mb to 0.
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-5774
>                 URL: https://issues.apache.org/jira/browse/YARN-5774
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>              Labels: oct16-easy
>         Attachments: YARN-5774.001.patch, YARN-5774.002.patch
>
>
> MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
> because there is no resource request for the AM. This happened when you 
> configure {{yarn.scheduler.minimum-allocation-mb}} to zero.
> The problem is in the code used by both Capacity Scheduler and Fair 
> Scheduler. {{scheduler.increment-allocation-mb}} is a concept in FS, but not 
> CS. So the common code in class RMAppManager passes the 
> {{yarn.scheduler.minimum-allocation-mb}} as incremental one because there is 
> no incremental one for CS when it tried to normalize the resource requests.
> {code}
>      SchedulerUtils.normalizeRequest(amReq, scheduler.getResourceCalculator(),
>           scheduler.getClusterResource(),
>           scheduler.getMinimumResourceCapability(),
>           scheduler.getMaximumResourceCapability(),
>           scheduler.getMinimumResourceCapability());  --> incrementResource 
> should be passed here.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org