[ https://issues.apache.org/jira/browse/YARN-8951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Szilard Nemeth reassigned YARN-8951: ------------------------------------ Assignee: (was: Szilard Nemeth) > Defining default queue placement rule in allocations file with create="false" > throws an NPE > ------------------------------------------------------------------------------------------- > > Key: YARN-8951 > URL: https://issues.apache.org/jira/browse/YARN-8951 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Szilard Nemeth > Priority: Major > Attachments: default-placement-rule-with-create-false.patch > > > If the default queue placement rule is defined with {{create="false"}} and a > scheduling request is created for queue {{"root.default"}}, then > {{FairScheduler#assignToQueue}} throws an NPE, while trying to construct an > error message in the catch block of {{IllegalStateException}}, relying on the > fact that the {{rmApp}} is not null but it is. > Example of such a config file: > {code:java} > <?xml version="1.0"?> > <allocations> > <queue name="parentq" type="parent"> > <minResources>1024mb,0vcores</minResources> > </queue> > <queuePlacementPolicy> > <rule name="default" create="false"/> > </queuePlacementPolicy> > </allocations> > {code} > This is suspicious, as there are some null checks for {{rmApp}} in the same > method. > Not sure if this is a special case for the tests or it is reproducable in a > cluster, this needs further investigation. > In any case, it's not good that we try to dereference the {{rmApp}} that is > null. > On the other hand, I'm not sure if the default queue placement rule with > {{create="false"}} makes sense at all. Looking at the documentation > ([https://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html):] > {quote}default: the app is placed into the queue specified in the ‘queue’ > attribute of the default rule. *If ‘queue’ attribute is not specified, the > app is placed into ‘root.default’ queue.* > A queuePlacementPolicy element: which contains a list of rule elements that > tell the scheduler how to place incoming apps into queues. Rules are applied > in the order that they are listed. Rules may take arguments. *All rules > accept the “create” argument, which indicates whether the rule can create a > new queue. “Create” defaults to true; if set to false and the rule would > place the app in a queue that is not configured in the allocations file, we > continue on to the next rule.* The last rule must be one that can never issue > a continue.... > {quote} > In this case, the rule has the queue property suppressed so the apps should > be placed to the {{root.default}} queue (which is an undefined queue > according to the config file), and create is false, meaning that the queue > {{root.default}} cannot be created at all. > *This seems to be a case of an invalid queue configuration file for me.* > [~jlowe], [~leftnoteasy]: What is your take on this? > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org