[ 
https://issues.apache.org/jira/browse/YARN-10082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015180#comment-17015180
 ] 

Hadoop QA commented on YARN-10082:
----------------------------------

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m 58s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m  3s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m 
10s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch 
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
30s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m  8s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10082 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12990863/YARN-10082-002.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  xml  |
| uname | Linux c0bd1869d0e3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1c51f36 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/25387/testReport/ |
| Max. process+thread count | 854 (vs. ulimit of 5500) |
| modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
| Console output | 
https://builds.apache.org/job/PreCommit-YARN-Build/25387/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> FS-CS converter: disable terminal placement rule checking
> ---------------------------------------------------------
>
>                 Key: YARN-10082
>                 URL: https://issues.apache.org/jira/browse/YARN-10082
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>            Priority: Critical
>         Attachments: YARN-10082-001.patch, YARN-10082-002.patch
>
>
> Before YARN-8967, {{QueuePlacementRule}} class had a method called 
> {{isTerminal()}}. However, sometimes this method was hard-coded to return 
> false, accepting such configurations as:
> {noformat}
>     <queuePlacementPolicy>
>         <rule name="specified" create="true"/>
>         <rule name="nestedUserQueue" create="true">
>             <rule name="default" create="true" queue="users"/>
>         </rule>
>         <rule name="default"/>
>     </queuePlacementPolicy>
> {noformat}
> It's because {{NestedUserQueue.isTerminal()}} always returns {{false}}.
> This changed after YARN-8967. Now, this configuration is not accepted because 
> {{QueuePlacementPolicy.fromXml()}} calculates a list of terminal rules 
> differently:
> https://github.com/apache/hadoop/blob/5257f50abb71905ef3068fd45541d00ce9e8f355/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java#L176-L183
> In order to migrate existing configuration that were created before 
> YARN-8967, we need a new switch (at least in migration mode) in FS to turn 
> off this validation, otherwise the tool will not be able to migrate these 
> configs and the following exception will be thrown:
> {noformat}
> ~$ ./yarn fs2cs -y /tmp/yarn-site.xml -f /tmp/fair-scheduler.xml -o /tmp
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of YARN_OPTS.
> 20/01/13 05:48:20 INFO converter.FSConfigToCSConfigConverter: Output 
> directory for yarn-site.xml and capacity-scheduler.xml is: /tmp
> 20/01/13 05:48:20 INFO converter.FSConfigToCSConfigConverter: Conversion 
> rules file is not defined, using default conversion config!
> 20/01/13 05:48:21 INFO converter.FSConfigToCSConfigConverter: Using 
> explicitly defined fair-scheduler.xml
> WARNING: This feature is experimental and not intended for production use!
> 20/01/13 05:48:21 INFO conf.Configuration: resource-types.xml not found
> 20/01/13 05:48:21 INFO resource.ResourceUtils: Unable to find 
> 'resource-types.xml'.
> 20/01/13 05:48:21 INFO security.YarnAuthorizationProvider: 
> org.apache.hadoop.yarn.security.ConfiguredYarnAuthorizer is instantiated.
> 20/01/13 05:48:21 INFO scheduler.AbstractYarnScheduler: Minimum allocation = 
> <memory:1024, vCores:1>
> 20/01/13 05:48:21 INFO scheduler.AbstractYarnScheduler: Maximum allocation = 
> <memory:8192, vCores:2>
> 20/01/13 05:48:21 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.SpecifiedPlacementRule
> 20/01/13 05:48:21 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.UserPlacementRule
> 20/01/13 05:48:21 INFO fair.AllocationFileLoaderService: Loading allocation 
> file file:/tmp/fair-scheduler.xml
> 20/01/13 05:48:22 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.SpecifiedPlacementRule
> 20/01/13 05:48:22 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.UserPlacementRule
> 20/01/13 05:48:22 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.DefaultPlacementRule
> 20/01/13 05:48:22 INFO placement.PlacementFactory: Creating PlacementRule 
> implementation: class 
> org.apache.hadoop.yarn.server.resourcemanager.placement.DefaultPlacementRule
> 20/01/13 05:48:22 INFO service.AbstractService: Service 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler 
> failed in state INITED
> java.io.IOException: Failed to initialize FairScheduler
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1438)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1479)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:206)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:101)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:116)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:44)
> Caused by: 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationConfigurationException:
>  Rules after rule 2 in queue placement policy can never be reached
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy.updateRuleSet(QueuePlacementPolicy.java:110)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy.fromXml(QueuePlacementPolicy.java:187)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.getQueuePlacementPolicy(AllocationFileLoaderService.java:276)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:257)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1436)
> ... 6 more
> 20/01/13 05:48:22 ERROR converter.FSConfigToCSConfigConverterMain: Error 
> while starting FS configuration conversion!
> org.apache.hadoop.service.ServiceStateException: java.io.IOException: Failed 
> to initialize FairScheduler
> at 
> org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:206)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:101)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:116)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:44)
> Caused by: java.io.IOException: Failed to initialize FairScheduler
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1438)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1479)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> ... 4 more
> Caused by: 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationConfigurationException:
>  Rules after rule 2 in queue placement policy can never be reached
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy.updateRuleSet(QueuePlacementPolicy.java:110)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy.fromXml(QueuePlacementPolicy.java:187)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.getQueuePlacementPolicy(AllocationFileLoaderService.java:276)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:257)
> at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1436)
> ... 6 more
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to