subject:"\[jira\] \[Commented\] \(YARN\-10386\) Create new JSON schema for Placement Rules"

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2021-10-28 Thread Steve Loughran (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435352#comment-17435352
 ] 

Steve Loughran commented on YARN-10386:
---

done: YARN-10992

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2021-10-25 Thread Tamas Domok (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433629#comment-17433629
 ] 

Tamas Domok commented on YARN-10386:


Hi [~ste...@apache.org]. Could you open a separate ticket for that compile 
issue with a reproduction step? (Locally I couldn't reproduce this yet, just a 
very similar issue: my generated code contained the wrong import statement 

_import javax.annotation.processing.Generated;_ instead of the correct one 
_import javax.annotation.Generated;_)

 

Probably a missing dependency in the resource manager's pom file would solve 
this.
{code:java}

  javax.annotation
  javax.annotation-api
  compile
 {code}

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2021-10-21 Thread Steve Loughran (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432649#comment-17432649
 ] 

Steve Loughran commented on YARN-10386:
---

This is somehow stopping java 17 (maybe 11_+?) compilation, as the generated 
classes have an import javax.annotation.Generated which is not in the JRE 
classes. 

{code}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on 
project hadoop-yarn-server-resourcemanager: Compilation failure: Compilation 
failure:
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/MappingRulesDescription.java:[8,24]
 cannot find symbol
[ERROR]   symbol:   class Generated
[ERROR]   location: package javax.annotation
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/MappingRulesDescription.java:[20,2]
 cannot find symbol
[ERROR]   symbol: class Generated
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/Rule.java:[6,24]
 cannot find symbol
[ERROR]   symbol:   class Generated
[ERROR]   location: package javax.annotation
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/Rule.java:[27,2]
 cannot find symbol
[ERROR]   symbol: class Generated
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/Rule.java:[304,6]
 cannot find symbol
[ERROR]   symbol:   class Generated
[ERROR]   location: class 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/Rule.java:[255,6]
 cannot find symbol
[ERROR]   symbol:   class Generated
[ERROR]   location: class 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule
[ERROR] 
/Users/stevel/hadoop/commit/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/placement/schema/Rule.java:[214,6]
 cannot find symbol
[ERROR]   symbol:   class Generated
[ERROR]   location: class 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-yarn-server-resourcemanager
{code}

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-09-01 Thread Adam Antal (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188421#comment-17188421
 ] 

Adam Antal commented on YARN-10386:
---

Thanks for the addendum patch [~pbacsko]. +1 from me, committed to trunk (let's 
not block other issues with this).

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Benjamin Teke (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187830#comment-17187830
 ] 

Benjamin Teke commented on YARN-10386:
--

[~pbacsko], the addendum looks good to me. +1 (non-binding)

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187811#comment-17187811
 ] 

Peter Bacsko commented on YARN-10386:
-

OK, no more ASF warning. I added the two generated files to the RAT plugin 
ignore list. Since this is a generated code, this shouldn't be a problem.

[~snemeth] [~bteke] pls review addendum-2.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch, YARN-10386-appendum2.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187799#comment-17187799
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
39m 23s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
36s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 15s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
34s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
29s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m  4s{color} 
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
27s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}156m  9s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: 
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/115/artifact/out/Dockerfile
 |
|

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187674#comment-17187674
 ] 

Peter Bacsko commented on YARN-10386:
-

The JSON schema now contains the license string.

The two files are automatically generated so it's a bit difficult to add to 
them, I'm not even sure if it's possible. We might need to resort to manual 
code generation + adding the license header.

cc [~bteke] [~snemeth].

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch, YARN-10386-appendum.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187655#comment-17187655
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 32m 
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
47s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
39m 26s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 19s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 49s{color} 
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
27s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}187m  8s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: 
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/113/artifact/out/Dockerfile
 |
| JIRA Issue | YARN-10386 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13010759/YARN-10386-appendum.patch
 |
| Optional Tests | dupname

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-31 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187504#comment-17187504
 ] 

Peter Bacsko commented on YARN-10386:
-

[~brahmareddy] I'll upload an addendum patch to solve this.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-30 Thread Brahma Reddy Battula (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187290#comment-17187290
 ] 

Brahma Reddy Battula commented on YARN-10386:
-

[~snemeth] and [~pbacsko].

 

Looks this patch introduced some ASF Warnings, please try to address check 
following for same. (Even last yetus report of the this Jira also show this 
errors.)

[https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/116/artifact/out/patch-asflicense-problems.txt]

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-28 Thread Szilard Nemeth (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186614#comment-17186614
 ] 

Szilard Nemeth commented on YARN-10386:
---

Hi [~pbacsko],
Thanks for working on this patch and thanks for the meaningful discussion with 
[~shuzirra], very informative.
I think the schema is more than good enough for the first commit, so I'm 
committing this right now.
As we discussed offline, if any change is required we can always create a 
subjira to this umbrella and fix / modify the schema.
Thanks again.
Committed to trunk.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-28 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186538#comment-17186538
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
59s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m  
1s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
56s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 24m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
45m 13s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
47s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
40s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 36m 
46s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
16s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
58s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
15s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
35s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 19s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
30s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-28 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186398#comment-17186398
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
48s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m 
51s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
36s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m 
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
40m 49s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
2s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
28s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 35m  
3s{color} | {color:blue} Used deprecated FindBugs config; considering switching 
to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
15s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m  
0s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 22m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
47s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 19m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 18s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
2s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17186087#comment-17186087
 ] 

Peter Bacsko commented on YARN-10386:
-

Some examples:

{{u:%user:%user}}
{noformat}
"rules":
{
{
"type": "user",
"matches": "*",
"policy": "user",
"fallbackResult": "placeDefault"
}
}
{noformat}
 

{{u:%user:root.dev.%primary_group.%user}}
{noformat}
"rules":
{   
{
"type": "user",
"matches": "*",
"policy": "primaryGroupUser",
"parentQueue": "root.dev",
"fallbackResult": "placeDefault"
}
}
{noformat}
 

{{u:%user:%secondary_group.%user}}
{noformat}
"rules":
{
{
"type": "user",
"matches": "*",
"policy": "secondaryGroupUser",
"fallbackResult": "placeDefault"
}
}
{noformat}
 

{{u:%user:root.users}}
{noformat}
"rules":
{   
{
"type": "user",
"matches": "*",
"policy": "setDefaultQueue",
"value": "root.users",
"fallbackResult": "skip"
},
{
"type": "user",
"matches": "*",
"policy": "defaultQueue",
"fallbackResult": "reject"
}
}
{noformat}

As we can see, we need two rules for a hard-coded path like "root.users". This 
might look weird, but this very similar to FS, where the rule is also called 
{{}} with an overridden default queue. So in theory it's the same. 

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch, YARN-10386-007.patch, 
> YARN-10386-008.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185899#comment-17185899
 ] 

Peter Bacsko commented on YARN-10386:
-

Ok, had a discussion with [~shuzirra]. Now I can better understand what he 
meant above.

The following changes will be introduced:
1. "nestedUserRule" will be removed, because it's used only in case of 
"nestedUserRule" policy.
2. Add two new policies "secondaryGroupUser" and "primaryGroupUser".
3. Remove "defaultQueue" setting. The default queue will be set with a 
different rule.
4. Also introduce "setDefaultQueue" policy. This will change the default queue 
until it's set to something else.
5. The naming of "queue" is misleading, let's change that to sth like 
"parentQueue" or "parentQueuePath".
6. Custom policy: we can keep this and add a "custom" field perhaps. This can 
contain any random definition like 
{{root.user.%primary_group.%secondary_group}} and it will not be rejected. The 
tool won't validate whether the string makes any sense or not.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185799#comment-17185799
 ] 

Peter Bacsko commented on YARN-10386:
-

"Why is the "defaultQueue" property a separate option not a policy?"

That's what I explained, it's confusing.  That's intended to set the fallback 
default queue, in case a fallback occurs and the the {{fallbackResult}} is 
{{placeToDefault}}. 

"I think we are overusing the defaultQueue in the schema, we should use other 
names like, changeDefaultQueue, placeToDefaultQueue, fallbackToDefaultQueue or 
something along these lines to make sure it is clear what it will do."

Yep, that's what my first command is about :)

"Nested" is already a policy itself and the "nestedUserRule" is only 
interpreted if this particular policy is used.

The reason for a separate nested policy is to prevent users from all kinds of 
crazy configurations. Yes, the engine itself can happily resolve stuff like 
"root.%default.%primary_group", but I don't think these kind of configurations 
should be allowed. So the schema itself takes care of the allowed combinations. 

Regarding changing the default, I think it's an overkill to have a separate 
changeDefault policy. It's certainly how it works under the hood as an action, 
but I don't see too much benefit of exposing this. 

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Gergely Pollak (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185790#comment-17185790
 ] 

Gergely Pollak commented on YARN-10386:
---

I'm not sure I understand your defaultQueue comment properly, but on the 
backend %default means root.default initially and if you user a 
VariableUpdateAction, it will change it for default placement targets and 
default fallbacks as well.

Why is the "defaultQueue" property a separate option not a policy? It is just a 
similar action to anything, and it cannot be added as an extra to any rule. I 
mean you cannot have a policy AND a change default queue, also default queue 
change is for the whole application submission, not only for the given rule.

I think we are overusing the defaultQueue in the schema, we should use other 
names like, changeDefaultQueue, placeToDefaultQueue, fallbackToDefaultQueue or 
something along these lines to make sure it is clear what it will do.

I think we have too many options under the properties and it can easily lead to 
incorrect configuration, I think we should move the nesterUserRule and the 
(change)defaultQueue into the policies. Nested user rules are just complex 
target paths, like root.%primary_group.%user, but nothing else, so I don't 
think we should move them to a separate property (I don't even think we need to 
do a separate policy for them, since we can just use regular queue placement).

The nested rules in FS always map to something like root.some.path..%user, 
and the rule decides the  part, if create is not allowed and the queue does 
not exist then it moves onto the next. (I know it also supports default, but 
it's like place to %default) So all this can be easily mapped to regular queue 
placement with rules like: root.%primary_group.%user on fallback skip.

I would also add a "custom" to the type, since matcher also can be custom, not 
only rules.

 

 

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185770#comment-17185770
 ] 

Peter Bacsko commented on YARN-10386:
-

[~adam.antal] the entire project is findbugs/style checked and compiled many 
times over due to the modification in {{hadoop-project/pom.xml}}. That makes 
the build very slow and eventually it runs out of resources. 

Anyway I'll make the changes.

Any thoughts about the JSON schema itself?

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Adam Antal (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185764#comment-17185764
 ] 

Adam Antal commented on YARN-10386:
---

Thanks for working on this [~pbacsko].

I saw you had some troubles with the jenkins. I don't see what exactly causes 
the jenkins failures but we should make sure that after we merge this, it won't 
appear in other jenkins results.

Regarding the patch I have two comments:

I am a little concerned about the checkstyle ignores:
{noformat}
@SuppressWarnings({"checkstyle:hideutilityclassconstructor", 
"checkstyle:linelength"})
{noformat}
- I don't see problems creating a private constructor for this class to prevent 
instantiation of this utility class.
- Also, this can be moved to a constant: {{ 
"org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema",}}
 any maybe we can add the checkstyle warning annotation to only to the constant 
to be as restrictive as we can.

Other: 
- In {{hadoop-yarn-server-resourcemanager/pom.xml}} can we also use the 
${jsonschema2pojo.version} constant for the version if possible?

I also add that I checked the new maven dependency and I saw no associated 
CVE-s to that, so I think it's fine to use it.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-27 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185738#comment-17185738
 ] 

Peter Bacsko commented on YARN-10386:
-

Ok, the build what matters is 
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/96/.

ASF license warnings and the unit test failures can be ignored. Tests were 
executed for the whole project, but several times the forked process ran into 
resource limits.



> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch, YARN-10386-003.patch, YARN-10386-004.patch, 
> YARN-10386-005.patch, YARN-10386-006.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-26 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185508#comment-17185508
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  9m 
24s{color} | {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 17m  
2s{color} | {color:red} root in trunk failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  2m 
17s{color} | {color:red} root in trunk failed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
41m  0s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
44s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
5s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 31m 
14s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
19s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
35s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 20m  
1s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m  
9s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 21m  9s{color} 
| {color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 210 new + 1841 unchanged 
- 0 fixed = 2051 total (was 1841) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
50s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 17m 50s{color} 
| {color:red} root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with 
JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1639 new + 
307 unchanged - 0 fixed = 1946 total (was 307) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 14m 
33s{color} | {color:red} patch has errors when

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-26 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185406#comment-17185406
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
17s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
46s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m  
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
36m 36s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
23s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
4s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 29m 
42s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
18s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
58s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
44s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 52s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
17s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-25 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184888#comment-17184888
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 33m 
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m 
13s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
49s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m 
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
39m  7s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
31s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
51s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  2m  
8s{color} | {color:blue} Used deprecated FindBugs config; considering switching 
to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
18s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 32m 
58s{color} | {color:red} root in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 23m 
55s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
22s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
49s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 14m 
22s{color} | {color:red} patch has errors when building and testing our client 
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  6m 
16s{color} | {color:red} root in the patch failed with JDK

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-14 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178135#comment-17178135
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
 7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
41s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
47s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
38m 49s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
23s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
58s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 31m  
4s{color} | {color:blue} Used deprecated FindBugs config; considering switching 
to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
19s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
35s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 19m 
34s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
59s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
34s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 
34s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
2m 55s{color} | {color:orange} root: The patch generated 1 new + 0 unchanged - 
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 14m 
51s{color} | {color:red} patch has errors when building and testing our client 
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  6m 
24s{color} | {color:red} root in the patch failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
|

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-13 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177006#comment-17177006
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 23m 
52s{color} | {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
32s{color} | {color:red} root in trunk failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
30s{color} | {color:red} root in trunk failed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 28s{color} | {color:orange} The patch fails to run checkstyle in root 
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 24m 
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
38m 44s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
20s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
5s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 32m 
30s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
19s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 19m 
43s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
23s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 19m 23s{color} 
| {color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 2047 new + 0 unchanged - 
0 fixed = 2047 total (was 0) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
40s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 16m 40s{color} 
| {color:red} root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with 
JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 1942 new + 0 
unchanged - 0 fixed = 1942 total (was 0) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
2m 34s{color} | {color:orange} root: The patch generated 1 new + 0 unchanged - 
0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red}

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-12 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176661#comment-17176661
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 30m 
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  3m 
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
44s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
27s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
38m 58s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
25s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
20s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  1m 
49s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
16s{color} | {color:blue} branch/hadoop-project no findbugs output file 
(findbugsXml.xml) {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
35s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
25s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch 
failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  6m 
51s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  9m  
7s{color} | {color:red} root in the patch failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  9m  7s{color} 
| {color:red} root in the patch failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  8m 
35s{color} | {color:red} root in the patch failed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  8m 35s{color} 
| {color:red} root in the patch failed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
2m 40s{color} | {color:orange} root: The patch generated 1 new + 0 unchanged - 
0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red}  3m 
24s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red}  4m 
21s{color} |

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-12 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176572#comment-17176572
 ] 

Peter Bacsko commented on YARN-10386:
-

[~snemeth] [~adam.antal] [~shuzirra] [~sunilg] - do you guys have any comments?

The maven extension was contributed by [~bteke].

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch, 
> YARN-10386-002.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-12 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176413#comment-17176413
 ] 

Peter Bacsko commented on YARN-10386:
-

Wow, there are a lot of failures.

I'll investigate the enforcer failure, because we added a new dependency to a 
pom.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json, YARN-10386-001.patch
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-12 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176210#comment-17176210
 ] 

Hadoop QA commented on YARN-10386:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
26s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
40s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m 
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
37m  5s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
39s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
16s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 41m 
17s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 41m 
10s{color} | {color:red} root in trunk has 2 extant findbugs warnings. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
42s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 26m 
28s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m 
28s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 24m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
26s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m 
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
3m  9s{color} | {color:orange} root: The patch generated 4 new + 0 unchanged - 
0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 14m 
51s{color} | {color:red} patch has errors when building and testing our client 
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  7m  
1s{color} | {color:red} root in the patch failed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. {color} |
| {color:red}-1{color} | {color:red}

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-11 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175571#comment-17175571
 ] 

Peter Bacsko commented on YARN-10386:
-

I've been thinking, probably "defaultQueue" should be renamed to something 
else. There's also a "defaultQueue" policy and users might think that they have 
to use "defaultQueue" to override "root.default". But it only affects 
"placeDefault" fallback rule, at least that's my intention right now. So we 
need to come up with some clever naming. I think "fallbackDefaultQueue" sounds 
better.

> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

2020-08-05 Thread Peter Bacsko (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-10386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171532#comment-17171532
 ] 

Peter Bacsko commented on YARN-10386:
-

Attached v1 version of the schema.

Explanation:
* {{type}}: user / group / application --- pretty much the first column of the 
current format.
* {{matches}}: it's either "\*" or a specific group/user -- the second column 
of the current format (will use "*" instead of "%user").
* {{policy}}: a fixed set of mapping policies, basically the same as an FS 
placement rule.
* {{queue}}: if the policy is {{user}}, {{primaryGroup}}, {{secondaryGroup}}, 
then this is prepended to the queue string. If the rule is {{defaultQueue}} 
then "root.default" will be overridden.
* {{nestedUserRule}}: optional, used only if {{nestedUser}} policy is defined. 
Note that unlike to Fair Scheduler, we don't have "outer" create flag, because 
it would require very deep changes in Capacity Scheduler. So we only support 
"innerCreate" for the "%user" part.
* {{fallbackResult}}: what happens if the placement fails, eg. the target queue 
does not exist or cannot be created. Right now in CS, we return "null" for the 
placement context which is interpreted as "root.default". In FS, we proceed to 
the next rule. Here, we can fine-tune this behavior: {{skip}} is the FS 
approach, {{placeDefault}} is the CS approach, {{reject}} is new and 
straightforward (submission will be rejected on the client side).
* {{defaultQueue}}: what should be the default queue in case of 
{{placeDefault}} fallback result.
* {{create}}: whether the queue should be created or not if it does not exist. 
Note that the parent queue of the leaf must be a managed parent in order for 
this switch to have effect. 


> Create new JSON schema for Placement Rules
> --
>
> Key: YARN-10386
> URL: https://issues.apache.org/jira/browse/YARN-10386
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: capacity scheduler, capacityscheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: MappingRulesDescription_v1.json
>
>
> Tasks in this JIRA:
>  # Create new JSON schema
>  # Add Maven plugin which generates Java POJOs based on the schema
>  # Add helper class which essentially does the same as #2 (for dev purposes)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

[jira] [Commented] (YARN-10386) Create new JSON schema for Placement Rules

32 matches

Site Navigation

Mail list logo

Footer information