[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347313#comment-17347313 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for update. The patch LGTM now. Waiting [~pbacsko] for double check. Thanks. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch, YARN-10771.004.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[
https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347307#comment-17347307
]
Hadoop QA commented on YARN-10771:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 6m
42s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 2
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
29s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
54s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
27s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
39s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
56s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 19s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 26m
37s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 4m
0s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
19s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
19s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
22s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
22s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 35s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/995/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt{color}
| {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 2 new +
95 unchanged - 0 fixed = 97 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
|
[jira] [Updated] (YARN-10725) Backport YARN-10120 to branch-3.3
[ https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T updated YARN-10725: - Attachment: YARN-10725-branch-3.3.v5.patch > Backport YARN-10120 to branch-3.3 > - > > Key: YARN-10725 > URL: https://issues.apache.org/jira/browse/YARN-10725 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10120-branch-3.3.patch, > YARN-10725-branch-3.3.patch, YARN-10725-branch-3.3.v2.patch, > YARN-10725-branch-3.3.v3.patch, YARN-10725-branch-3.3.v4.patch, > YARN-10725-branch-3.3.v5.patch, image-2021-04-05-16-48-57-034.png, > image-2021-04-05-16-50-55-238.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10778) Bump handlebars from 3.0.7 to 4.7.7 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp
[ https://issues.apache.org/jira/browse/YARN-10778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-10778: -- Labels: pull-request-available (was: ) > Bump handlebars from 3.0.7 to 4.7.7 in > /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp > -- > > Key: YARN-10778 > URL: https://issues.apache.org/jira/browse/YARN-10778 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 3.4.0 >Reporter: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10778) Bump handlebars from 3.0.7 to 4.7.7 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp
Wei-Chiu Chuang created YARN-10778: -- Summary: Bump handlebars from 3.0.7 to 4.7.7 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp Key: YARN-10778 URL: https://issues.apache.org/jira/browse/YARN-10778 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.4.0 Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10777) Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp
[ https://issues.apache.org/jira/browse/YARN-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved YARN-10777. Fix Version/s: 3.4.0 Resolution: Fixed > Bump node-sass from 4.13.0 to 4.14.1 in > /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp > --- > > Key: YARN-10777 > URL: https://issues.apache.org/jira/browse/YARN-10777 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.4.0 >Reporter: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10777) Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp
[ https://issues.apache.org/jira/browse/YARN-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-10777: -- Labels: pull-request-available (was: ) > Bump node-sass from 4.13.0 to 4.14.1 in > /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp > --- > > Key: YARN-10777 > URL: https://issues.apache.org/jira/browse/YARN-10777 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.4.0 >Reporter: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10777) Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp
Wei-Chiu Chuang created YARN-10777: -- Summary: Bump node-sass from 4.13.0 to 4.14.1 in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp Key: YARN-10777 URL: https://issues.apache.org/jira/browse/YARN-10777 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Affects Versions: 3.4.0 Reporter: Wei-Chiu Chuang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347248#comment-17347248 ] chaosju commented on YARN-10771: Thanks [~zhuqi] for review, fixed the checkstyle warn and update a new patch-004 > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch, YARN-10771.004.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Attachment: YARN-10771.004.patch > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch, YARN-10771.004.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[
https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347190#comment-17347190
]
Hadoop QA commented on YARN-10258:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 26m
9s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 2
new or modified test files. {color} |
|| || || || {color:brown} branch-3.3 Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
54s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 39s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 19m
28s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
21s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green}{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 70 unchanged - 1 fixed = 70 total (was 71) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 19s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
35s{color} | {color:green}{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
20s{color} | {color:green}{color} | {color:green}
hadoop-yarn-server-nodemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green}{color} | {color:green} The patch does not generate
ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}114m 53s{color} |
{color:black}{color} | {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/994/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10258 |
| JIRA Patch URL |
[jira] [Updated] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10258: Attachment: YARN-10258-branch-3.3.001.patch > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Fix For: 3.4.0 > > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258-branch-3.3.001.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko reopened YARN-10258: - > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Fix For: 3.4.0 > > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347005#comment-17347005 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for update. The patch LGTM, go on to fix the checkstyle. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[
https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346987#comment-17346987
]
Hadoop QA commented on YARN-10771:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
48s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 2
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
6s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
32s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
8s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 11s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
48s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
1s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 25m
1s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 4m
3s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
33s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
33s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
50s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 37s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/993/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt{color}
| {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 9 new +
95 unchanged - 0 fixed = 104 total (was 95) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
|
[jira] [Commented] (YARN-8564) Add queue level application lifetime monitor in FairScheduler
[ https://issues.apache.org/jira/browse/YARN-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346911#comment-17346911 ] Qi Zhu commented on YARN-8564: -- Thanks [~tarunparimi] for reminder. I reopened it, you can take it if you are interested in improving this. > Add queue level application lifetime monitor in FairScheduler > -- > > Key: YARN-8564 > URL: https://issues.apache.org/jira/browse/YARN-8564 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.1.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-8564.001.patch, test1~3.jpg, test4.jpg > > > I wish to have application lifetime monitor for queue level in FairSheduler. > In our large yarn cluster, sometimes there are too many small jobs in one > minor queue but may run too long, it may affect our our high priority and > very important queue . If we can have queue level application lifetime > monitor in the queue level, and set small lifetime in the minor queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-8564) Add queue level application lifetime monitor in FairScheduler
[ https://issues.apache.org/jira/browse/YARN-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu reopened YARN-8564: -- > Add queue level application lifetime monitor in FairScheduler > -- > > Key: YARN-8564 > URL: https://issues.apache.org/jira/browse/YARN-8564 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.1.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-8564.001.patch, test1~3.jpg, test4.jpg > > > I wish to have application lifetime monitor for queue level in FairSheduler. > In our large yarn cluster, sometimes there are too many small jobs in one > minor queue but may run too long, it may affect our our high priority and > very important queue . If we can have queue level application lifetime > monitor in the queue level, and set small lifetime in the minor queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8564) Add queue level application lifetime monitor in FairScheduler
[ https://issues.apache.org/jira/browse/YARN-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346849#comment-17346849 ] Tarun Parimi commented on YARN-8564: [~zhuqi], Any reason this jira got resolved? I don't see this patch committed anywhere. And it doesn't seem to be a duplicate. > Add queue level application lifetime monitor in FairScheduler > -- > > Key: YARN-8564 > URL: https://issues.apache.org/jira/browse/YARN-8564 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.1.0 >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-8564.001.patch, test1~3.jpg, test4.jpg > > > I wish to have application lifetime monitor for queue level in FairSheduler. > In our large yarn cluster, sometimes there are too many small jobs in one > minor queue but may run too long, it may affect our our high priority and > very important queue . If we can have queue level application lifetime > monitor in the queue level, and set small lifetime in the minor queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10759) Encapsulate queue config modes
[
https://issues.apache.org/jira/browse/YARN-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346841#comment-17346841
]
Hadoop QA commented on YARN-10759:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
24s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
37s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 10s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 20m
19s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
51s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/992/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt{color}
| {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 130 unchanged - 6 fixed = 132 total (was 136)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 56s{color} |
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346830#comment-17346830 ] chaosju commented on YARN-10771: add new patch-003 > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Attachment: YARN-10771.003.patch > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch, > YARN-10771.003.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10776) Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router randomly
[ https://issues.apache.org/jira/browse/YARN-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-10776: -- Labels: pull-request-available (was: ) > Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router > randomly > > > Key: YARN-10776 > URL: https://issues.apache.org/jira/browse/YARN-10776 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: zhengchenyu >Assignee: zhengchenyu >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > I think we need to select resourcemanager or router randomly, especially for > router in yarn federation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10776) Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router randomly
zhengchenyu created YARN-10776: -- Summary: Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router randomly Key: YARN-10776 URL: https://issues.apache.org/jira/browse/YARN-10776 Project: Hadoop YARN Issue Type: Improvement Reporter: zhengchenyu I think we need to select resourcemanager or router randomly, especially for router in yarn federation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10776) Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router randomly
[ https://issues.apache.org/jira/browse/YARN-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu reassigned YARN-10776: -- Assignee: zhengchenyu > Make ConfiguredRMFailoverProxyProvider select ResourceManager or Router > randomly > > > Key: YARN-10776 > URL: https://issues.apache.org/jira/browse/YARN-10776 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: zhengchenyu >Assignee: zhengchenyu >Priority: Major > > I think we need to select resourcemanager or router randomly, especially for > router in yarn federation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10774) Federation: Normalize the yarn federation queue name
[ https://issues.apache.org/jira/browse/YARN-10774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan LUO updated YARN-10774: Attachment: YARN-10774.001.patch > Federation: Normalize the yarn federation queue name > > > Key: YARN-10774 > URL: https://issues.apache.org/jira/browse/YARN-10774 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, yarn >Reporter: Yuan LUO >Priority: Major > Attachments: YARN-10774.001.patch > > > While in YARN at root.abc is equivalent to the abc queue, the routing > behavior of both should be consistent in yarn federation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10775) Federation: Yarn running app web can't be unable to connect, because AppMaster can't redirect to the right address.
[ https://issues.apache.org/jira/browse/YARN-10775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu reassigned YARN-10775: -- Assignee: zhengchenyu > Federation: Yarn running app web can't be unable to connect, because > AppMaster can't redirect to the right address. > > > Key: YARN-10775 > URL: https://issues.apache.org/jira/browse/YARN-10775 > Project: Hadoop YARN > Issue Type: Bug >Reporter: zhengchenyu >Assignee: zhengchenyu >Priority: Major > > I setup a yarn federation cluster, I can't connect the running app web, but > the completed app's web works. > By some debug info, I found the reason. Because AmIpFilter.PROXY_HOSTS is set > by configuration (do it in > AmFilterInitializer.getProxyHostsAndPortsForAmFilter). In client > configuration, there are no configuation about 'rm web port', so the url > can't not be redirect. > I think client should not know rm's web port, especially in yarn federation > mode. > Note: > Here is my specially configuration for client. I use ha for select router > randomly. > || config || value || > | yarn.resourcemanager.ha.enabled | true | > | yarn.resourcemanager.ha.rm-ids | r1 | > |yarn.resourcemanager.address.r1 | router_host:8050 | > |yarn.resourcemanager.scheduler.address.r1 | localhost:8049 | -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10775) Federation: Yarn running app web can't be unable to connect, because AppMaster can't redirect to the right address.
zhengchenyu created YARN-10775: -- Summary: Federation: Yarn running app web can't be unable to connect, because AppMaster can't redirect to the right address. Key: YARN-10775 URL: https://issues.apache.org/jira/browse/YARN-10775 Project: Hadoop YARN Issue Type: Bug Reporter: zhengchenyu I setup a yarn federation cluster, I can't connect the running app web, but the completed app's web works. By some debug info, I found the reason. Because AmIpFilter.PROXY_HOSTS is set by configuration (do it in AmFilterInitializer.getProxyHostsAndPortsForAmFilter). In client configuration, there are no configuation about 'rm web port', so the url can't not be redirect. I think client should not know rm's web port, especially in yarn federation mode. Note: Here is my specially configuration for client. I use ha for select router randomly. || config || value || | yarn.resourcemanager.ha.enabled | true | | yarn.resourcemanager.ha.rm-ids | r1 | |yarn.resourcemanager.address.r1 | router_host:8050 | |yarn.resourcemanager.scheduler.address.r1 | localhost:8049 | -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10774) Federation: Normalize the yarn federation queue name
Yuan LUO created YARN-10774: --- Summary: Federation: Normalize the yarn federation queue name Key: YARN-10774 URL: https://issues.apache.org/jira/browse/YARN-10774 Project: Hadoop YARN Issue Type: Bug Components: federation, yarn Reporter: Yuan LUO While in YARN at root.abc is equivalent to the abc queue, the routing behavior of both should be consistent in yarn federation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[
https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346746#comment-17346746
]
Hadoop QA commented on YARN-10771:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
48s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
54s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
52s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
46s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
20s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
56s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 9s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 26m
26s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 3m
56s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
19s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
19s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
21s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
21s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 37s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/991/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt{color}
| {color:orange} hadoop-yarn-project/hadoop-yarn: The patch generated 9 new +
91 unchanged - 0 fixed = 100 total (was 91) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
|
[jira] [Commented] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346724#comment-17346724
]
Wei-Chiu Chuang commented on YARN-10772:
I don't know of any projects using the API. This issue was found by running the
Java API Compliance Checker tool.
(FWIW I am not aware of any projects under the CDP umbrella that use the API)
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Assignee: Akira Ajisaka
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> The startRange and finishRange changed type from LongRange to Range.
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346692#comment-17346692
]
Akira Ajisaka edited comment on YARN-10772 at 5/18/21, 8:51 AM:
I tried to write a patch to add the removed API, however, after adding the API,
the following code gets a compile error:
{code:title=TestClientRMService.java}
// Check tags
request = GetApplicationsRequest.newInstance(
ApplicationsRequestScope.ALL, null, null, null, null, null, null,
null, null);
{code}
{noformat}
[ERROR] both method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang3.Range,org.apache.commons.lang3.Range,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest and
method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang.math.LongRange,org.apache.commons.lang.math.LongRange,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest match
{noformat}
Perhaps it is better for users to keep the signature as-is. Hi [~weichiu], are
there any projects calling the old API?
was (Author: ajisakaa):
I tried to write a patch to add the removed API, however, after adding the API,
the following code gets a compile error:
{code:title=TestClientRMService.java}
// Check tags
request = GetApplicationsRequest.newInstance(
ApplicationsRequestScope.ALL, null, null, null, null, null, null,
null, null);
{code}
{noformat}
[ERROR] both method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang3.Range,org.apache.commons.lang3.Range,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest and
method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang.math.LongRange,org.apache.commons.lang.math.LongRange,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest match
{noformat}
Perhaps it is better for the user to keep the signature as-is. Hi [~weichiu],
are there any projects calling the old API?
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Assignee: Akira Ajisaka
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> The startRange and finishRange changed type from LongRange to Range.
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10759) Encapsulate queue config modes
[ https://issues.apache.org/jira/browse/YARN-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10759: Attachment: YARN-10759.005.patch > Encapsulate queue config modes > -- > > Key: YARN-10759 > URL: https://issues.apache.org/jira/browse/YARN-10759 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10759.001.patch, YARN-10759.002.patch, > YARN-10759.003.patch, YARN-10759.004.patch, YARN-10759.005.patch > > > Capacity Scheduler queues have three modes: > * relative/percentage > * weight > * absolute > Most of them have their own: > * validation logic > * config setting logic > * effective capacity calculation logic > These logics can be easily extracted and encapsulated in separate config mode > classes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10759) Encapsulate queue config modes
[ https://issues.apache.org/jira/browse/YARN-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346715#comment-17346715 ] Andras Gyori commented on YARN-10759: - Thank you [~pbacsko] for the review. This is indeed a valid concern, however, I can confirm that allowCapacitySum is only used in validation, which is in turn only used for ParentQueue. I think its safe for LeafQueue to have a false value, as it is only set in ParentQueue. I have realised that minRatioPerResource was cached for a parent. I have incorporated the same for the config modes. Also found out, that minRatioPerResource was overwritten by each node label before, which is a bug. Current revision fixes these issues. > Encapsulate queue config modes > -- > > Key: YARN-10759 > URL: https://issues.apache.org/jira/browse/YARN-10759 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10759.001.patch, YARN-10759.002.patch, > YARN-10759.003.patch, YARN-10759.004.patch, YARN-10759.005.patch > > > Capacity Scheduler queues have three modes: > * relative/percentage > * weight > * absolute > Most of them have their own: > * validation logic > * config setting logic > * effective capacity calculation logic > These logics can be easily extracted and encapsulated in separate config mode > classes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346692#comment-17346692
]
Akira Ajisaka commented on YARN-10772:
--
I tried to write a patch to add the removed API, however, after adding the API,
the following code gets a compile error:
{code:title=TestClientRMService.java}
// Check tags
request = GetApplicationsRequest.newInstance(
ApplicationsRequestScope.ALL, null, null, null, null, null, null,
null, null);
{code}
{noformat}
[ERROR] both method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang3.Range,org.apache.commons.lang3.Range,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest and
method
newInstance(org.apache.hadoop.yarn.api.protocolrecords.ApplicationsRequestScope,java.util.Set,java.util.Set,java.util.Set,java.util.Set,java.util.EnumSet,org.apache.commons.lang.math.LongRange,org.apache.commons.lang.math.LongRange,java.lang.Long)
in org.apache.hadoop.yarn.api.protocolrecords.GetApplicationsRequest match
{noformat}
Perhaps it is better for the user to keep the signature as-is. Hi [~weichiu],
are there any projects calling the old API?
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Assignee: Akira Ajisaka
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> The startRange and finishRange changed type from LongRange to Range.
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9698) [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler
[ https://issues.apache.org/jira/browse/YARN-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346668#comment-17346668 ] Szilard Nemeth commented on YARN-9698: -- Hi [~pbacsko], +1 on your idea about splitting. > [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler > > > Key: YARN-9698 > URL: https://issues.apache.org/jira/browse/YARN-9698 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Reporter: Weiwei Yang >Priority: Major > Labels: fs2cs > Attachments: FS-CS Migration.pdf > > > We see some users want to migrate from Fair Scheduler to Capacity Scheduler, > this Jira is created as an umbrella to track all related efforts for the > migration, the scope contains > * Bug fixes > * Add missing features > * Migration tools that help to generate CS configs based on FS, validate > configs etc > * Documents > this is part of CS component, the purpose is to make the migration process > smooth. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-10772:
Assignee: Akira Ajisaka
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Assignee: Akira Ajisaka
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> The startRange and finishRange changed type from LongRange to Range.
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
