[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346589#comment-17346589 ] chaosju commented on YARN-10771: Thank [~zhuqi] for review, add a new patch and changed the issue to "PATCH AVALABE" state. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Attachment: YARN-10771.002.patch > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch, YARN-10771.002.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10744) add example for yarn.federation.policy-manager-params
[
https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346584#comment-17346584
]
Hadoop QA commented on YARN-10744:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
0s{color} | {color:blue}{color} | {color:blue} markdownlint was not available.
{color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
42s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
41m 50s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 8s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green}{color} | {color:green} The patch does not generate
ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 30s{color} |
{color:black}{color} | {color:black}{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/990/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10744 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13025562/YARN-10744.001.patch |
| Optional Tests | dupname asflicense mvnsite markdownlint |
| uname | Linux b55c46924d6c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 110cda3de63 |
| Max. process+thread count | 722 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/990/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> add example for yarn.federation.policy-manager-params
> -
>
> Key: YARN-10744
> URL: https://issues.apache.org/jira/browse/YARN-10744
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Reporter: chaosju
>Assignee: chaosju
>Priority: Trivial
> Attachments: YARN-10744.001.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> add example for yarn.federation.policy-manager-params
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8363) Upgrade commons-lang version to 3.7 in hadoop-yarn-project
[ https://issues.apache.org/jira/browse/YARN-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346582#comment-17346582 ] Akira Ajisaka commented on YARN-8363: - Marking as incompatible change: YARN-10772 > Upgrade commons-lang version to 3.7 in hadoop-yarn-project > -- > > Key: YARN-8363 > URL: https://issues.apache.org/jira/browse/YARN-8363 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-8363.1.patch, YARN-8363.2.patch > > > commons-lang 2.6 is widely used. Let's upgrade to 3.6. > This jira is separated from HADOOP-10783. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8363) Upgrade commons-lang version to 3.7 in hadoop-yarn-project
[ https://issues.apache.org/jira/browse/YARN-8363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-8363: Hadoop Flags: Incompatible change,Reviewed (was: Reviewed) > Upgrade commons-lang version to 3.7 in hadoop-yarn-project > -- > > Key: YARN-8363 > URL: https://issues.apache.org/jira/browse/YARN-8363 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-8363.1.patch, YARN-8363.2.patch > > > commons-lang 2.6 is widely used. Let's upgrade to 3.6. > This jira is separated from HADOOP-10783. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346577#comment-17346577 ] Bilwa S T commented on YARN-10258: -- Thanks [~pbacsko] for committing this. please backport it to 3.3.1 also. > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Fix For: 3.4.0 > > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346575#comment-17346575 ] Qi Zhu edited comment on YARN-10771 at 5/18/21, 3:59 AM: - Thanks [~chaosju] for patch. I think we should change:"rm event queue size" to "rm dispatcher event queue size" and change "scheduler event queue size" to "scheduler dispatcher event queue size". Then you should submit the patch to trigger the jenkins. was (Author: zhuqi): Thanks [~chaosju] for patch. I think we should change:"rm event queue size" to "rm dispatcher queue size" and change "scheduler event queue size" to "scheduler dispatcher queue size". Then you should submit the patch to trigger the jenkins. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346575#comment-17346575 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for patch. I think we should change:"rm event queue size" to "rm dispatcher queue size" and change "scheduler event queue size" to "scheduler dispatcher queue size". Then you should submit the patch to trigger the jenkins. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10744) add example for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346565#comment-17346565 ] chaosju commented on YARN-10744: [~goirix] [~zhuqi] could you please take a look? > add example for yarn.federation.policy-manager-params > - > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Assignee: chaosju >Priority: Trivial > Attachments: YARN-10744.001.patch > > Time Spent: 20m > Remaining Estimate: 0h > > add example for yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10744) add example for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10744: --- Attachment: YARN-10744.001.patch > add example for yarn.federation.policy-manager-params > - > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Assignee: chaosju >Priority: Trivial > Attachments: YARN-10744.001.patch > > Time Spent: 20m > Remaining Estimate: 0h > > add example for yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10744) add example for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10744: --- Summary: add example for yarn.federation.policy-manager-params (was: add more doc for yarn.federation.policy-manager-params) > add example for yarn.federation.policy-manager-params > - > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Assignee: chaosju >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > add example for yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10744) add more doc for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10744: --- Description: add example for yarn.federation.policy-manager-params (was: add example of yarn.federation.policy-manager-params) > add more doc for yarn.federation.policy-manager-params > -- > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Assignee: chaosju >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > add example for yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10762) docker builds having problems with non root user on MAC OS
[ https://issues.apache.org/jira/browse/YARN-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10762: --- Description: Error logs： [exec] CC erasure_code/ec_base.lo [exec] CC raid/raid_base.lo [exec] CC erasure_code/ec_highlevel_func.lo [exec] CCLD libisal.la [exec] ar: `u' modifier ignored since `D' is the default (see `U') [exec] CCLD programs/igzip [exec] /usr/bin/install: cannot create regular file '/usr/lib/libisal.so.2.0.30': Permission denied [exec] make[2]: *** [install-libLTLIBRARIES] Error 1 [exec] make[1]: *** [install-am] Error 2 [exec] make: *** [install] Error 2 [exec] /bin/mkdir -p '/usr/lib' [exec] /bin/bash ./libtool --mode=install /usr/bin/install -c libisal.la '/usr/lib' [exec] libtool: install: /usr/bin/install -c .libs/libisal.so.2.0.30 /usr/lib/libisal.so.2.0.30 [exec] Makefile:2099: recipe for target 'install-libLTLIBRARIES' failed [exec] Makefile:3992: recipe for target 'install-am' failed [exec] Makefile:3986: recipe for target 'install' failed Full logs： [INFO] — maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-common — [INFO] Executing tasks main: [mkdir] Created dir: /home/chaosju/hadoop/hadoop-common-project/hadoop-common/target/test-dir [mkdir] Created dir: /home/chaosju/hadoop/hadoop-common-project/hadoop-common/target/test/data [INFO] Executed tasks [INFO] [INFO] — maven-enforcer-plugin:3.0.0-M1:enforce (enforce-os) @ hadoop-common — [INFO] Skipping Rule Enforcement. [INFO] [INFO] — protobuf-maven-plugin:0.5.1:compile (src-compile-protoc) @ hadoop-common — [INFO] Compiling 14 proto file(s) to /home/chaosju/hadoop/hadoop-common-project/hadoop-common/target/generated-sources/java [INFO] [INFO] — maven-antrun-plugin:1.7:run (isal) @ hadoop-common — [INFO] Executing tasks main: [exec] make --no-print-directory install-am [exec] CC erasure_code/ec_base.lo [exec] CC raid/raid_base.lo [exec] CC erasure_code/ec_highlevel_func.lo [exec] CCLD libisal.la [exec] ar: `u' modifier ignored since `D' is the default (see `U') [exec] CCLD programs/igzip [exec] /usr/bin/install: cannot create regular file '/usr/lib/libisal.so.2.0.30': Permission denied [exec] make[2]: *** [install-libLTLIBRARIES] Error 1 [exec] make[1]: *** [install-am] Error 2 [exec] make: *** [install] Error 2 [exec] /bin/mkdir -p '/usr/lib' [exec] /bin/bash ./libtool --mode=install /usr/bin/install -c libisal.la '/usr/lib' [exec] libtool: install: /usr/bin/install -c .libs/libisal.so.2.0.30 /usr/lib/libisal.so.2.0.30 [exec] Makefile:2099: recipe for target 'install-libLTLIBRARIES' failed [exec] Makefile:3992: recipe for target 'install-am' failed [exec] Makefile:3986: recipe for target 'install' failed [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main . SUCCESS [ 2.416 s] [INFO] Apache Hadoop Build Tools .. SUCCESS [ 27.978 s] [INFO] Apache Hadoop Project POM .. SUCCESS [ 2.207 s] [INFO] Apache Hadoop Annotations .. SUCCESS [ 1.917 s] [INFO] Apache Hadoop Assemblies ... SUCCESS [ 0.631 s] [INFO] Apache Hadoop Project Dist POM . SUCCESS [ 2.373 s] [INFO] Apache Hadoop Maven Plugins SUCCESS [ 5.202 s] [INFO] Apache Hadoop MiniKDC .. SUCCESS [ 1.915 s] [INFO] Apache Hadoop Auth . SUCCESS [ 8.674 s] [INFO] Apache Hadoop Auth Examples SUCCESS [ 2.773 s] [INFO] tq-security SUCCESS [ 2.880 s] [INFO] Apache Hadoop Common ... FAILURE [ 5.035 s] [INFO] Apache Hadoop NFS .. SKIPPED [INFO] Apache Hadoop KMS .. SKIPPED [INFO] Apache Hadoop Common Project ... SKIPPED [INFO] Apache Hadoop HDFS Client .. SKIPPED [INFO] Apache Hadoop HDFS . SKIPPED [INFO] Apache Hadoop HDFS Native Client ... SKIPPED [INFO] Apache Hadoop HttpFS ... SKIPPED [INFO] Apache Hadoop HDFS-NFS . SKIPPED [INFO] Apache Hadoop HDFS-RBF . SKIPPED [INFO] Apache Hadoop HDFS Project . SKIPPED [INFO] Apache Hadoop YARN . SKIPPED [INFO] Apache Hadoop YARN API . SKIPPED [INFO] Apache Hadoop YARN Common .. SKIPPED [INFO] Apache Hadoop YARN Registry SKIPPED [INFO] Apache Hadoop YARN Server .. SKIPPED [INFO] Apache Hadoop YARN Server Common ... SKIPPED [INFO] Apache Hadoop YARN NodeManager . SKIPPED [INFO] Apache Hadoop YARN Web Proxy
[jira] [Commented] (YARN-10712) Fix word errors in class comments
[
https://issues.apache.org/jira/browse/YARN-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346552#comment-17346552
]
chaosju commented on YARN-10712:
YARN-9661 fixed
> Fix word errors in class comments
> -
>
> Key: YARN-10712
> URL: https://issues.apache.org/jira/browse/YARN-10712
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: amrmproxy
>Affects Versions: 3.2.1
>Reporter: chaosju
>Priority: Trivial
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Class of LocalityMulticastAMRMProxyPolicy’s class comment has error word
> “{color:#FF}thsi{color}”
>
> Part of the comment：
> Rack localized \{@link ResourceRequest}s are forwarded to the RMs that owns
> the corresponding rack. Note that in some deployments each rack could be
> striped across multiple RMs. {color:#ff}Thsi{color} policy respects
> that. If the
> \{@link SubClusterResolver} cannot resolve this rack we default to
> forwarding
> the \{@link ResourceRequest} to the home sub-cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10712) Fix word errors in class comments
[
https://issues.apache.org/jira/browse/YARN-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chaosju resolved YARN-10712.
Resolution: Won't Do
> Fix word errors in class comments
> -
>
> Key: YARN-10712
> URL: https://issues.apache.org/jira/browse/YARN-10712
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: amrmproxy
>Affects Versions: 3.2.1
>Reporter: chaosju
>Priority: Trivial
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Class of LocalityMulticastAMRMProxyPolicy’s class comment has error word
> “{color:#FF}thsi{color}”
>
> Part of the comment：
> Rack localized \{@link ResourceRequest}s are forwarded to the RMs that owns
> the corresponding rack. Note that in some deployments each rack could be
> striped across multiple RMs. {color:#ff}Thsi{color} policy respects
> that. If the
> \{@link SubClusterResolver} cannot resolve this rack we default to
> forwarding
> the \{@link ResourceRequest} to the home sub-cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10744) add more doc for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju reassigned YARN-10744: -- Assignee: chaosju > add more doc for yarn.federation.policy-manager-params > -- > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Assignee: chaosju >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > add example of yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Deleted] (YARN-10773) Situs Slot Online Gacor Hari Ini 2021
[ https://issues.apache.org/jira/browse/YARN-10773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka deleted YARN-10773: - > Situs Slot Online Gacor Hari Ini 2021 > - > > Key: YARN-10773 > URL: https://issues.apache.org/jira/browse/YARN-10773 > Project: Hadoop YARN > Issue Type: New Feature > Environment: Karakter Liar yang bersimbol lembah berbatu ini punya > dua fungsi. Paling awal, bila digunakan pada putaran biasa akan bisa menjadi > pengganti semua gambar yang ada ketika dibutuhkan. Satu lagi simbol [judi > slot|https://www.arnobbd.com] yang dapat menambah hasil kemenangan adalah > munculnya bonus gambar yang bergambar koin emas banteng, yang bila didapat > apabila 3 karakter saja sudah langsung langsung memberi dua puluh dua kali > dan bisa mendapatkan bayaran bertaruh dengan 100x, dan ketika free spin yang > diraih tersebut dimainkan, akan diperoleh fungsi lainnya dari dapat wild, > yaitu akan meraih penambahan 2x, tiga kali dan lima kali kepada hasil > peraihan yang sudah lebih dulu diraih. Dari gambar wild yang terdapat > perhitungan ini sangat memungkinkan untuk memperoleh lebih dari satu karakter > pada tiap permainan, dan akan bertambah terus pada perhitungannya. Benar > benar perolehan yang berganda seperti yang dijanjikan, yang bisa saja > memenangkan Sensasional. > Harga yang akan dibayarkan oleh pemain [slot online|http://5.77.39.126] > seperti enam gambar yang sama muncul berbarengan saat 1 putaran. Tiap simbol > memiliki jumlah nominal yang berbeda, apabila seperti Simbol Buffalo King > untuk memperoleh enam gambar yang sama, harga yang dapat diraih sebesar dua > puluh kali nilai uang taruhan, ini yakni nilai yang tertinggi didapat oleh > simbol Banteng, yang akan lain jumlahnya bila karakter serigala untuk datang > 6 karakter yang sama, harga yang akan diperoleh sebesar 1,5x nominal taruhan. > Bila peserta sudah mengatur uang taruhan dengan menekan tanda kurang tambah > saat mengatur taruhan dengan nilai besar, maka dapat pemain hitung berapa > harga yang akan diperoleh. kita dapat mengklik tanda -+ untuk taruhan kecil > dan menaikkan taruhan, atau krusor dapat digeser geser ke kiri untuk > mengecilkan taruhan dan geser kesamping kanan menambah taruhan. semua pemain > slot memainkan nilai kecil untuk memanaskan mesin slot. awal putaran untuk > mempelajari untuk langkah selanjutnya. > Fitur susun kebawah, orang kenal dengan mode runtuh, menghilangkan karakter > yang menghasilkan kombinasi menang dan menggantinya dengan simbol grafis yang > bergeser menggantikan dari geseran atas. Karakter yang sisa di layar tampilan > setelah dapat kemenangan jatuh turun bergeser kebawah gulungan. Dengan maksud > lain, anda akan bermain dengan grafis baru setelah setiap kemenangan > sepanjang bertaruh.Urutan gesernya simbol berakhir ketika sudah berhenti > setelah memperoleh kemenangan dikarenakan reel jatuh bergeser kebawah. >Reporter: Damien Sapien >Priority: Minor > > h1. Slot Online Gacor Hari Ini 2021 > h2. Main menang di Situs Slot Online Sering Jackpot > Kagak dikira ternyata datang idul fitri kembali,tertinggal berapa hari kita > semua untuk masuk ke waktunya saling minta maaf sesudah selagi satu bulan > menjalani amalan menahan emosi. Dengan menyesal hari fitri sekarang ini kita > masuki lewat status yang persis ibarat hari fitri warsa yang lewat. virus > berbahaya covid bersisa menemani suasana hari fitri sekarang ini. anda tengah > harus senantiasa menjamin ruang maka menyingkir menerapkan kerubungan besok > sedang tatkala berjumpa lewat keluarga. Memang mau tak mau memerlukan > kesadaran pula wawasan luas berpikir menghadapi lingkungan ini. Mempunyai > membela pula anti disekitar semua orang tiap-tiap menyimpan ragam paham diri > sendiri akan mengerjakannya. Bagaimanapun cara tingkah laku untuk mengerjakan > berhadapan hari fitri kemudian. Buktikan tata cara Kesegaran mau tak mau > selamanya diamalkan tidak teledor. Mambasahi tangan sebelum saling salam > harus dilakukan, atau simpan alat pancar cairan pembasmi dekat baju bisa > memungkinkan menjadi bersih ulang telapak kalian. Silih berganti saling tos > dapat kalian tukar lewat beradu cengkram tangan saja sudah buat silih > berganti menghormati kala ini. Kalian memahami penghalang aksi yang masih > semua orang laksanakan. Mengenakan kedok penjaga wajahpun tidak lalai. Bakal > mengurangi menyebarnya bakteri dari tumpahan air ludah bisa terbentuk tatkala > semua orang sama sama bertutur, dan mencermati langkah nyaman untuk > meningkatkan kemungkinan tidak bisa tergapai percikan ke sisi anda. Bila tiga > tata cara kesehatan ini digelar seraya teratur, bertemu lewat sanak saudara > jua diinginkan berlangsung gampang jangan ada konsekuensi di anda termuat 2 > pekan setelah hari raya, menjadi
[jira] [Updated] (YARN-10744) add more doc for yarn.federation.policy-manager-params
[ https://issues.apache.org/jira/browse/YARN-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10744: --- Labels: (was: pull-request-available) > add more doc for yarn.federation.policy-manager-params > -- > > Key: YARN-10744 > URL: https://issues.apache.org/jira/browse/YARN-10744 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: chaosju >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > > add example of yarn.federation.policy-manager-params -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10763) Add the number of containers assigned per second metrics to ClusterMetrics
[ https://issues.apache.org/jira/browse/YARN-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346546#comment-17346546 ] chaosju commented on YARN-10763: Thanks [~pbacsko] [~zhuqi] for the review. It's my first contribution to Hadoop. This has great sentimental value for me. > Add the number of containers assigned per second metrics to ClusterMetrics > --- > > Key: YARN-10763 > URL: https://issues.apache.org/jira/browse/YARN-10763 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: chaosju >Assignee: chaosju >Priority: Minor > Fix For: 3.4.0 > > Attachments: YARN-10763.001.patch, YARN-10763.002.patch, > YARN-10763.003.patch, YARN-10763.004.patch, YARN-10763.005.patch, > YARN-10763.006.patch, YARN-10763.007.patch, YARN-10763.008.patch, > screenshot-1.png > > > It'd be good to have ContainerAssignedNum/Second in ClusterMetrics for > measuring cluster throughput. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346543#comment-17346543
]
Akira Ajisaka commented on YARN-10772:
--
Hi [~weichiu], thank you for your report. LongRange comes from commons-lang
2.x, so we need to add back commons-lang 2.x dependency to fix the
compatibility.
BTW, in YARN-8363, we upgraded the signature to use commons-lang 3.x, which is
still not good. If we upgrade commons-lang 4.x in the future, we break
compatibility again. Instead, I want to create a public API to avoid the use of
external library.
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> The startRange and finishRange changed type from LongRange to Range.
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Attachment: YARN-10763.001.patch > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > Attachments: YARN-10763.001.patch > > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10773) Situs Slot Online Gacor Hari Ini 2021
Damien Sapien created YARN-10773: Summary: Situs Slot Online Gacor Hari Ini 2021 Key: YARN-10773 URL: https://issues.apache.org/jira/browse/YARN-10773 Project: Hadoop YARN Issue Type: New Feature Components: api Affects Versions: 2.0.5-alpha Environment: Karakter Liar yang bersimbol lembah berbatu ini punya dua fungsi. Paling awal, bila digunakan pada putaran biasa akan bisa menjadi pengganti semua gambar yang ada ketika dibutuhkan. Satu lagi simbol [judi slot|https://www.arnobbd.com] yang dapat menambah hasil kemenangan adalah munculnya bonus gambar yang bergambar koin emas banteng, yang bila didapat apabila 3 karakter saja sudah langsung langsung memberi dua puluh dua kali dan bisa mendapatkan bayaran bertaruh dengan 100x, dan ketika free spin yang diraih tersebut dimainkan, akan diperoleh fungsi lainnya dari dapat wild, yaitu akan meraih penambahan 2x, tiga kali dan lima kali kepada hasil peraihan yang sudah lebih dulu diraih. Dari gambar wild yang terdapat perhitungan ini sangat memungkinkan untuk memperoleh lebih dari satu karakter pada tiap permainan, dan akan bertambah terus pada perhitungannya. Benar benar perolehan yang berganda seperti yang dijanjikan, yang bisa saja memenangkan Sensasional. Harga yang akan dibayarkan oleh pemain [slot online|http://5.77.39.126] seperti enam gambar yang sama muncul berbarengan saat 1 putaran. Tiap simbol memiliki jumlah nominal yang berbeda, apabila seperti Simbol Buffalo King untuk memperoleh enam gambar yang sama, harga yang dapat diraih sebesar dua puluh kali nilai uang taruhan, ini yakni nilai yang tertinggi didapat oleh simbol Banteng, yang akan lain jumlahnya bila karakter serigala untuk datang 6 karakter yang sama, harga yang akan diperoleh sebesar 1,5x nominal taruhan. Bila peserta sudah mengatur uang taruhan dengan menekan tanda kurang tambah saat mengatur taruhan dengan nilai besar, maka dapat pemain hitung berapa harga yang akan diperoleh. kita dapat mengklik tanda -+ untuk taruhan kecil dan menaikkan taruhan, atau krusor dapat digeser geser ke kiri untuk mengecilkan taruhan dan geser kesamping kanan menambah taruhan. semua pemain slot memainkan nilai kecil untuk memanaskan mesin slot. awal putaran untuk mempelajari untuk langkah selanjutnya. Fitur susun kebawah, orang kenal dengan mode runtuh, menghilangkan karakter yang menghasilkan kombinasi menang dan menggantinya dengan simbol grafis yang bergeser menggantikan dari geseran atas. Karakter yang sisa di layar tampilan setelah dapat kemenangan jatuh turun bergeser kebawah gulungan. Dengan maksud lain, anda akan bermain dengan grafis baru setelah setiap kemenangan sepanjang bertaruh.Urutan gesernya simbol berakhir ketika sudah berhenti setelah memperoleh kemenangan dikarenakan reel jatuh bergeser kebawah. Reporter: Damien Sapien Fix For: 2.0.5-alpha h1. Slot Online Gacor Hari Ini 2021 h2. Main menang di Situs Slot Online Sering Jackpot Kagak dikira ternyata datang idul fitri kembali,tertinggal berapa hari kita semua untuk masuk ke waktunya saling minta maaf sesudah selagi satu bulan menjalani amalan menahan emosi. Dengan menyesal hari fitri sekarang ini kita masuki lewat status yang persis ibarat hari fitri warsa yang lewat. virus berbahaya covid bersisa menemani suasana hari fitri sekarang ini. anda tengah harus senantiasa menjamin ruang maka menyingkir menerapkan kerubungan besok sedang tatkala berjumpa lewat keluarga. Memang mau tak mau memerlukan kesadaran pula wawasan luas berpikir menghadapi lingkungan ini. Mempunyai membela pula anti disekitar semua orang tiap-tiap menyimpan ragam paham diri sendiri akan mengerjakannya. Bagaimanapun cara tingkah laku untuk mengerjakan berhadapan hari fitri kemudian. Buktikan tata cara Kesegaran mau tak mau selamanya diamalkan tidak teledor. Mambasahi tangan sebelum saling salam harus dilakukan, atau simpan alat pancar cairan pembasmi dekat baju bisa memungkinkan menjadi bersih ulang telapak kalian. Silih berganti saling tos dapat kalian tukar lewat beradu cengkram tangan saja sudah buat silih berganti menghormati kala ini. Kalian memahami penghalang aksi yang masih semua orang laksanakan. Mengenakan kedok penjaga wajahpun tidak lalai. Bakal mengurangi menyebarnya bakteri dari tumpahan air ludah bisa terbentuk tatkala semua orang sama sama bertutur, dan mencermati langkah nyaman untuk meningkatkan kemungkinan tidak bisa tergapai percikan ke sisi anda. Bila tiga tata cara kesehatan ini digelar seraya teratur, bertemu lewat sanak saudara jua diinginkan berlangsung gampang jangan ada konsekuensi di anda termuat 2 pekan setelah hari raya, menjadi hitungan tepat adakah penyakit yang bergerak kali itu pada raga kalian. adat Kesegaran ini sanggup kita semua awali seorang diri, supaya sanak yang bercanda kali lebaran untuk tercegah penularan penyakit
[jira] [Updated] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated YARN-10772:
---
Description:
YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
Unfortunately, it changed the API signature of
{code:java}
/**
*
* The request from clients to get a report of Applications matching the
* giving application types in the cluster from the
* ResourceManager.
*
*
* @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
*
* Setting any of the parameters to null, would just disable that
* filter
*
* @param scope {@link ApplicationsRequestScope} to filter by
* @param users list of users to filter by
* @param queues list of scheduler queues to filter by
* @param applicationTypes types of applications
* @param applicationTags application tags to filter by
* @param applicationStates application states to filter by
* @param startRange range of application start times to filter by
* @param finishRange range of application finish times to filter by
* @param limit number of applications to limit to
* @return {@link GetApplicationsRequest} to be used with
* {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
*/
@Public
@Stable
public static GetApplicationsRequest newInstance(
ApplicationsRequestScope scope,
Set users,
Set queues,
Set applicationTypes,
Set applicationTags,
EnumSet applicationStates,
Range startRange,
Range finishRange,
Long limit) { {code}
The startRange and finishRange changed type from LongRange to Range.
It could cause problems when migrating applications, for example, from Hadoop
3.1 to 3.3.
was:
YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
Unfortunately, it changed the API signature of
{code:java}
/**
*
* The request from clients to get a report of Applications matching the
* giving application types in the cluster from the
* ResourceManager.
*
*
* @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
*
* Setting any of the parameters to null, would just disable that
* filter
*
* @param scope {@link ApplicationsRequestScope} to filter by
* @param users list of users to filter by
* @param queues list of scheduler queues to filter by
* @param applicationTypes types of applications
* @param applicationTags application tags to filter by
* @param applicationStates application states to filter by
* @param startRange range of application start times to filter by
* @param finishRange range of application finish times to filter by
* @param limit number of applications to limit to
* @return {@link GetApplicationsRequest} to be used with
* {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
*/
@Public
@Stable
public static GetApplicationsRequest newInstance(
ApplicationsRequestScope scope,
Set users,
Set queues,
Set applicationTypes,
Set applicationTags,
EnumSet applicationStates,
Range startRange,
Range finishRange,
Long limit) { {code}
It could cause problems when migrating applications, for example, from Hadoop
3.1 to 3.3.
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set
[jira] [Commented] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
[
https://issues.apache.org/jira/browse/YARN-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346510#comment-17346510
]
Wei-Chiu Chuang commented on YARN-10772:
CC: [~tasanuma] [~aajisaka]
> Stable API GetApplicationsRequest#newInstance compatibility broken by
> YARN-8363
> ---
>
> Key: YARN-10772
> URL: https://issues.apache.org/jira/browse/YARN-10772
> Project: Hadoop YARN
> Issue Type: Bug
> Components: api
>Affects Versions: 3.2.0
>Reporter: Wei-Chiu Chuang
>Priority: Major
>
> YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
>
> Unfortunately, it changed the API signature of
> {code:java}
> /**
> *
> * The request from clients to get a report of Applications matching the
> * giving application types in the cluster from the
> * ResourceManager.
> *
> *
> * @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
> *
> * Setting any of the parameters to null, would just disable that
> * filter
> *
> * @param scope {@link ApplicationsRequestScope} to filter by
> * @param users list of users to filter by
> * @param queues list of scheduler queues to filter by
> * @param applicationTypes types of applications
> * @param applicationTags application tags to filter by
> * @param applicationStates application states to filter by
> * @param startRange range of application start times to filter by
> * @param finishRange range of application finish times to filter by
> * @param limit number of applications to limit to
> * @return {@link GetApplicationsRequest} to be used with
> * {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
> */
> @Public
> @Stable
> public static GetApplicationsRequest newInstance(
> ApplicationsRequestScope scope,
> Set users,
> Set queues,
> Set applicationTypes,
> Set applicationTags,
> EnumSet applicationStates,
> Range startRange,
> Range finishRange,
> Long limit) { {code}
> It could cause problems when migrating applications, for example, from Hadoop
> 3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10772) Stable API GetApplicationsRequest#newInstance compatibility broken by YARN-8363
Wei-Chiu Chuang created YARN-10772:
--
Summary: Stable API GetApplicationsRequest#newInstance
compatibility broken by YARN-8363
Key: YARN-10772
URL: https://issues.apache.org/jira/browse/YARN-10772
Project: Hadoop YARN
Issue Type: Bug
Components: api
Affects Versions: 3.2.0
Reporter: Wei-Chiu Chuang
YARN-8363 migrated our usage of commons-lang to commons-lang3 in 3.2.0.
Unfortunately, it changed the API signature of
{code:java}
/**
*
* The request from clients to get a report of Applications matching the
* giving application types in the cluster from the
* ResourceManager.
*
*
* @see ApplicationClientProtocol#getApplications(GetApplicationsRequest)
*
* Setting any of the parameters to null, would just disable that
* filter
*
* @param scope {@link ApplicationsRequestScope} to filter by
* @param users list of users to filter by
* @param queues list of scheduler queues to filter by
* @param applicationTypes types of applications
* @param applicationTags application tags to filter by
* @param applicationStates application states to filter by
* @param startRange range of application start times to filter by
* @param finishRange range of application finish times to filter by
* @param limit number of applications to limit to
* @return {@link GetApplicationsRequest} to be used with
* {@link ApplicationClientProtocol#getApplications(GetApplicationsRequest)}
*/
@Public
@Stable
public static GetApplicationsRequest newInstance(
ApplicationsRequestScope scope,
Set users,
Set queues,
Set applicationTypes,
Set applicationTags,
EnumSet applicationStates,
Range startRange,
Range finishRange,
Long limit) { {code}
It could cause problems when migrating applications, for example, from Hadoop
3.1 to 3.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346397#comment-17346397 ] Brahma Reddy Battula commented on YARN-10745: - [~dmmkr] Thanks for reporting and working on this. At first glance IMO, could have split to module wise for easier maintain/clean.. let me know your guys thought on this. > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch, YARN-10745.005.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346396#comment-17346396 ] Peter Bacsko commented on YARN-10258: - +1 Thanks for the patch [~gb.ana...@gmail.com] and [~BilwaST] / [~zhuqi] for the review, committed to trunk. > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2774) shared cache service should authorize calls properly
[
https://issues.apache.org/jira/browse/YARN-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346266#comment-17346266
]
Hadoop QA commented on YARN-2774:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
57s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:blue}0{color} | {color:blue} markdownlint {color} | {color:blue} 0m
1s{color} | {color:blue}{color} | {color:blue} markdownlint was not available.
{color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 3
new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 12m
26s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
43s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
39s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
7s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
30s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
24m 20s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
38s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
22s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 40m
44s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 8m
29s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m
57s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m
57s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
6s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 19m
6s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 56s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/989/artifact/out/diff-checkstyle-root.txt{color}
| {color:orange} root: The patch generated 1 new + 238 unchanged - 1 fixed =
239 total (was 239) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
24s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} |
[jira] [Updated] (YARN-10727) ParentQueue does not validate the queue on removal
[ https://issues.apache.org/jira/browse/YARN-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Gyori updated YARN-10727: Attachment: YARN-10727.001.patch > ParentQueue does not validate the queue on removal > -- > > Key: YARN-10727 > URL: https://issues.apache.org/jira/browse/YARN-10727 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Major > Attachments: YARN-10727.001.patch > > > With the addition of YARN-10532 ParentQueue has a public method, removeQueue, > which allows the deletion of a queue at runtime. However, there is no > validation regarding the queue which is to be removed, therefore it is > possible to remove a queue from the CSQueueManager that is not a child of the > ParentQueue. Since it is a public method, there must be validations such as: > * check, if the parent of the queue to be removed is the current ParentQueue > * check, if the parent actually contains the queue in its childQueues > collection -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346181#comment-17346181 ] Qi Zhu commented on YARN-10771: --- Thanks [~chaosju] for this. This is useful for user to know the event load, we can add it to cluster metrics. I will help review this. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346178#comment-17346178 ] D M Murali Krishna Reddy commented on YARN-10745: - [~ebadger] Can you review the patch, so that it can be merged to 3.3.1 Thanks > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch, YARN-10745.005.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346154#comment-17346154 ] Peter Bacsko commented on YARN-10258: - [~BilwaST] yes, I'll check this out. > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2774) shared cache service should authorize calls properly
[ https://issues.apache.org/jira/browse/YARN-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-2774: - Labels: pull-request-available (was: ) > shared cache service should authorize calls properly > > > Key: YARN-2774 > URL: https://issues.apache.org/jira/browse/YARN-2774 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sangjin Lee >Assignee: zhenzhao wang >Priority: Major > Labels: pull-request-available > Attachments: YARN-2774.001.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The shared cache manager (SCM) services should authorize calls properly. > Currently, the uploader service (done in YARN-2186) does not authorize calls > to notify the SCM on newly uploaded resource. Proper security/authorization > needs to be done in this RPC call. Also, the use/release calls (done in > YARN-2188) and the scmAdmin commands (done in YARN-2189) are not properly > authorized. The SCM UI done in YARN-2203 as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10258) Add metrics for 'ApplicationsRunning' in NodeManager
[ https://issues.apache.org/jira/browse/YARN-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346112#comment-17346112 ] Bilwa S T commented on YARN-10258: -- Thanks [~gb.ana...@gmail.com] for patch. Looks good to me. [~pbacsko] can you please commit this? > Add metrics for 'ApplicationsRunning' in NodeManager > > > Key: YARN-10258 > URL: https://issues.apache.org/jira/browse/YARN-10258 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.1.3 >Reporter: ANANDA G B >Assignee: ANANDA G B >Priority: Minor > Attachments: YARN-10258-001.patch, YARN-10258-002.patch, > YARN-10258-003.patch, YARN-10258-005.patch, YARN-10258-006.patch, > YARN-10258-007.patch, YARN-10258-008.patch, YARN-10258-009.patch, > YARN-10258-010.patch, YARN-10258_004.patch > > > Add metrics for 'ApplicationsRunning' in NodeManagers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10763) Add the number of containers assigned per second metrics to ClusterMetrics
[ https://issues.apache.org/jira/browse/YARN-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346098#comment-17346098 ] Peter Bacsko commented on YARN-10763: - +1 Thanks [~chaosju] for the patch and [~zhuqi] for the review. Committed to trunk. > Add the number of containers assigned per second metrics to ClusterMetrics > --- > > Key: YARN-10763 > URL: https://issues.apache.org/jira/browse/YARN-10763 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: chaosju >Assignee: chaosju >Priority: Minor > Attachments: YARN-10763.001.patch, YARN-10763.002.patch, > YARN-10763.003.patch, YARN-10763.004.patch, YARN-10763.005.patch, > YARN-10763.006.patch, YARN-10763.007.patch, YARN-10763.008.patch, > screenshot-1.png > > > It'd be good to have ContainerAssignedNum/Second in ClusterMetrics for > measuring cluster throughput. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10763) Add the number of containers assigned per second metrics to ClusterMetrics
[ https://issues.apache.org/jira/browse/YARN-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Bacsko updated YARN-10763: Summary: Add the number of containers assigned per second metrics to ClusterMetrics (was: add the speed of containers assigned metrics to ClusterMetrics) > Add the number of containers assigned per second metrics to ClusterMetrics > --- > > Key: YARN-10763 > URL: https://issues.apache.org/jira/browse/YARN-10763 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: chaosju >Assignee: chaosju >Priority: Minor > Attachments: YARN-10763.001.patch, YARN-10763.002.patch, > YARN-10763.003.patch, YARN-10763.004.patch, YARN-10763.005.patch, > YARN-10763.006.patch, YARN-10763.007.patch, YARN-10763.008.patch, > screenshot-1.png > > > It'd be good to have ContainerAssignedNum/Second in ClusterMetrics for > measuring cluster throughput. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10111) In Federation cluster Distributed Shell Application submission fails as YarnClient#getQueueInfo is not implemented
[ https://issues.apache.org/jira/browse/YARN-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346094#comment-17346094 ] Yuan LUO commented on YARN-10111: - Hi [~zhuqi] Is there any new discussion progress on this patch？ I have a question，in the case of yarn federation, queue names of all subclusters need to be consistent? Looking forward to your reply, thanks！ > In Federation cluster Distributed Shell Application submission fails as > YarnClient#getQueueInfo is not implemented > -- > > Key: YARN-10111 > URL: https://issues.apache.org/jira/browse/YARN-10111 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sushanta Sen >Assignee: Qi Zhu >Priority: Blocker > Attachments: YARN-10111.001.patch > > > In Federation cluster Distributed Shell Application submission fails as > YarnClient#getQueueInfo is not implemented. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10465) Support getClusterNodes, getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346090#comment-17346090 ] Yuan LUO edited comment on YARN-10465 at 5/17/21, 11:21 AM: Hi [~ayushsaxena] [~tangzhankun] [~zhuqi] Can you help review this，Thanks！ was (Author: luoyuan): Hi [~ayushsaxena] [~tangzhankun] [~zhuqi]] Can you help review this，Thanks！ > Support getClusterNodes, getNodeToLabels, getLabelsToNodes, > getClusterNodeLabels API's for Federation > - > > Key: YARN-10465 > URL: https://issues.apache.org/jira/browse/YARN-10465 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10465.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10465) Support getClusterNodes, getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346090#comment-17346090 ] Yuan LUO commented on YARN-10465: - Hi [~ayushsaxena] [~tangzhankun] [~zhuqi]] Can you help review this，Thanks！ > Support getClusterNodes, getNodeToLabels, getLabelsToNodes, > getClusterNodeLabels API's for Federation > - > > Key: YARN-10465 > URL: https://issues.apache.org/jira/browse/YARN-10465 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10465.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346085#comment-17346085 ] chaosju commented on YARN-10771: [~zhuqi] [~pbacsko] what's your opinion about this? > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju reassigned YARN-10771: -- Assignee: chaosju > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Assignee: chaosju >Priority: Major > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Description: Add cluster metric for size of Scheduler event queue and RM event queue, This lets us know the load of the RM and convenient monitoring the metrics. was:Add cluster metric for size of Scheduler Event Queue and RM Event Queue, This lets us know the load of the RM and convenient monitoring the metrics > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Priority: Major > > Add cluster metric for size of Scheduler event queue and RM event queue, This > lets us know the load of the RM and convenient monitoring the metrics. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Description: Add cluster metric for size of Scheduler Event Queue and RM Event Queue, This lets us know the load of the RM and convenient monitoring the metrics (was: add ) > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Priority: Major > > Add cluster metric for size of Scheduler Event Queue and RM Event Queue, This > lets us know the load of the RM and convenient monitoring the metrics -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Description: add (was: add scheduler event queue ) > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Priority: Major > > add -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Description: It'd be good to have counts for each event type in RM async dispatcher and scheduler async dispatcher. > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Priority: Major > > It'd be good to have counts for each event type in RM async dispatcher and > scheduler async dispatcher. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
[ https://issues.apache.org/jira/browse/YARN-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaosju updated YARN-10771: --- Description: add scheduler event queue (was: It'd be good to have counts for each event type in RM async dispatcher and scheduler async dispatcher.) > Add cluster metric for size of SchedulerEventQueue and RMEventQueue > --- > > Key: YARN-10771 > URL: https://issues.apache.org/jira/browse/YARN-10771 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: chaosju >Priority: Major > > add scheduler event queue -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10487) Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346079#comment-17346079 ] Yuan LUO commented on YARN-10487: - Hi [~dmmkr] I don't think it's necessary for these APIs to request all subclusters, you can just send request to homesubcluster, what do you think？ > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation > --- > > Key: YARN-10487 > URL: https://issues.apache.org/jira/browse/YARN-10487 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10487.001.patch > > > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10771) Add cluster metric for size of SchedulerEventQueue and RMEventQueue
chaosju created YARN-10771: -- Summary: Add cluster metric for size of SchedulerEventQueue and RMEventQueue Key: YARN-10771 URL: https://issues.apache.org/jira/browse/YARN-10771 Project: Hadoop YARN Issue Type: Sub-task Reporter: chaosju -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8173) [Router] Implement missing FederationClientInterceptor#getApplications()
[
https://issues.apache.org/jira/browse/YARN-8173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346073#comment-17346073
]
Yuan LUO commented on YARN-8173:
Hi [~dmmkr] [~giovanni.fumarola]
We want use Livy to run in YARN Federation, the current implementation relies
on this interface and is blocked. Is there any new progress about this
patch？Looking forward to your reply, thanks！
> [Router] Implement missing FederationClientInterceptor#getApplications()
>
>
> Key: YARN-8173
> URL: https://issues.apache.org/jira/browse/YARN-8173
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-8173.001.patch, YARN-8173.002.patch,
> YARN-8173.003.patch, YARN-8173.004.patch, YARN-8173.005.patch,
> YARN-8173.006.patch, YARN-8173.007.patch, YARN-8173.008.patch
>
>
> oozie dependent method Implement
> {code:java}
> getApplications()
> getDeglationToken()
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10555) Missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-10555:
-
Fix Version/s: 2.10.2
> Missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0, 3.3.1, 3.1.5, 2.10.2, 3.2.3
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10555) Missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17346018#comment-17346018
]
Qi Zhu commented on YARN-10555:
---
Thanks [~aajisaka] for backport.
> Missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0, 3.3.1, 3.1.5, 3.2.3
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10555) Missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-10555:
-
Summary: Missing access check before getAppAttempts (was: missing access
check before getAppAttempts)
> Missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0, 3.3.1, 3.1.5, 3.2.3
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10555) missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-10555:
-
Fix Version/s: 3.2.3
3.1.5
3.3.1
> missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0, 3.3.1, 3.1.5, 3.2.3
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10555) missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345998#comment-17345998
]
Akira Ajisaka commented on YARN-10555:
--
Thank you [~zhuqi] and [~xiaoheipangzi]. I'm backporting to the lower branches,
will update the fix versions.
> missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10555) missing access check before getAppAttempts
[
https://issues.apache.org/jira/browse/YARN-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345995#comment-17345995
]
Qi Zhu commented on YARN-10555:
---
It was merged by [~aajisaka] , i just make it fixed.
Thanks [~xiaoheipangzi] for contribution, [~aajisaka] for merge.
> missing access check before getAppAttempts
> ---
>
> Key: YARN-10555
> URL: https://issues.apache.org/jira/browse/YARN-10555
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Reporter: lujie
>Assignee: lujie
>Priority: Critical
> Labels: pull-request-available, security
> Fix For: 3.4.0
>
> Attachments: YARN-10555_1.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It seems that we miss a security check before getAppAttempts, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1127]
> thus we can get the some sensitive information, like logs link.
> {code:java}
> application_1609318368700_0002 belong to user2
> user1@hadoop11$ curl --negotiate -u :
> http://hadoop11:8088/ws/v1/cluster/apps/application_1609318368700_0002/appattempts/|jq
> {
> "appAttempts": {
> "appAttempt": [
> {
> "id": 1,
> "startTime": 1609318411566,
> "containerId": "container_1609318368700_0002_01_01",
> "nodeHttpAddress": "hadoop12:8044",
> "nodeId": "hadoop12:36831",
> "logsLink":
> "http://hadoop12:8044/node/containerlogs/container_1609318368700_0002_01_01/user2;,
> "blacklistedNodes": "",
> "nodesBlacklistedBySystem": ""
> }
> ]
> }
> }
> {code}
> Other apis, like getApps and getApp, has access check like "hasAccess(app,
> hsr)", they would hide the logs link if the appid do not belong to query
> user, see
> [https://github.com/apache/hadoop/blob/513f1995adc9b73f9c7f4c7beb89725b51b313ac/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java#L1098]
> We need add hasAccess(app, hsr) for getAppAttempts.
>
> Besides, at
> [https://github.com/apache/hadoop/blob/580a6a75a3e3d3b7918edeffd6e93fc211166884/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMAppBlock.java#L145]
> it seems that we have a access check in its caller, so now i pass "true" to
> AppAttemptInfo in the patch.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10725) Backport YARN-10120 to branch-3.3
[
https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345955#comment-17345955
]
Hadoop QA commented on YARN-10725:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
2s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color}
| {color:green}test4tests{color} | {color:green} The patch appears to include 1
new or modified test files. {color} |
|| || || || {color:brown} branch-3.3 Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
51s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
48s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
3s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
29s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 29s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 22m
9s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
54s{color} | {color:green}{color} | {color:green} branch-3.3 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for
patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
46s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
46s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 52s{color} |
{color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/988/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt{color}
| {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch
generated 2 new + 3 unchanged - 1 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green}{color} | {color:green} The patch has no ill-formed
XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 24s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
25s{color} |
{color:red}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/988/artifact/out/diff-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt{color}
| {color:red}
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common
generated 2 new + 45 unchanged - 0 fixed = 47 total (was 45) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} |
[jira] [Created] (YARN-10770) container-executor permission is wrong in SecureContainer.md
Akira Ajisaka created YARN-10770:
Summary: container-executor permission is wrong in
SecureContainer.md
Key: YARN-10770
URL: https://issues.apache.org/jira/browse/YARN-10770
Project: Hadoop YARN
Issue Type: Bug
Components: documentation
Reporter: Akira Ajisaka
{noformat}
The `container-executor` program must be owned by `root` and have the
permission set `---sr-s---`.
{noformat}
It should be 6050 {noformat}---Sr-s---{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10770) container-executor permission is wrong in SecureContainer.md
[
https://issues.apache.org/jira/browse/YARN-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-10770:
-
Labels: newbie (was: )
> container-executor permission is wrong in SecureContainer.md
>
>
> Key: YARN-10770
> URL: https://issues.apache.org/jira/browse/YARN-10770
> Project: Hadoop YARN
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> The `container-executor` program must be owned by `root` and have the
> permission set `---sr-s---`.
> {noformat}
> It should be 6050 {noformat}---Sr-s---{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10725) Backport YARN-10120 to branch-3.3
[ https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T updated YARN-10725: - Attachment: YARN-10725-branch-3.3.v4.patch > Backport YARN-10120 to branch-3.3 > - > > Key: YARN-10725 > URL: https://issues.apache.org/jira/browse/YARN-10725 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10120-branch-3.3.patch, > YARN-10725-branch-3.3.patch, YARN-10725-branch-3.3.v2.patch, > YARN-10725-branch-3.3.v3.patch, YARN-10725-branch-3.3.v4.patch, > image-2021-04-05-16-48-57-034.png, image-2021-04-05-16-50-55-238.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10769) Reconnect transition in RMNodeImpl all containers are considered as G
[ https://issues.apache.org/jira/browse/YARN-10769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345897#comment-17345897 ] Cyrus Jackson commented on YARN-10769: -- I would like to work on it. Assigning it to me. > Reconnect transition in RMNodeImpl all containers are considered as G > -- > > Key: YARN-10769 > URL: https://issues.apache.org/jira/browse/YARN-10769 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin Chundatt >Priority: Minor > > In RMNodeImpl#handleNMContainerStatus *createContainerStatus* is not > considering the container execution type for creating ContainerStatus -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10769) Reconnect transition in RMNodeImpl all containers are considered as G
[ https://issues.apache.org/jira/browse/YARN-10769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson reassigned YARN-10769: Assignee: Cyrus Jackson > Reconnect transition in RMNodeImpl all containers are considered as G > -- > > Key: YARN-10769 > URL: https://issues.apache.org/jira/browse/YARN-10769 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin Chundatt >Assignee: Cyrus Jackson >Priority: Minor > > In RMNodeImpl#handleNMContainerStatus *createContainerStatus* is not > considering the container execution type for creating ContainerStatus -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10769) Reconnect transition in RMNodeImpl all containers are considered as G
Bibin Chundatt created YARN-10769: - Summary: Reconnect transition in RMNodeImpl all containers are considered as G Key: YARN-10769 URL: https://issues.apache.org/jira/browse/YARN-10769 Project: Hadoop YARN Issue Type: Bug Reporter: Bibin Chundatt In RMNodeImpl#handleNMContainerStatus *createContainerStatus* is not considering the container execution type for creating ContainerStatus -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
