[jira] [Updated] (YARN-6977) Node information is not provided for non am containers in RM logs
[
https://issues.apache.org/jira/browse/YARN-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-6977:
-
Fix Version/s: 3.0.0-beta1
> Node information is not provided for non am containers in RM logs
> -
>
> Key: YARN-6977
> URL: https://issues.apache.org/jira/browse/YARN-6977
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sumana Sathish
>Assignee: Suma Shivaprasad
> Labels: newbie
> Fix For: 3.0.0-beta1
>
> Attachments: YARN-6977.002.patch, YARN-6977.patch
>
>
> There is no information on which node non am container is being assigned in
> the trunk for hadoop 3.0
> Earlier we used to have logs for non am container in the similar way
> {code}
> Assigned container container_ of capacity on host
> , which has 1 containers, used and
> available after allocation
> {code}
> 3.0 has information for am container alone in the following way
> {code}
> Done launching container Container: [ContainerId: container_,
> AllocationRequestId: 0, Version: 0, NodeId:nodeID, NodeHttpAddress:
> nodeAddress, Resource: , Priority: 0, Token: Token {
> kind: ContainerToken, service: service}, ExecutionType: GUARANTEED, ] for AM
> appattempt_
> {code}
> Can we please have similar message for Non am container too ??
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[
https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168793#comment-16168793
]
Hadoop QA commented on YARN-7192:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
52s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 40s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7192 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887474/YARN-7192.006.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux fdc4118168ab 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ef8cd5d |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
[jira] [Commented] (YARN-7193) Implement REST API for register Yarnfile in application catalog
[ https://issues.apache.org/jira/browse/YARN-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168788#comment-16168788 ] Hadoop QA commented on YARN-7193: - (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-YARN-Build/17493/console in case of problems. > Implement REST API for register Yarnfile in application catalog > --- > > Key: YARN-7193 > URL: https://issues.apache.org/jira/browse/YARN-7193 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications >Affects Versions: 3.1.0 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: YARN-7193.yarn-native-services.001.patch, > YARN-7193.yarn-native-services.002.patch, > YARN-7193.yarn-native-services.003.patch, > YARN-7193.yarn-native-services.004.patch > > > For support ability to register and index Yarnfile, we need a set of REST API > to register, search, and recommend applications from application catalog. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7193) Implement REST API for register Yarnfile in application catalog
[ https://issues.apache.org/jira/browse/YARN-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7193: Attachment: YARN-7193.yarn-native-services.004.patch Fixed checkstyle, white space and firebugs issues. The unit test failures are test case race conditions with network timeout. The test failure exists independent of this patch. > Implement REST API for register Yarnfile in application catalog > --- > > Key: YARN-7193 > URL: https://issues.apache.org/jira/browse/YARN-7193 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications >Affects Versions: 3.1.0 >Reporter: Eric Yang >Assignee: Eric Yang > Attachments: YARN-7193.yarn-native-services.001.patch, > YARN-7193.yarn-native-services.002.patch, > YARN-7193.yarn-native-services.003.patch, > YARN-7193.yarn-native-services.004.patch > > > For support ability to register and index Yarnfile, we need a set of REST API > to register, search, and recommend applications from application catalog. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168773#comment-16168773
]
Hadoop QA commented on YARN-7203:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
23s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 55s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 56 unchanged - 0 fixed = 57 total (was 56) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 49s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7203 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887461/YARN-7203.v3.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 4cdda3b4a25e 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git
[jira] [Updated] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7192: -- Attachment: YARN-7192.006.patch fixing checkstyles > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch, YARN-7192.005.patch, > YARN-7192.006.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[
https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168767#comment-16168767
]
Hadoop QA commented on YARN-7192:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
47s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 3s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 470 unchanged - 0 fixed = 472 total (was 470) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 25s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7192 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887464/YARN-7192.005.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux ae3f80c0d6c8 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ef8cd5d |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
|
[jira] [Commented] (YARN-2664) Improve RM webapp to expose info about reservations.
[
https://issues.apache.org/jira/browse/YARN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168739#comment-16168739
]
Hadoop QA commented on YARN-2664:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m
17s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
6s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 56s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
35s{color} | {color:red} The patch generated 5 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-2664 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887437/YARN-2664.21.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 3036f7a16e42 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 90894c7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17485/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17485/testReport/ |
| asflicense |
[jira] [Updated] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[
https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-7192:
--
Attachment: YARN-7192.005.patch
Thanks [~jlowe] for the rev.
Updating the patch (005) with a {{MultiStateTranstitionListener}} as well :)
I also updated the NodeManager config to take a list of listeners, rather than
just one.
Also updated the testcases to verify the wiring works.
Do take a look.
Caveat - This is similar to the {{AsyncDispatcher::MultListenerHandler}} and
calls the listeners in order. And like you noted, an earlier one can screw up a
later one etc. I have put in a javadoc comment stating the same.
> Add a pluggable StateMachine Listener that is notified of NM Container State
> changes
>
>
> Key: YARN-7192
> URL: https://issues.apache.org/jira/browse/YARN-7192
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>Assignee: Arun Suresh
> Attachments: YARN-7192.001.patch, YARN-7192.002.patch,
> YARN-7192.003.patch, YARN-7192.004.patch, YARN-7192.005.patch
>
>
> This JIRA is to add support for a plugggable class in the NodeManager that is
> notified of changes to the Container StateMachine state and the events that
> caused the change.
> The proposal is to modify the basic StateMachine class add support for a hook
> that is called before and after a transition.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[
https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168732#comment-16168732
]
Miklos Szegedi commented on YARN-7207:
--
Thank you [~yufeigu] for reporting this. Please also run a pre-check host name
resolution when resource manager starts up and if it is very slow, it would be
nice to place a warning message into the logs.
> Cache the local host name when getting application list in RM
> -
>
> Key: YARN-7207
> URL: https://issues.apache.org/jira/browse/YARN-7207
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: RM
>Affects Versions: 3.1.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>
> {{getLocalHostName()}} is invoked for generating the report for each
> application, which means it is called 1000 times for each
> {{getApplications()}} if there are 1000 apps in RM. Some user got a
> performance issue when {{getLocalHostName()}} is slow under some network envs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168728#comment-16168728 ] Miklos Szegedi commented on YARN-5534: -- [~eyang] I would approach this from the user point of new. container-executor and container-executor.cfg should govern the rules, how the yarn user can get root access or access it does not have otherwise. If the yarn user cannot access a directory, then mounting it should be whitelisted in container-executor.cfg. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7203: --- Attachment: YARN-7203.v3.patch > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch, > YARN-7203.v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168719#comment-16168719
]
Hadoop QA commented on YARN-7203:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 56s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 56 unchanged - 0 fixed = 58 total (was 56) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
47s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 3s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}110m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7203 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887438/YARN-7203.v2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux 25145bb0eabb 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 90894c7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1
[jira] [Commented] (YARN-7208) CMAKE_C_STANDARD take effect in NodeManager package.
[
https://issues.apache.org/jira/browse/YARN-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168720#comment-16168720
]
Wangda Tan commented on YARN-7208:
--
Resolved as dup of HADOOP-14873
> CMAKE_C_STANDARD take effect in NodeManager package.
>
>
> Key: YARN-7208
> URL: https://issues.apache.org/jira/browse/YARN-7208
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Priority: Blocker
>
> I just checked changes of this JIRA doesn't relate to issues I saw, I tried
> to revert this patch but issue is still the same.
> It seems the set (CMAKE_C_STANDARD) doesn't work for the nodemanager project.
> I hardcoded to change set (CMAKE_C_STANDARD 99) to set (CMAKE_C_STANDARD 90)
> in nodemanager project. (Since we have code uses C99-only syntax, so changing
> to 90 should fail build).
> I tried on two different environment:
> 1) Centos 6, cmake version 3.1.0, gcc 4.4.7
> For both 99/90 standard, all fail.
> 2) OSX v10.12.4, cmake version 3.5.2, cc = "Apple LLVM version 8.1.0
> (clang-802.0.42)".
> For both 99/90 standard, all succeeded.
> At least for the for loop in gpu-module.c is C99 only:
> {code}
> for (int i = 0; i < n_minor_devices_to_block; i++) {
>// ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7208) CMAKE_C_STANDARD take effect in NodeManager package.
[
https://issues.apache.org/jira/browse/YARN-7208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan resolved YARN-7208.
--
Resolution: Duplicate
> CMAKE_C_STANDARD take effect in NodeManager package.
>
>
> Key: YARN-7208
> URL: https://issues.apache.org/jira/browse/YARN-7208
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Wangda Tan
>Priority: Blocker
>
> I just checked changes of this JIRA doesn't relate to issues I saw, I tried
> to revert this patch but issue is still the same.
> It seems the set (CMAKE_C_STANDARD) doesn't work for the nodemanager project.
> I hardcoded to change set (CMAKE_C_STANDARD 99) to set (CMAKE_C_STANDARD 90)
> in nodemanager project. (Since we have code uses C99-only syntax, so changing
> to 90 should fail build).
> I tried on two different environment:
> 1) Centos 6, cmake version 3.1.0, gcc 4.4.7
> For both 99/90 standard, all fail.
> 2) OSX v10.12.4, cmake version 3.5.2, cc = "Apple LLVM version 8.1.0
> (clang-802.0.42)".
> For both 99/90 standard, all succeeded.
> At least for the for loop in gpu-module.c is C99 only:
> {code}
> for (int i = 0; i < n_minor_devices_to_block; i++) {
>// ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7208) CMAKE_C_STANDARD take effect in NodeManager package.
Wangda Tan created YARN-7208:
Summary: CMAKE_C_STANDARD take effect in NodeManager package.
Key: YARN-7208
URL: https://issues.apache.org/jira/browse/YARN-7208
Project: Hadoop YARN
Issue Type: Bug
Reporter: Wangda Tan
Priority: Blocker
I just checked changes of this JIRA doesn't relate to issues I saw, I tried to
revert this patch but issue is still the same.
It seems the set (CMAKE_C_STANDARD) doesn't work for the nodemanager project.
I hardcoded to change set (CMAKE_C_STANDARD 99) to set (CMAKE_C_STANDARD 90) in
nodemanager project. (Since we have code uses C99-only syntax, so changing to
90 should fail build).
I tried on two different environment:
1) Centos 6, cmake version 3.1.0, gcc 4.4.7
For both 99/90 standard, all fail.
2) OSX v10.12.4, cmake version 3.5.2, cc = "Apple LLVM version 8.1.0
(clang-802.0.42)".
For both 99/90 standard, all succeeded.
At least for the for loop in gpu-module.c is C99 only:
{code}
for (int i = 0; i < n_minor_devices_to_block; i++) {
// ...
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6977) Node information is not provided for non am containers in RM logs
[
https://issues.apache.org/jira/browse/YARN-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168702#comment-16168702
]
Hadoop QA commented on YARN-6977:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 48s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands |
| |
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
| | org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886703/YARN-6977.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2af252027d2b 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 1a84c24 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17488/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17488/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
[jira] [Commented] (YARN-6977) Node information is not provided for non am containers in RM logs
[
https://issues.apache.org/jira/browse/YARN-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168690#comment-16168690
]
Hadoop QA commented on YARN-6977:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 52s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12886703/YARN-6977.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux cf4e84e998f3 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 1a84c24 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17489/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17489/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17489/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168649#comment-16168649
]
Hadoop QA commented on YARN-7196:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m
27s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7196 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887442/YARN-7196.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3a87a152fa48 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 90894c7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17487/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17487/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17487/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter:
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168647#comment-16168647 ] Hudson commented on YARN-7174: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12890 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12890/]) YARN-7174. Add retry logic in LogsCLI when fetch running application (junping_du: rev 1a84c24b0cf6674fa755403971fa57d8e412b320) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/LogsCLI.java > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Fix For: 2.9.0, 3.0.0-beta1, 3.1.0 > > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168638#comment-16168638
]
Arun Suresh commented on YARN-7196:
---
Thanks for looking in to it and the patch [~djp],
I think a better fix might be to just move both the
{{testContainerUpdateExecTypeOpportunisticToGuaranteed()}} and
{{testContainerUpdateExecTypeGuaranteedToOpportunistic}} testcases to the
{{TestContainerSchedulerQueuing}} test class (it is very similar to the
{{TestContainerManager}} but it has a setup with the queue length > 0 already).
Then we can just remove the setup from the {{TestContainerManager}}.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
> Attachments: YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168632#comment-16168632
]
Hadoop QA commented on YARN-6570:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
47s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 19s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 177 unchanged - 0 fixed = 179 total (was 177)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 27s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6570 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887434/YARN-6570-v3.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 7c358a45f7cf 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 90894c7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17484/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17484/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17484/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17484/testReport/ |
| modules | C:
[jira] [Commented] (YARN-7198) Add jsvc support for RegistryDNS
[ https://issues.apache.org/jira/browse/YARN-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168631#comment-16168631 ] Billie Rinaldi commented on YARN-7198: -- The behavior of deleting the pid file when the daemon is stopped appears to be due to using a newer version of jsvc (see DAEMON-188). > Add jsvc support for RegistryDNS > > > Key: YARN-7198 > URL: https://issues.apache.org/jira/browse/YARN-7198 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi >Priority: Critical > Attachments: YARN-7198-yarn-native-services.01.patch > > > RegistryDNS should have jsvc support and be managed through the shell > scripts, rather than being started manually. See original comments on > YARN-7191. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168614#comment-16168614
]
Junping Du commented on YARN-6570:
--
BTW, TestContainerManager failure get tracked in YARN-7196
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[
https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168611#comment-16168611
]
Hadoop QA commented on YARN-7199:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
45s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7199 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887430/YARN-7199.v3.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f3da63c4801c 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 90894c7 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17483/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17483/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is
> failing in trunk
> -
>
> Key: YARN-7199
> URL: https://issues.apache.org/jira/browse/YARN-7199
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Botong Huang
>Assignee: Botong Huang
>Priority: Blocker
> Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch,
>
[jira] [Updated] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-7196:
-
Attachment: YARN-7196.patch
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
> Attachments: YARN-7196.patch
>
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168606#comment-16168606 ] Jason Lowe commented on YARN-7192: -- +1 lgtm. I'd rather see the multi-listener support added up front. It's such a standard form of listener APIs that it sticks out like a sore thumb when there's only one listener supported. Adding it later has the potential of churning the public API adding deprecations, new configs, etc. I'd rather just avoid that if it's easy to fix now. However I also understand the desire to start simple with the known use-case and the YAGNI principle. I guess we'll see. ;-) > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-7196:
--
Description:
The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
fail every once in a while. Maybe have to change the way the event is triggered.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>
> The Testcase {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to
> fail every once in a while. Maybe have to change the way the event is
> triggered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-7196:
--
Environment: (was: The Testcase
{{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to fail every
once in a while. Maybe have to change the way the event is triggered.)
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Arun Suresh
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7196) Fix finicky TestContainerManager tests
[
https://issues.apache.org/jira/browse/YARN-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168600#comment-16168600
]
Junping Du commented on YARN-7196:
--
It failed consistently now, from YARN-6570 to YARN-7034. Take a quick look, and
find the failed reason is container never get chance to running due to queue
number is 0:
{noformat}
2017-09-15 15:08:53,159 INFO [NM ContainerManager dispatcher]
scheduler.ContainerScheduler (ContainerScheduler.java:enqueueContainer(392)) -
Opportunistic container [container_0__01_00] will not be queued at the
NMsince max queue length [0] has been reached
2017-09-15 15:08:53,162 INFO [NM ContainerManager dispatcher]
container.ContainerImpl (ContainerImpl.java:handle(1893)) - Container
container_0__01_00 transitioned from SCHEDULED to KILLING
{noformat}
Putting a simple patch to fix the test failure (and the log typo) here.
> Fix finicky TestContainerManager tests
> --
>
> Key: YARN-7196
> URL: https://issues.apache.org/jira/browse/YARN-7196
> Project: Hadoop YARN
> Issue Type: Bug
> Environment: The Testcase
> {{testContainerUpdateExecTypeGuaranteedToOpportunistic}} seem to fail every
> once in a while. Maybe have to change the way the event is triggered.
>Reporter: Arun Suresh
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7203) Add container ExecutionType into ContainerReport
[ https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7203: --- Attachment: YARN-7203.v2.patch > Add container ExecutionType into ContainerReport > > > Key: YARN-7203 > URL: https://issues.apache.org/jira/browse/YARN-7203 > Project: Hadoop YARN > Issue Type: Task >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Minor > Attachments: YARN-7203.v1.patch, YARN-7203.v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168584#comment-16168584 ] Eric Yang edited comment on YARN-5534 at 9/15/17 9:58 PM: -- Yarn-site.xml and core-site.xml are trusted configuration from Hdoop server point of view. Hadoop Kerberos, and proxy users configuration are defined in the *.xml files. WIthout trusting those configurations, Hadoop security fall apart. There is keyword like final to lock configuration in place therefore an overlay of Hadoop configuration can not alter the configuration values. Volume white list in core-site.xml or yarn-site.xml are secured. There should be very little configuration in container-executor.cfg file to govern uid and banned user. The rest of the logic in core-site.xml is preferred to ensure the logic is preprocessed in yarn user before handing off to root for execution. YARN can act as a shielding user in pre-processing to make exploits more difficult. was (Author: eyang): Yarn-site.xml and core-site.xml are trusted configuration from Hdoop server point of view. Hadoop Kerberos, and proxy users configuration are defined in the *.xml files. WIthout trusting those configurations, Hadoop security fall apart. There is keyword like final to lock configuration in place therefore an overlay of Hadoop configuration can not alter the configuration values. Volume white list in core-site.xml or yarn-site.xml are secured. There should be very little configuration in container-executor.cfg file to govern uid and banned user. The rest of the logic in core-site.xml is preferred to ensure the logic is preprocessed in yarn user before handing off to root for execution. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7207) Cache the local host name when getting application list in RM
Yufei Gu created YARN-7207:
--
Summary: Cache the local host name when getting application list
in RM
Key: YARN-7207
URL: https://issues.apache.org/jira/browse/YARN-7207
Project: Hadoop YARN
Issue Type: Improvement
Components: RM
Affects Versions: 3.1.0
Reporter: Yufei Gu
Assignee: Yufei Gu
{{getLocalHostName()}} is invoked for generating the report for each
application, which means it is called 1000 times for each {{getApplications()}}
if there are 1000 apps in RM. Some user got a performance issue when
{{getLocalHostName()}} is slow under some network envs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2664) Improve RM webapp to expose info about reservations.
[
https://issues.apache.org/jira/browse/YARN-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated YARN-2664:
--
Attachment: YARN-2664.21.patch
Fixed {{NavBlock}} and check style.
> Improve RM webapp to expose info about reservations.
>
>
> Key: YARN-2664
> URL: https://issues.apache.org/jira/browse/YARN-2664
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Carlo Curino
>Assignee: Íñigo Goiri
> Labels: BB2015-05-TBR
> Attachments: legal.patch, PlannerPage_screenshot.pdf,
> screenshot_reservation_UI.pdf, YARN-2664.10.patch, YARN-2664.11.patch,
> YARN-2664.12.patch, YARN-2664.13.patch, YARN-2664.14.patch,
> YARN-2664.15.patch, YARN-2664.16.patch, YARN-2664.17.patch,
> YARN-2664.18.patch, YARN-2664.19.patch, YARN-2664.1.patch,
> YARN-2664.20.patch, YARN-2664.21.patch, YARN-2664.2.patch, YARN-2664.3.patch,
> YARN-2664.4.patch, YARN-2664.5.patch, YARN-2664.6.patch, YARN-2664.7.patch,
> YARN-2664.8.patch, YARN-2664.9.patch, YARN-2664.patch
>
>
> YARN-1051 provides a new functionality in the RM to ask for reservation on
> resources. Exposing this through the webapp GUI is important.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168584#comment-16168584 ] Eric Yang commented on YARN-5534: - Yarn-site.xml and core-site.xml are trusted configuration from Hdoop server point of view. Hadoop Kerberos, and proxy users configuration are defined in the *.xml files. WIthout trusting those configurations, Hadoop security fall apart. There is keyword like final to lock configuration in place therefore an overlay of Hadoop configuration can not alter the configuration values. Volume white list in core-site.xml or yarn-site.xml are secured. There should be very little configuration in container-executor.cfg file to govern uid and banned user. The rest of the logic in core-site.xml is preferred to ensure the logic is preprocessed in yarn user before handing off to root for execution. > Allow whitelisted volume mounts > > > Key: YARN-5534 > URL: https://issues.apache.org/jira/browse/YARN-5534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: luhuichun >Assignee: Shane Kumpf > Attachments: YARN-5534.001.patch, YARN-5534.002.patch, > YARN-5534.003.patch > > > Introduction > Mounting files or directories from the host is one way of passing > configuration and other information into a docker container. > We could allow the user to set a list of mounts in the environment of > ContainerLaunchContext (e.g. /dir1:/targetdir1,/dir2:/targetdir2). > These would be mounted read-only to the specified target locations. This has > been resolved in YARN-4595 > 2.Problem Definition > Bug mounting arbitrary volumes into a Docker container can be a security risk. > 3.Possible solutions > one approach to provide safe mounts is to allow the cluster administrator to > configure a set of parent directories as white list mounting directories. > Add a property named yarn.nodemanager.volume-mounts.white-list, when > container executor do mount checking, only the allowed directories or > sub-directories can be mounted. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7206) IllegalArgumentException in AMRMClient
[
https://issues.apache.org/jira/browse/YARN-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168583#comment-16168583
]
Jian He commented on YARN-7206:
---
Yep, thanks for the pointers, close this
> IllegalArgumentException in AMRMClient
>
>
> Key: YARN-7206
> URL: https://issues.apache.org/jira/browse/YARN-7206
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jian He
>
> {code}
> java.lang.IllegalArgumentException: The profile name cannot be null
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7206) IllegalArgumentException in AMRMClient
[
https://issues.apache.org/jira/browse/YARN-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He resolved YARN-7206.
---
Resolution: Duplicate
> IllegalArgumentException in AMRMClient
>
>
> Key: YARN-7206
> URL: https://issues.apache.org/jira/browse/YARN-7206
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jian He
>
> {code}
> java.lang.IllegalArgumentException: The profile name cannot be null
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168567#comment-16168567
]
Wangda Tan commented on YARN-7149:
--
Thanks [~eepayne]! I will commit the patch shortly.
> Cross-queue preemption sometimes starves an underserved queue
> -
>
> Key: YARN-7149
> URL: https://issues.apache.org/jira/browse/YARN-7149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: YARN-7149.001.patch, YARN-7149.002.patch,
> YARN-7149.demo.unit-test.patch
>
>
> In branch 2 and trunk, I am consistently seeing some use cases where
> cross-queue preemption does not happen when it should. I do not see this in
> branch-2.8.
> Use Case:
> | | *Size* | *Minimum Container Size* |
> |MyCluster | 20 GB | 0.5 GB |
> | *Queue Name* | *Capacity* | *Absolute Capacity* | *Minimum User Limit
> Percent (MULP)* | *User Limit Factor (ULF)* |
> |Q1 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> |Q2 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> - {{User1}} launches {{App1}} in {{Q1}} and consumes all resources (20 GB)
> - {{User2}} launches {{App2}} in {{Q2}} and requests 10 GB
> - _Note: containers are 0.5 GB._
> - Preemption monitor kills 2 containers (equals 1 GB) from {{App1}} in {{Q1}}.
> - Capacity Scheduler assigns 2 containers (equals 1 GB) to {{App2}} in {{Q2}}.
> - _No more containers are ever preempted, even though {{Q2}} is far
> underserved_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6570) No logs were found for running application, running container
[
https://issues.apache.org/jira/browse/YARN-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-6570:
-
Attachment: YARN-6570-v3.patch
Update the patch to fix the test failures in TestEventFlow.
The failure of TestContainerManager is not related to the patch. From my
verification, it get failed on trunk without patch here.
TestNodeManagerResync doesn't get failed in my local machine. It may just get
intermittent failure and shouldn't related to patch here.
> No logs were found for running application, running container
> -
>
> Key: YARN-6570
> URL: https://issues.apache.org/jira/browse/YARN-6570
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Sumana Sathish
>Assignee: Junping Du
>Priority: Critical
> Attachments: YARN-6570.poc.patch, YARN-6570-v2.patch,
> YARN-6570-v3.patch
>
>
> 1.Obtain running containers from the following CLI for running application:
> yarn container -list appattempt
> 2. Couldnot fetch logs
> {code}
> Can not find any log file matching the pattern: ALL for the container
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[
https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168550#comment-16168550
]
Hadoop QA commented on YARN-6840:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} YARN-5734 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
28s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
46s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
2s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} YARN-5734 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
54s{color} | {color:green} YARN-5734 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
36s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 26s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 5 new + 364 unchanged - 3 fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
56s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 48m 57s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}122m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6840 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887408/YARN-6840-YARN-5734.007.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux ddbd7d16e119 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5734 / 98c4a52 |
| Default Java | 1.8.0_144 |
|
[jira] [Updated] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7199: --- Attachment: YARN-7199.v3.patch Thanks [~asuresh] for the comment. v3 uploaded. > TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is > failing in trunk > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch, > YARN-7199.v3.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168542#comment-16168542 ] Junping Du commented on YARN-7174: -- got it. v2 patch LGTM. +1. Will commit it shortly. > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168499#comment-16168499
]
Eric Payne commented on YARN-7149:
--
The following unit tests are succeeding for me in my environment:
{code}
TestOpportunisticContainerAllocatorAMService
TestZKRMStateStore
TestSubmitApplicationWithRMHA
{code}
{{TestContainerAllocation}} was modified by this patch, and the new test is
succeeding. The failure in
{{TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable}} is a
pre-existing issue: YARN-7044.
> Cross-queue preemption sometimes starves an underserved queue
> -
>
> Key: YARN-7149
> URL: https://issues.apache.org/jira/browse/YARN-7149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: YARN-7149.001.patch, YARN-7149.002.patch,
> YARN-7149.demo.unit-test.patch
>
>
> In branch 2 and trunk, I am consistently seeing some use cases where
> cross-queue preemption does not happen when it should. I do not see this in
> branch-2.8.
> Use Case:
> | | *Size* | *Minimum Container Size* |
> |MyCluster | 20 GB | 0.5 GB |
> | *Queue Name* | *Capacity* | *Absolute Capacity* | *Minimum User Limit
> Percent (MULP)* | *User Limit Factor (ULF)* |
> |Q1 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> |Q2 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> - {{User1}} launches {{App1}} in {{Q1}} and consumes all resources (20 GB)
> - {{User2}} launches {{App2}} in {{Q2}} and requests 10 GB
> - _Note: containers are 0.5 GB._
> - Preemption monitor kills 2 containers (equals 1 GB) from {{App1}} in {{Q1}}.
> - Capacity Scheduler assigns 2 containers (equals 1 GB) to {{App2}} in {{Q2}}.
> - _No more containers are ever preempted, even though {{Q2}} is far
> underserved_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6953) Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources()
[
https://issues.apache.org/jira/browse/YARN-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168480#comment-16168480
]
Daniel Templeton commented on YARN-6953:
LGTM +1
> Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and
> setMaximumAllocationForMandatoryResources()
> --
>
> Key: YARN-6953
> URL: https://issues.apache.org/jira/browse/YARN-6953
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
>Priority: Minor
> Labels: newbie
> Attachments: YARN-6953-YARN-3926.001.patch,
> YARN-6953-YARN-3926.002.patch, YARN-6953-YARN-3926.003.patch,
> YARN-6953-YARN-3926.004.patch, YARN-6953-YARN-3926.005.patch,
> YARN-6953-YARN-3926.006.patch, YARN-6953-YARN-3926-WIP.patch
>
>
> The {{setMinimumAllocationForMandatoryResources()}} and
> {{setMaximumAllocationForMandatoryResources()}} methods are quite convoluted.
> They'd be much simpler if they just handled CPU and memory manually instead
> of trying to be clever about doing it in a loop. There are also issues, such
> as the log warning always talking about memory or the last element of the
> inner array being a copy of the first element.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7174) Add retry logic in LogsCLI when fetch running application logs
[ https://issues.apache.org/jira/browse/YARN-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-7174: - Target Version/s: 2.9.0, 3.1.0 > Add retry logic in LogsCLI when fetch running application logs > -- > > Key: YARN-7174 > URL: https://issues.apache.org/jira/browse/YARN-7174 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Xuan Gong >Assignee: Xuan Gong > Attachments: YARN-7174.1.patch, YARN-7174.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168473#comment-16168473 ] Arun Suresh commented on YARN-7192: --- Thanks for the review [~miklos.szeg...@cloudera.com].. [~jlowe], are you fine with the latest patch ? > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[ https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7199: - Priority: Blocker (was: Major) > TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is > failing in trunk > - > > Key: YARN-7199 > URL: https://issues.apache.org/jira/browse/YARN-7199 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Botong Huang >Assignee: Botong Huang >Priority: Blocker > Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch > > > java.lang.IllegalArgumentException: The profile name cannot be null > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) > at > org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512) > at > org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7206) IllegalArgumentException in AMRMClient
[
https://issues.apache.org/jira/browse/YARN-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168461#comment-16168461
]
Arun Suresh commented on YARN-7206:
---
This is already being tracked here : YARN-7199
> IllegalArgumentException in AMRMClient
>
>
> Key: YARN-7206
> URL: https://issues.apache.org/jira/browse/YARN-7206
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jian He
>
> {code}
> java.lang.IllegalArgumentException: The profile name cannot be null
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7206) NPE in AMRMClient
[
https://issues.apache.org/jira/browse/YARN-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-7206:
--
Description:
{code}
java.lang.IllegalArgumentException: The profile name cannot be null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
at
org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
at
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
{code}
> NPE in AMRMClient
>
>
> Key: YARN-7206
> URL: https://issues.apache.org/jira/browse/YARN-7206
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jian He
>
> {code}
> java.lang.IllegalArgumentException: The profile name cannot be null
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7206) IllegalArgumentException in AMRMClient
[
https://issues.apache.org/jira/browse/YARN-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-7206:
--
Summary: IllegalArgumentException in AMRMClient (was: NPE in
AMRMClient )
> IllegalArgumentException in AMRMClient
>
>
> Key: YARN-7206
> URL: https://issues.apache.org/jira/browse/YARN-7206
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jian He
>
> {code}
> java.lang.IllegalArgumentException: The profile name cannot be null
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.addContainerRequest(AMRMClientAsyncImpl.java:194)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7206) NPE in AMRMClient
Jian He created YARN-7206: - Summary: NPE in AMRMClient Key: YARN-7206 URL: https://issues.apache.org/jira/browse/YARN-7206 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jian He -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[
https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168451#comment-16168451
]
Hadoop QA commented on YARN-7192:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
46s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
29s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 32s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7192 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887406/YARN-7192.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux af508db0418d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3a8d57a |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
[jira] [Commented] (YARN-2162) add ability in Fair Scheduler to optionally configure maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168421#comment-16168421 ] Daniel Templeton commented on YARN-2162: Oh, and the patch needs to be rebased. > add ability in Fair Scheduler to optionally configure maxResources in terms > of percentage > - > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > Attachments: YARN-2162.001.patch, YARN-2162.002.patch, > YARN-2162.003.patch, YARN-2162.004.patch, YARN-2162.005.patch > > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2162) add ability in Fair Scheduler to optionally configure maxResources in terms of percentage
[
https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168420#comment-16168420
]
Daniel Templeton commented on YARN-2162:
My comments:
# In {{AllocationFileLoaderService.loadQueue()}}, instead of removing the max >
min check, can we make it conditional? Use {{fitsIn()}} if they're both
absolute and numerical comparison if they're both percentages, and ignore the
check otherwise, maybe with a warning. Similar in
{{AllocationConfiguration.getMaxResources()}}. Even though
{{FSQueue.getMaxShare()}} does the check, it would be nice to have the warning
earlier while you still have the configuration details to put in the message.
# Looks like {{AllocationConfiguration.DEFAULT_CALCULATOR}} is unused. Not
your fault, so you can file a newbie JIRA if you prefer.
# In {{AllocationConfiguration.initFSQueue()}}, you should probably change all
the metrics inits to use {{queue.get...()}}.
# In the tests, why do we need to set the calculator on the mock?
# It seems like we should use "vcores" instead of "cpu" for consistency.
# In {{FairSchedulerConfiguration.findPercentage()}}, you can probably drop the
trailing while space in the pattern. You also don't need the group 2
parentheses. And the first '*' can probably be a '+', but you might have done
that intentionally.
# In {{TestFairSchedulerConfiguration.testParseResourceConfigValue()}}, you're
missing some negative tests, like one percentage and one absolute.
# Even though you have the tests in {{testParseResourceConfigValue()}}, it
would probably be a good idea to add some tests for {{ResourceConfiguration}}
itself.
# Have you considered performance impact? Every call to {{getMaxShare()}}
causes a read lock to be locked and unlocked and clones a resource. That
sounds kinda heavy to me for how often it gets called. Probably worth testing.
> add ability in Fair Scheduler to optionally configure maxResources in terms
> of percentage
> -
>
> Key: YARN-2162
> URL: https://issues.apache.org/jira/browse/YARN-2162
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler, scheduler
>Reporter: Ashwin Shankar
>Assignee: Yufei Gu
> Labels: scheduler
> Attachments: YARN-2162.001.patch, YARN-2162.002.patch,
> YARN-2162.003.patch, YARN-2162.004.patch, YARN-2162.005.patch
>
>
> minResources and maxResources in fair scheduler configs are expressed in
> terms of absolute numbers X mb, Y vcores.
> As a result, when we expand or shrink our hadoop cluster, we need to
> recalculate and change minResources/maxResources accordingly, which is pretty
> inconvenient.
> We can circumvent this problem if we can optionally configure these
> properties in terms of percentage of cluster capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168417#comment-16168417
]
Hadoop QA commented on YARN-7149:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 33s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 413 unchanged - 0 fixed = 415 total (was 413)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 16s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore |
| |
org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7149 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887401/YARN-7149.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4c211e950d4e 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / de197fc |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17480/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17480/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17480/testReport/ |
| modules | C:
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168403#comment-16168403 ] Miklos Szegedi commented on YARN-7192: -- +1 (non-binding) Thank you [~asuresh], the latest patch looks good to me pending Jenkins results. > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7203) Add container ExecutionType into ContainerReport
[
https://issues.apache.org/jira/browse/YARN-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168397#comment-16168397
]
Arun Suresh commented on YARN-7203:
---
Thanks for the patch [~botong]
Some comments:
* You don't need to create new methods to convert the Exectype in
{{ContainerReportPBImpl}}. You should just directly use ProtoUtils.convert
methods.
* In the following snippet, since you want the default value to be GUARANTEED,
make sure you return GUARANTEED, since out PBImpls are wrappers.
{code}
public ExecutionType getExecutionType() {
ContainerReportProtoOrBuilder p = viaProto ? proto : builder;
if (!p.hasExecutionType()) {
return null;
}
return convertFromProtoFormat(p.getExecutionType());
}
{code}
> Add container ExecutionType into ContainerReport
>
>
> Key: YARN-7203
> URL: https://issues.apache.org/jira/browse/YARN-7203
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Botong Huang
>Assignee: Botong Huang
>Priority: Minor
> Attachments: YARN-7203.v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7162) Remove XML excludes file format
[ https://issues.apache.org/jira/browse/YARN-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168390#comment-16168390 ] Hudson commented on YARN-7162: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12886 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12886/]) YARN-7162. Remove XML excludes file format (rkanter) (rkanter: rev 3a8d57a0a2e047b34be82f602a2b6cf5593d2125) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobResourceUploader.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml > Remove XML excludes file format > --- > > Key: YARN-7162 > URL: https://issues.apache.org/jira/browse/YARN-7162 > Project: Hadoop YARN > Issue Type: Sub-task > Components: graceful >Affects Versions: 2.9.0, 3.0.0-beta1 >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Blocker > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-7162.001.patch, YARN-7162.branch-2.001.patch > > > YARN-5536 aims to replace the XML format for the excludes file with a JSON > format. However, it looks like we won't have time for that for Hadoop 3 Beta > 1. The concern is that if we release it as-is, we'll now have to support the > XML format as-is for all of Hadoop 3.x, which we're either planning on > removing, or rewriting using a pluggable framework. > [This comment in > YARN-5536|https://issues.apache.org/jira/browse/YARN-5536?focusedCommentId=16126194=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16126194] > proposed two quick solutions to prevent this compat issue. In this JIRA, > we're going to remove the XML format. If we later want to add it back in, > YARN-5536 can add it back, rewriting it to be in the pluggable framework. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168380#comment-16168380 ] Jonathan Hung commented on YARN-6840: - Thanks [~leftnoteasy], attached 007 patch to fix javadoc/unit test. > Implement zookeeper based store for scheduler configuration updates > --- > > Key: YARN-6840 > URL: https://issues.apache.org/jira/browse/YARN-6840 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Jonathan Hung > Attachments: YARN-6840-YARN-5734.001.patch, > YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch, > YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch, > YARN-6840-YARN-5734.006.patch, YARN-6840-YARN-5734.007.patch > > > Right now there is only in-memory and leveldb based configuration store > supported. Need one which supports RM HA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-6840: Attachment: YARN-6840-YARN-5734.007.patch > Implement zookeeper based store for scheduler configuration updates > --- > > Key: YARN-6840 > URL: https://issues.apache.org/jira/browse/YARN-6840 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Jonathan Hung > Attachments: YARN-6840-YARN-5734.001.patch, > YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch, > YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch, > YARN-6840-YARN-5734.006.patch, YARN-6840-YARN-5734.007.patch > > > Right now there is only in-memory and leveldb based configuration store > supported. Need one which supports RM HA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[
https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168375#comment-16168375
]
Arun Suresh commented on YARN-7199:
---
Thanks for the patch [~botong]. But I think the actual issue is that the
{{ContainerRequestBuilder#build}} method is not populating the resource profile
field with the default value {{ProfileCapability.DEFAULT_PROFILE}}. It looks
the public {{ContainerRequest}} constructors that sets the value correctly -
unfortunately, the builder uses the private constructor that does not.
The proper fix should be to modify the private constructor to set the value to
default AND add the resourceProfile() method in the builder, which can be used
by a client to set it to something else.
> TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is
> failing in trunk
> -
>
> Key: YARN-7199
> URL: https://issues.apache.org/jira/browse/YARN-7199
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Botong Huang
>Assignee: Botong Huang
> Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch
>
>
> java.lang.IllegalArgumentException: The profile name cannot be null
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7199) TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is failing in trunk
[
https://issues.apache.org/jira/browse/YARN-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168375#comment-16168375
]
Arun Suresh edited comment on YARN-7199 at 9/15/17 7:12 PM:
Thanks for the patch [~botong]. But I think the actual issue is that the
{{ContainerRequestBuilder#build}} method is not populating the resource profile
field with the default value {{ProfileCapability.DEFAULT_PROFILE}}. It looks
the public {{ContainerRequest}} constructors do set the default value correctly
- unfortunately, the builder uses the private constructor that does not.
The proper fix should be to modify the private constructor to set the value to
default AND add the resourceProfile() method in the builder, which can be used
by a client to set it to something else.
was (Author: asuresh):
Thanks for the patch [~botong]. But I think the actual issue is that the
{{ContainerRequestBuilder#build}} method is not populating the resource profile
field with the default value {{ProfileCapability.DEFAULT_PROFILE}}. It looks
the public {{ContainerRequest}} constructors that sets the value correctly -
unfortunately, the builder uses the private constructor that does not.
The proper fix should be to modify the private constructor to set the value to
default AND add the resourceProfile() method in the builder, which can be used
by a client to set it to something else.
> TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests is
> failing in trunk
> -
>
> Key: YARN-7199
> URL: https://issues.apache.org/jira/browse/YARN-7199
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Botong Huang
>Assignee: Botong Huang
> Attachments: YARN-7199.v1.patch, YARN-7199.v2.patch
>
>
> java.lang.IllegalArgumentException: The profile name cannot be null
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> org.apache.hadoop.yarn.api.records.ProfileCapability.newInstance(ProfileCapability.java:68)
> at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.addContainerRequest(AMRMClientImpl.java:512)
> at
> org.apache.hadoop.yarn.client.api.impl.TestAMRMClientContainerRequest.testOpportunisticAndGuaranteedRequests(TestAMRMClientContainerRequest.java:59)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7192: -- Attachment: YARN-7192.004.patch Updating patch bq. A single null check in a function consumes less CPU cycles than a fetch + call and a ret to a virtual function. There are two cases it can cause a problem. Code ran very frequently like this may add up and make the overall product slower. It creates a precedence that others will follow, and it may cause performance issues in other code added later. To allay your concerns, I have made the noop listener a static field and made the 'listener' field final. Both these will ensure that JIT inlines the method call to an actual no-op. This should be even better than the if null check. To be honest, I really dislike null checks, not just becuase the code looks ugly, but all future code that might use the listener will have to re-perform the null check. I would've used the jdk 8 optional, but Id like it to backport to branch-2 without modification. Fixed the checckstyles. > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch, YARN-7192.004.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7205) Log improvements for the ResourceUtils
[
https://issues.apache.org/jira/browse/YARN-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168296#comment-16168296
]
Jian He commented on YARN-7205:
---
another thing notices on the UI is:
{code}
[jira] [Updated] (YARN-7149) Cross-queue preemption sometimes starves an underserved queue
[
https://issues.apache.org/jira/browse/YARN-7149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-7149:
-
Attachment: YARN-7149.002.patch
bq. Do you think does it make sense to merge the
{{YARN-7149.demo.unit-test.patch}} to your patch?
Thanks [~leftnoteasy]. I spent some time looking through the test patch to make
sure I understand its purpose. I think it makes sense to merge it with this
change. Attaching an updated patch.
> Cross-queue preemption sometimes starves an underserved queue
> -
>
> Key: YARN-7149
> URL: https://issues.apache.org/jira/browse/YARN-7149
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Eric Payne
>Assignee: Eric Payne
> Attachments: YARN-7149.001.patch, YARN-7149.002.patch,
> YARN-7149.demo.unit-test.patch
>
>
> In branch 2 and trunk, I am consistently seeing some use cases where
> cross-queue preemption does not happen when it should. I do not see this in
> branch-2.8.
> Use Case:
> | | *Size* | *Minimum Container Size* |
> |MyCluster | 20 GB | 0.5 GB |
> | *Queue Name* | *Capacity* | *Absolute Capacity* | *Minimum User Limit
> Percent (MULP)* | *User Limit Factor (ULF)* |
> |Q1 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> |Q2 | 50% = 10 GB | 100% = 20 GB | 10% = 1 GB | 2.0 |
> - {{User1}} launches {{App1}} in {{Q1}} and consumes all resources (20 GB)
> - {{User2}} launches {{App2}} in {{Q2}} and requests 10 GB
> - _Note: containers are 0.5 GB._
> - Preemption monitor kills 2 containers (equals 1 GB) from {{App1}} in {{Q1}}.
> - Capacity Scheduler assigns 2 containers (equals 1 GB) to {{App2}} in {{Q2}}.
> - _No more containers are ever preempted, even though {{Q2}} is far
> underserved_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3661) Basic Federation UI
[
https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168279#comment-16168279
]
Hadoop QA commented on YARN-3661:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 23s{color}
| {color:red} hadoop-yarn-server-router in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.router.webapp.TestRouterWebServicesREST |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-3661 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887381/YARN-3661-007.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml findbugs checkstyle |
| uname | Linux 4817f51fd65a 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2d2d97f |
| Default Java
[jira] [Commented] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168272#comment-16168272
]
Jason Lowe commented on YARN-7190:
--
I wasn't referring specifically to that JIRA, more to the fact that tools
things were specifically partitioned aside and not put in the classpath by
default at least partially due to the additional dependencies those tools pull
in. HADOOP-12857 or something similar may make it easier to turn these on and
off for starting the nodemanager with the additional classpath settings though.
> Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user
> classpath
>
>
> Key: YARN-7190
> URL: https://issues.apache.org/jira/browse/YARN-7190
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineclient, timelinereader, timelineserver
>Reporter: Vrushali C
>
> [~jlowe] had a good observation about the user classpath getting extra jars
> in hadoop 2.x brought in with TSv2. If users start picking up Hadoop 2,x's
> version of HBase jars instead of the ones they shipped with their job, it
> could be a problem.
> So when TSv2 is to be used in 2,x, the hbase related jars should come into
> only the NM classpath not the user classpath.
> Here is a list of some jars
> {code}
> commons-csv-1.0.jar
> commons-el-1.0.jar
> commons-httpclient-3.1.jar
> disruptor-3.3.0.jar
> findbugs-annotations-1.3.9-1.jar
> hbase-annotations-1.2.6.jar
> hbase-client-1.2.6.jar
> hbase-common-1.2.6.jar
> hbase-hadoop2-compat-1.2.6.jar
> hbase-hadoop-compat-1.2.6.jar
> hbase-prefix-tree-1.2.6.jar
> hbase-procedure-1.2.6.jar
> hbase-protocol-1.2.6.jar
> hbase-server-1.2.6.jar
> htrace-core-3.1.0-incubating.jar
> jamon-runtime-2.4.1.jar
> jasper-compiler-5.5.23.jar
> jasper-runtime-5.5.23.jar
> jcodings-1.0.8.jar
> joni-2.1.2.jar
> jsp-2.1-6.1.14.jar
> jsp-api-2.1-6.1.14.jar
> jsr311-api-1.1.1.jar
> metrics-core-2.2.0.jar
> servlet-api-2.5-6.1.14.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor
[
https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168267#comment-16168267
]
Hadoop QA commented on YARN-7034:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
39s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 39s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7034 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887386/YARN-7034.006.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 08ddb444f587 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2d2d97f |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17479/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17479/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17479/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17479/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was
[jira] [Commented] (YARN-3661) Basic Federation UI
[ https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168261#comment-16168261 ] Giovanni Matteo Fumarola commented on YARN-3661: The code looks good to me. I had an offline meeting with [~elgoiri] in which he showed the new UI from Router. It has 4 views: * The current status of all the SubClusters. If Federation is not enabled, the page is not enabled. * The aggregations of all the Metrics. * All the applications in the entire Federated cluster. * All the nodes in the entire Federated cluster. > Basic Federation UI > > > Key: YARN-3661 > URL: https://issues.apache.org/jira/browse/YARN-3661 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Íñigo Goiri > Attachments: YARN-3661-000.patch, YARN-3661-001.patch, > YARN-3661-002.patch, YARN-3661-003.patch, YARN-3661-004.patch, > YARN-3661-005.patch, YARN-3661-006.patch, YARN-3661-007.patch > > > The UIs provided by each RM, provide a correct "local" view of what is > running in a sub-cluster. In the context of federation we need new > UIs that can track load, jobs, users across sub-clusters. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168238#comment-16168238 ] Miklos Szegedi commented on YARN-7192: -- Thank you [~asuresh] for the updated patch. bq. Hmm.. it is not technically needed, I just prefer it to having if checks. Do you have a strong preference against it ? I do not think it is blocking. A single null check in a function consumes less CPU cycles than a fetch + call and a ret to a virtual function. There are two cases it can cause a problem. Code ran very frequently like this may add up and make the overall product slower. It creates a precedence that others will follow, and it may cause performance issues in other code added later. Similarly, I would fix the checkstyle issues. It is much easier to capture and fix new checkstyle issues later if there are only a few of them in the file at least in some developer environments. Again, I think this is not blocking either. > Add a pluggable StateMachine Listener that is notified of NM Container State > changes > > > Key: YARN-7192 > URL: https://issues.apache.org/jira/browse/YARN-7192 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Arun Suresh > Attachments: YARN-7192.001.patch, YARN-7192.002.patch, > YARN-7192.003.patch > > > This JIRA is to add support for a plugggable class in the NodeManager that is > notified of changes to the Container StateMachine state and the events that > caused the change. > The proposal is to modify the basic StateMachine class add support for a hook > that is called before and after a transition. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7205) Log improvements for the ResourceUtils
Jian He created YARN-7205:
-
Summary: Log improvements for the ResourceUtils
Key: YARN-7205
URL: https://issues.apache.org/jira/browse/YARN-7205
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Jian He
I've seen below logs printed at the service client console after the merge, can
this be moved to debug level log ? cc [~sunilg], [~leftnoteasy]
{code}
17/09/15 10:26:32 INFO conf.Configuration: resource-types.xml not found
17/09/15 10:26:32 INFO resource.ResourceUtils: Unable to find
'resource-types.xml'. Falling back to memory and vcores as resources.
17/09/15 10:26:32 INFO resource.ResourceUtils: Adding resource type - name =
memory-mb, units = Mi, type = COUNTABLE
17/09/15 10:26:32 INFO resource.ResourceUtils: Adding resource type - name =
vcores, units = , type = COUNTABLE
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7198) Add jsvc support for RegistryDNS
[ https://issues.apache.org/jira/browse/YARN-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168231#comment-16168231 ] Billie Rinaldi commented on YARN-7198: -- I tried killing the RegistryDNS daemon manually, and found that when I do this the daemon pid file disappears. The shell scripts are still expecting the pid file to exist after the daemon is killed, which is why an error is printed. But this does not interfere with the killing of the process. I'll fix the javadoc error in my next patch. > Add jsvc support for RegistryDNS > > > Key: YARN-7198 > URL: https://issues.apache.org/jira/browse/YARN-7198 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-native-services >Reporter: Billie Rinaldi >Assignee: Billie Rinaldi >Priority: Critical > Attachments: YARN-7198-yarn-native-services.01.patch > > > RegistryDNS should have jsvc support and be managed through the shell > scripts, rather than being started manually. See original comments on > YARN-7191. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.006.patch YARN-7034.branch-2.006.patch YARN-7034.branch-2.8.006.patch Updating branch-2 patches and increasing the trunk patch index, too. > DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client > environment variables to container-executor > - > > Key: YARN-7034 > URL: https://issues.apache.org/jira/browse/YARN-7034 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Miklos Szegedi >Assignee: Miklos Szegedi >Priority: Critical > Attachments: YARN-7034.000.patch, YARN-7034.001.patch, > YARN-7034.002.patch, YARN-7034.003.patch, YARN-7034.004.patch, > YARN-7034.005.patch, YARN-7034.006.patch, YARN-7034.branch-2.000.patch, > YARN-7034.branch-2.004.patch, YARN-7034.branch-2.005.patch, > YARN-7034.branch-2.006.patch, YARN-7034.branch-2.8.000.patch, > YARN-7034.branch-2.8.004.patch, YARN-7034.branch-2.8.005.patch, > YARN-7034.branch-2.8.006.patch > > > This behavior is unnecessary since there is nothing that is used from the > environment right now. One option is to whitelist these variables before > passing them. Are there any known use cases for this to justify? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3661) Basic Federation UI
[ https://issues.apache.org/jira/browse/YARN-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated YARN-3661: -- Attachment: YARN-3661-007.patch > Basic Federation UI > > > Key: YARN-3661 > URL: https://issues.apache.org/jira/browse/YARN-3661 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Giovanni Matteo Fumarola >Assignee: Íñigo Goiri > Attachments: YARN-3661-000.patch, YARN-3661-001.patch, > YARN-3661-002.patch, YARN-3661-003.patch, YARN-3661-004.patch, > YARN-3661-005.patch, YARN-3661-006.patch, YARN-3661-007.patch > > > The UIs provided by each RM, provide a correct "local" view of what is > running in a sub-cluster. In the context of federation we need new > UIs that can track load, jobs, users across sub-clusters. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168180#comment-16168180 ] Miklos Szegedi commented on YARN-7034: -- Thank you, [~shaneku...@gmail.com] for the review. executePrivilegedOperation environment inheritance change is missing from branch-2 and 2.8, not to affect compatibility. I will update the branch 2 patches. > DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client > environment variables to container-executor > - > > Key: YARN-7034 > URL: https://issues.apache.org/jira/browse/YARN-7034 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Miklos Szegedi >Assignee: Miklos Szegedi >Priority: Critical > Attachments: YARN-7034.000.patch, YARN-7034.001.patch, > YARN-7034.002.patch, YARN-7034.003.patch, YARN-7034.004.patch, > YARN-7034.005.patch, YARN-7034.branch-2.000.patch, > YARN-7034.branch-2.004.patch, YARN-7034.branch-2.005.patch, > YARN-7034.branch-2.8.000.patch, YARN-7034.branch-2.8.004.patch, > YARN-7034.branch-2.8.005.patch > > > This behavior is unnecessary since there is nothing that is used from the > environment right now. One option is to whitelist these variables before > passing them. Are there any known use cases for this to justify? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6799) Remove the duplicated code in CGroupsHandlerImp.java
[
https://issues.apache.org/jira/browse/YARN-6799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168179#comment-16168179
]
Yufei Gu commented on YARN-6799:
Committed to branch-3.0.
> Remove the duplicated code in CGroupsHandlerImp.java
>
>
> Key: YARN-6799
> URL: https://issues.apache.org/jira/browse/YARN-6799
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.0.0-alpha3
>Reporter: Yufei Gu
>Assignee: weiyuan
>Priority: Trivial
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: YARN-6799.001.patch, YARN-6799.002.patch
>
>
> The else clause in initializeCGroupController is not necessary.
> {code}
> public void initializeCGroupController(CGroupController controller) throws
> ResourceHandlerException {
> if (enableCGroupMount) {
> // We have a controller that needs to be mounted
> mountCGroupController(controller);
> } else {
> String controllerPath = getControllerPath(controller);
> if (controllerPath == null) {
> throw new ResourceHandlerException(
> String.format("Controller %s not mounted."
> + " You either need to mount it with %s"
> + " or mount cgroups before launching Yarn",
> controller.getName(), YarnConfiguration.
> NM_LINUX_CONTAINER_CGROUPS_MOUNT));
> }
> }
> // We are working with a pre-mounted contoller
> // Make sure that Yarn cgroup hierarchy path exists
> initializePreMountedCGroupController(controller);
> }
> private void initializePreMountedCGroupController(CGroupController
> controller)
> throws ResourceHandlerException {
> // Check permissions to cgroup hierarchy and
> // create YARN cgroup if it does not exist, yet
> String controllerPath = getControllerPath(controller);
> if (controllerPath == null) {
> throw new ResourceHandlerException(
> String.format("Controller %s not mounted."
> + " You either need to mount it with %s"
> + " or mount cgroups before launching Yarn",
> controller.getName(), YarnConfiguration.
> NM_LINUX_CONTAINER_CGROUPS_MOUNT));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7057) FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption
[ https://issues.apache.org/jira/browse/YARN-7057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168166#comment-16168166 ] Yufei Gu commented on YARN-7057: Committed to branch-3.0. > FSAppAttempt#getResourceUsage doesn't need to consider resources queued for > preemption > -- > > Key: YARN-7057 > URL: https://issues.apache.org/jira/browse/YARN-7057 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.9.0 >Reporter: Karthik Kambatla >Assignee: Karthik Kambatla > Fix For: 2.9.0, 3.0.0-beta1, 3.1.0 > > Attachments: YARN-7057.001.patch, YARN-7057.002.patch > > > FSAppAttempt#getResourceUsage excludes resources that are currently allocated > to the app but are about to be preempted. This inconsistency shows in the UI > and can affect scheduling of containers. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6840) Implement zookeeper based store for scheduler configuration updates
[ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168085#comment-16168085 ] Wangda Tan commented on YARN-6840: -- [~jhung], Thanks for updating the patch, latest patch LGTM, I can commit it once jenkins gets clean report. Could you please check TestRMWebServicesConfigurationMutation failure and javadocs warning? > Implement zookeeper based store for scheduler configuration updates > --- > > Key: YARN-6840 > URL: https://issues.apache.org/jira/browse/YARN-6840 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Jonathan Hung > Attachments: YARN-6840-YARN-5734.001.patch, > YARN-6840-YARN-5734.002.patch, YARN-6840-YARN-5734.003.patch, > YARN-6840-YARN-5734.004.patch, YARN-6840-YARN-5734.005.patch, > YARN-6840-YARN-5734.006.patch > > > Right now there is only in-memory and leveldb based configuration store > supported. Need one which supports RM HA. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7204) Localizer errors on archive without any files
[ https://issues.apache.org/jira/browse/YARN-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-7204: Description: If a user sends an archive without any files in it (only directories), yarn fails to localize it with the error below. I ran into this specifically running spark job but looks generic to localizer. Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_03 exited with exitCode: -1000 Failing this attempt.Diagnostics: No such file or directory ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767) at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218) at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945) at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) For more detailed output, check the application tracking page: https://rm.com:50708/applicationhistory/app/application_1505252418630_25423 Then click on links to logs of each attempt. . Failing the application. was: If a user sends an archive without any files in it (only directories), yarn fails to localize it with the error below. I ran into this specifically running spark job but looks generic to localizer. Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_03 exited with exitCode: -1000 Failing this attempt.Diagnostics: No such file or directory ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767) at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218) at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945) at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[jira] [Updated] (YARN-7204) Localizer errors on archive without any files
[ https://issues.apache.org/jira/browse/YARN-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-7204: Description: If a user sends an archive without any files in it (only directories), yarn fails to localize it with the error below. I ran into this specifically running spark job but looks generic to localizer. Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_03 exited with exitCode: -1000 Failing this attempt.Diagnostics: No such file or directory ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767) at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218) at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945) at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) For more detailed output, check the application tracking page: https://rm.com:50508/applicationhistory/app/application_1505252418630_25423 Then click on links to logs of each attempt. . Failing the application. was: If a user sends an archive without any files in it (only directories), yarn fails to localize it with the error below. I ran into this specifically running spark job but looks generic to localizer. Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_03 exited with exitCode: -1000 Failing this attempt.Diagnostics: No such file or directory ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767) at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218) at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945) at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214) at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[jira] [Created] (YARN-7204) Localizer errors on archive without any files
Thomas Graves created YARN-7204: --- Summary: Localizer errors on archive without any files Key: YARN-7204 URL: https://issues.apache.org/jira/browse/YARN-7204 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.8.1 Reporter: Thomas Graves If a user sends an archive without any files in it (only directories), yarn fails to localize it with the error below. I ran into this specifically running spark job but looks generic to localizer. Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_03 exited with exitCode: -1000 Failing this attempt.Diagnostics: No such file or directory ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767) at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218) at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009) at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005) at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90) at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421) at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945) at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) For more detailed output, check the application tracking page: https://axonitered-jt1.red.ygrid.yahoo.com:50508/applicationhistory/app/application_1505252418630_25423 Then click on links to logs of each attempt. . Failing the application. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7197) Add support for a volume blacklist for docker containers
[
https://issues.apache.org/jira/browse/YARN-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167890#comment-16167890
]
Eric Badger commented on YARN-7197:
---
{quote}
For example, if a system admin would like to allow mounting of /mnt/hdfs/user/$
\{USER\}
but not /mnt/hdfs/user/yarn using nfs gateway.
{quote}
Ok that makes sense from a usability point of view. I think this should be
identified in a comment/in the docs so that it's clear what the intentions of
both the whitelist and blacklist are. Most situations should be handled via the
whitelist and won't have to touch the blacklist at all
> Add support for a volume blacklist for docker containers
>
>
> Key: YARN-7197
> URL: https://issues.apache.org/jira/browse/YARN-7197
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Shane Kumpf
>
> Docker supports bind mounting host directories into containers. Work is
> underway to allow admins to configure a whilelist of volume mounts. While
> this is a much needed and useful feature, it opens the door for
> misconfiguration that may lead to users being able to compromise or crash the
> system.
> One example would be allowing users to mount /run from a host running
> systemd, and then running systemd in that container, rendering the host
> mostly unusable.
> This issue is to add support for a default blacklist. The default blacklist
> would be where we put files and directories that if mounted into a container,
> are likely to have negative consequences. Users are encouraged not to remove
> items from the default blacklist, but may do so if necessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor
[
https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167860#comment-16167860
]
Shane Kumpf commented on YARN-7034:
---
Thanks [~miklos.szeg...@cloudera.com] - The failed test doesn't seem to be
related and is being addressed by YARN-7196. Latest trunk patch looks good to
me. I did notice a couple inconsistencies for the branch-2 and branch-2.8
patches.
branch-2:
1) The executePrivilegedOperation change is missing, below is what is in the
trunk patch, but missing from branch-2:
{code}
public String executePrivilegedOperation(PrivilegedOperation operation,
boolean grabOutput) throws PrivilegedOperationException {
return executePrivilegedOperation(null, operation, null, null, grabOutput,
-true);
+false);
}
{code}
2) testContainerLaunch() has the unnecessary FROM_CLIENT environment variable
set.
{code}
-
+env.put("FROM_CLIENT", "1");
+
{code}
3) Privileged is still misspelled in TestLinuxContainerExecutorWithMocks:
{code}
private PrivilegedOperationExecutor mockPriviligedExec;
{code}
branch-2.8:
1) The executePrivilegedOperation change is missing, below is what is in the
trunk patch, but missing from branch-2.8:
{code}
public String executePrivilegedOperation(PrivilegedOperation operation,
boolean grabOutput) throws PrivilegedOperationException {
return executePrivilegedOperation(null, operation, null, null, grabOutput,
-true);
+false);
}
{code}
2) Privileged is still misspelled in TestLinuxContainerExecutorWithMocks:
{code}
private PrivilegedOperationExecutor mockPriviligedExec;
{code}
> DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client
> environment variables to container-executor
> -
>
> Key: YARN-7034
> URL: https://issues.apache.org/jira/browse/YARN-7034
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Critical
> Attachments: YARN-7034.000.patch, YARN-7034.001.patch,
> YARN-7034.002.patch, YARN-7034.003.patch, YARN-7034.004.patch,
> YARN-7034.005.patch, YARN-7034.branch-2.000.patch,
> YARN-7034.branch-2.004.patch, YARN-7034.branch-2.005.patch,
> YARN-7034.branch-2.8.000.patch, YARN-7034.branch-2.8.004.patch,
> YARN-7034.branch-2.8.005.patch
>
>
> This behavior is unnecessary since there is nothing that is used from the
> environment right now. One option is to whitelist these variables before
> passing them. Are there any known use cases for this to justify?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7197) Add support for a volume blacklist for docker containers
[ https://issues.apache.org/jira/browse/YARN-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167752#comment-16167752 ] Shane Kumpf commented on YARN-7197: --- [~ebadger] - Thanks for the feedback. I had similar concerns, but after having this occur in a real world scenario, I believe there is value. This would help make sure an admin has to think twice before enabling a known dangerous mount. I'm confident a lazy admin will whitelist / and come complain when the system gets broken beyond all repair. :) > Add support for a volume blacklist for docker containers > > > Key: YARN-7197 > URL: https://issues.apache.org/jira/browse/YARN-7197 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Shane Kumpf > > Docker supports bind mounting host directories into containers. Work is > underway to allow admins to configure a whilelist of volume mounts. While > this is a much needed and useful feature, it opens the door for > misconfiguration that may lead to users being able to compromise or crash the > system. > One example would be allowing users to mount /run from a host running > systemd, and then running systemd in that container, rendering the host > mostly unusable. > This issue is to add support for a default blacklist. The default blacklist > would be where we put files and directories that if mounted into a container, > are likely to have negative consequences. Users are encouraged not to remove > items from the default blacklist, but may do so if necessary. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6612) Update fair scheduler policies to be aware of resource types
[ https://issues.apache.org/jira/browse/YARN-6612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167601#comment-16167601 ] Daniel Templeton commented on YARN-6612: Thanks, [~yufeigu]. Let's keep it on 3.1 for now. > Update fair scheduler policies to be aware of resource types > > > Key: YARN-6612 > URL: https://issues.apache.org/jira/browse/YARN-6612 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler >Affects Versions: YARN-3926 >Reporter: Daniel Templeton >Assignee: Daniel Templeton > Fix For: 3.1.0 > > Attachments: YARN-6612.015.patch, YARN-6612.016.patch, > YARN-6612.YARN-3926.001.patch, YARN-6612.YARN-3926.002.patch, > YARN-6612.YARN-3926.004.patch, YARN-6612.YARN-3926.005.patch, > YARN-6612.YARN-3926.006.patch, YARN-6612.YARN-3926.007.patch, > YARN-6612.YARN-3926.008.patch, YARN-6612.YARN-3926.009.patch, > YARN-6612.YARN-3926.010.patch, YARN-6612.YARN-3926.011.patch, > YARN-6612.YARN-3926.012.patch, YARN-6612.YARN-3926.013.patch, > YARN-6612.YARN-3926.014.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167595#comment-16167595
]
Hadoop QA commented on YARN-7157:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
49s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 19s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
40s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
|
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication |
| | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect |
| |
[jira] [Commented] (YARN-7193) Implement REST API for register Yarnfile in application catalog
[
https://issues.apache.org/jira/browse/YARN-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167564#comment-16167564
]
Hadoop QA commented on YARN-7193:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 13 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
46s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
17s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m
4s{color} | {color:green} yarn-native-services passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services
hadoop-yarn-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
9s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 3s{color}
| {color:red} root generated 46 new + 1285 unchanged - 0 fixed = 1331 total
(was 1285) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 32s{color} | {color:orange} root: The patch generated 1043 new + 3 unchanged
- 0 fixed = 1046 total (was 3) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
11s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services
hadoop-yarn-project {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
59s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services-api
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 32s{color}
| {color:red} hadoop-yarn-services in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 25s{color}
| {color:red} hadoop-yarn-services-core in the patch
[jira] [Commented] (YARN-6727) Improve getQueueUserAcls API to query for specific queue and user
[
https://issues.apache.org/jira/browse/YARN-6727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167496#comment-16167496
]
Hadoop QA commented on YARN-6727:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} hadoop-yarn-project/hadoop-yarn: The patch generated
0 new + 75 unchanged - 1 fixed = 75 total (was 76) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common
generated 0 new + 4458 unchanged - 1 fixed = 4458 total (was 4459) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
38s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 51s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6727 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887062/YARN-6727.006.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle cc |
| uname | Linux
[jira] [Commented] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[
https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167494#comment-16167494
]
Hadoop QA commented on YARN-7157:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
46s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
16s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
56s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s{color} | {color:green} branch-3.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} branch-3.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 37s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}108m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7157 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887263/YARN-7157-branch-3.0.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux d13eb28c001d 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-3.0 / a3d6ce2 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
[jira] [Updated] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[ https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7157: -- Attachment: YARN-7157-branch-2.002.patch Attaching branch-2 patch as well. > Add admin configuration to filter per-user's apps in secure cluster > --- > > Key: YARN-7157 > URL: https://issues.apache.org/jira/browse/YARN-7157 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-7157.001.patch, YARN-7157.002.patch, > YARN-7157.003.patch, YARN-7157.004.patch, YARN-7157.005.patch, > YARN-7157-branch-2.001.patch, YARN-7157-branch-2.002.patch, > YARN-7157-branch-3.0.001.patch, YARN-7157-branch-3.0.002.patch > > > A user who is accessing a secure cluster via a secure UI should be able to > see only his/her own apps. > This feature will not break any compatibility as it will turned off by default -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7157) Add admin configuration to filter per-user's apps in secure cluster
[ https://issues.apache.org/jira/browse/YARN-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7157: -- Attachment: YARN-7157-branch-3.0.002.patch Reattaching branch-3 patch > Add admin configuration to filter per-user's apps in secure cluster > --- > > Key: YARN-7157 > URL: https://issues.apache.org/jira/browse/YARN-7157 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-7157.001.patch, YARN-7157.002.patch, > YARN-7157.003.patch, YARN-7157.004.patch, YARN-7157.005.patch, > YARN-7157-branch-2.001.patch, YARN-7157-branch-3.0.001.patch, > YARN-7157-branch-3.0.002.patch > > > A user who is accessing a secure cluster via a secure UI should be able to > see only his/her own apps. > This feature will not break any compatibility as it will turned off by default -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
