[jira] [Created] (YARN-7288) ContaninerLocalizer with multiple JVM Options
Prabhu Joseph created YARN-7288: --- Summary: ContaninerLocalizer with multiple JVM Options Key: YARN-7288 URL: https://issues.apache.org/jira/browse/YARN-7288 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.7.3 Reporter: Prabhu Joseph Assignee: Prabhu Joseph Currently ContaninerLocalizer can be configured with a single JVM option through yarn.nodemanager.container-localizer.java.opts. There are cases where we need more than one like adding -Dlog4j.debug / -verbose to debug issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6943) Update Yarn to YARN in documentation
[ https://issues.apache.org/jira/browse/YARN-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190840#comment-16190840 ] Chetna Chaudhari commented on YARN-6943: Thanks [~sunilg] for reviewing and [~ajisakaa] for committing this patch. > Update Yarn to YARN in documentation > > > Key: YARN-6943 > URL: https://issues.apache.org/jira/browse/YARN-6943 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Reporter: Miklos Szegedi >Assignee: Chetna Chaudhari >Priority: Minor > Labels: newbie > Fix For: 3.0.0, 3.1.0 > > Attachments: YARN-6943-1.patch, YARN-6943-2.patch, YARN-6943-3.patch, > YARN-6943-4.patch > > > Based on the discussion with [~templedf] in YARN-6757 the official case of > YARN is YARN, not Yarn, so we should update all the md files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7284) NodeManager crashes with OOM when Debug log enabled for ContainerLocalizer
[
https://issues.apache.org/jira/browse/YARN-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-7284:
Component/s: nodemanager
> NodeManager crashes with OOM when Debug log enabled for ContainerLocalizer
> ---
>
> Key: YARN-7284
> URL: https://issues.apache.org/jira/browse/YARN-7284
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.7.3
>Reporter: Prabhu Joseph
> Attachments: Screen Shot 2017-10-03 at 1.29.35 PM.png, Screen Shot
> 2017-10-03 at 1.29.48 PM.png
>
>
> NodeManager crashes with OOM when DEBUG log enabled for ContainerLocalizer.
> {code}
> 2017-10-03 07:25:20,066 FATAL yarn.YarnUncaughtExceptionHandler
> (YarnUncaughtExceptionHandler.java:uncaughtException(51)) - Thread
> Thread[Thread-2114,5,main] threw an Error. Shutting down now...
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOf(Arrays.java:3332)
> at
> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
> at
> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
> at
> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:421)
> at java.lang.StringBuffer.append(StringBuffer.java:272)
> at org.apache.hadoop.util.Shell$1.run(Shell.java:900)
> {code}
> errThread part of Hadoop Common Shell reads all the DEBUG log lines and
> appends to StringBuffer errMsg. As per the heap dump, the errMsg stores more
> than 1GB of contents. (attached image)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7198) Add jsvc support for RegistryDNS
[
https://issues.apache.org/jira/browse/YARN-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190753#comment-16190753
]
Allen Wittenauer edited comment on YARN-7198 at 10/4/17 3:41 AM:
-
I tried to run this based upon the documentation here:
https://github.com/apache/hadoop/blob/63d1084e9781e0fee876916190b69f6242dd00e4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/RegistryDNS.md
Playing with this tonight, one thing has become really obvious:
I'm looking in a directory called 'yarn-service' at documentation. If this is
the 'yarn-service', then what is the rest of YARN? service is really not a
good word to use. In fact, service is so over used in the YARN documentation
(here and elsewhere) as a whole that it's lost all meaning. :(
Anyway
I tried to run this with these configs:
{code}
hadoop.registry.dns.enabled
true
hadoop.registry.dns.domain-name
hadoop.example.com
hadoop.registry.dns.bind-port
54
hadoop.registry.dns.zone-subnet
172.16.170.0
hadoop.registry.dns.zone-subnet
255.255.255.0
{code}
It's interesting that it failed to start because it couldn't connect to ZK on
localhost/127.0.0.1:2181. So I'm guessing there is a chunk of documentation
missing?
That said:
bq. hadoop.registry.dns.zone-subnet
How is this specified? CIDR notation?
bq. hadoop.registry.dns.zone-mask
OK, maybe not?
An example really needs to exist in the markdown docs for these two entries
about what is actually wanted here. For example, if I want 192.168.100.0/22
(aka 192.168.100.0 to 192.168.103.255), what does the configuration look like?
If it supports CIDR, then zone-mask is pointless. But I'm guessing it only
supports the classic class A, B, and C?
This is really important because reverse addressing for non-standard network
blocks is a bit wacky to configure even on standard DNS servers.
Thanks.
was (Author: aw):
I tried to run this based upon the documentation here:
https://github.com/apache/hadoop/blob/63d1084e9781e0fee876916190b69f6242dd00e4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/RegistryDNS.md
Playing with this tonight, one thing has become really obvious:
I'm looking in a directory called 'yarn-service' at documentation. If this is
the 'yarn-service', then what is the rest of YARN? service is really not a
good word to use. In fact, service is so over used in the YARN documentation
(here and elsewhere) as a whole that it's lost all meaning. :(
Anyway
I tried to run this with these configs:
{code}
hadoop.registry.dns.enabled
true
hadoop.registry.dns.domain-name
hadoop.example.com
hadoop.registry.dns.bind-port
54
hadoop.registry.dns.zone-subnet
172.16.170.0
hadoop.registry.dns.zone-subnet
255.255.255.0
{code}
It's interesting that it failed to start because it couldn't connect to ZK on
localhost/127.0.0.1:2181. So I'm guessing there is a chunk of documentation
missing?
That said:
bq. hadoop.registry.dns.zone-subnet
How is this specified? CIDR notation?
bq. hadoop.registry.dns.zone-mask
OK, maybe not?
An example really needs to exist in the markdown docs for these two entries
about what is actually wanted here. For example, if I want 192.168.100.0/22
(aka 192.168.100.0 to 192.168.103.255), what does the configuration look like?
If it supports CIDR, then zone-mask is pointless. But I'm guessing it only
supports the classic class A, B, and C?
This is really important because reverse addressing is a bit wacky to configure
even on standard DNS servers.
Thanks.
> Add jsvc support for RegistryDNS
>
>
> Key: YARN-7198
> URL: https://issues.apache.org/jira/browse/YARN-7198
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
>Priority: Critical
> Attachments: YARN-7198-yarn-native-services.01.patch,
> YARN-7198-yarn-native-services.02.patch,
> YARN-7198-yarn-native-services.03.patch,
> YARN-7198-yarn-native-services.04.patch
>
>
> RegistryDNS should have jsvc support and be managed through the shell
> scripts, rather than being started manually. See original comments on
> YARN-7191.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7198) Add jsvc support for RegistryDNS
[
https://issues.apache.org/jira/browse/YARN-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190753#comment-16190753
]
Allen Wittenauer commented on YARN-7198:
I tried to run this based upon the documentation here:
https://github.com/apache/hadoop/blob/63d1084e9781e0fee876916190b69f6242dd00e4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/RegistryDNS.md
Playing with this tonight, one thing has become really obvious:
I'm looking in a directory called 'yarn-service' at documentation. If this is
the 'yarn-service', then what is the rest of YARN? service is really not a
good word to use. In fact, service is so over used in the YARN documentation
(here and elsewhere) as a whole that it's lost all meaning. :(
Anyway
I tried to run this with these configs:
{code}
hadoop.registry.dns.enabled
true
hadoop.registry.dns.domain-name
hadoop.example.com
hadoop.registry.dns.bind-port
54
hadoop.registry.dns.zone-subnet
172.16.170.0
hadoop.registry.dns.zone-subnet
255.255.255.0
{code}
It's interesting that it failed to start because it couldn't connect to ZK on
localhost/127.0.0.1:2181. So I'm guessing there is a chunk of documentation
missing?
That said:
bq. hadoop.registry.dns.zone-subnet
How is this specified? CIDR notation?
bq. hadoop.registry.dns.zone-mask
OK, maybe not?
An example really needs to exist in the markdown docs for these two entries
about what is actually wanted here. For example, if I want 192.168.100.0/22
(aka 192.168.100.0 to 192.168.103.255), what does the configuration look like?
If it supports CIDR, then zone-mask is pointless. But I'm guessing it only
supports the classic class A, B, and C?
This is really important because reverse addressing is a bit wacky to configure
even on standard DNS servers.
Thanks.
> Add jsvc support for RegistryDNS
>
>
> Key: YARN-7198
> URL: https://issues.apache.org/jira/browse/YARN-7198
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Reporter: Billie Rinaldi
>Assignee: Billie Rinaldi
>Priority: Critical
> Attachments: YARN-7198-yarn-native-services.01.patch,
> YARN-7198-yarn-native-services.02.patch,
> YARN-7198-yarn-native-services.03.patch,
> YARN-7198-yarn-native-services.04.patch
>
>
> RegistryDNS should have jsvc support and be managed through the shell
> scripts, rather than being started manually. See original comments on
> YARN-7191.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[
https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190714#comment-16190714
]
Hadoop QA commented on YARN-2037:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 45s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 179 unchanged - 3 fixed = 180 total (was 182) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 51m 6s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 86m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:eaf5c66 |
| JIRA Issue | YARN-2037 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890221/YARN-2037-branch-2.v1.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 71dac4ae477f 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / 175ff05 |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17760/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| unit |
[jira] [Commented] (YARN-2960) Add documentation for the YARN shared cache
[
https://issues.apache.org/jira/browse/YARN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190709#comment-16190709
]
Hadoop QA commented on YARN-2960:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
22m 41s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 38s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-2960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890275/YARN-2960-trunk-002.patch
|
| Optional Tests | asflicense mvnsite xml |
| uname | Linux 39720f30b992 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b34b3ff |
| modules | C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
U: . |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17762/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add documentation for the YARN shared cache
> ---
>
> Key: YARN-2960
> URL: https://issues.apache.org/jira/browse/YARN-2960
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Chris Trezzo
>Assignee: Chris Trezzo
> Attachments: YARN-2960-trunk-001.patch, YARN-2960-trunk-002.patch
>
>
> Add documentation around the architecture, api's and administration of the
> YARN shared cache.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190680#comment-16190680
]
Hadoop QA commented on YARN-7226:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} YARN-7226 does not apply to branch-2.8. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-7226 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890178/YARN-7226-branch-2.8.006.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17761/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch,
> YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190674#comment-16190674
]
Sidharta Seethana commented on YARN-7226:
-
[~jlowe] I committed the branch-2.8 patch. Unfortunately, YARN-6550 was
committed to branch-2 a couple of hours after you uploaded the patch and the
branch-2 patch no longer applies. Apologies for the confusion - do you mind
rebasing the branch-2 patch?
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch,
> YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2960) Add documentation for the YARN shared cache
[ https://issues.apache.org/jira/browse/YARN-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated YARN-2960: --- Attachment: YARN-2960-trunk-002.patch Thanks for the review [~mingma]! Attached is a trunk v2 patch to address your comments. Please let me know if there is anything else. > Add documentation for the YARN shared cache > --- > > Key: YARN-2960 > URL: https://issues.apache.org/jira/browse/YARN-2960 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Chris Trezzo >Assignee: Chris Trezzo > Attachments: YARN-2960-trunk-001.patch, YARN-2960-trunk-002.patch > > > Add documentation around the architecture, api's and administration of the > YARN shared cache. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7283) Nodemanager can't start
[
https://issues.apache.org/jira/browse/YARN-7283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190657#comment-16190657
]
Nguyen Xuan Tinh commented on YARN-7283:
Thanks [~raviprak] . I really resolved it . And sorry about made issue at here.
Can i make questions with jira? If i can, Where ? This is first time for me.
> Nodemanager can't start
> ---
>
> Key: YARN-7283
> URL: https://issues.apache.org/jira/browse/YARN-7283
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.4
>Reporter: Nguyen Xuan Tinh
>
> i installed hadoop with psedou mode follow
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html
> . Then when i start-all.sh i got
> 26177 SecondaryNameNode
> 26355 ResourceManager
> 12211 Jps
> 25814 NameNode
> 25976 DataNode
> so i saw log of nodemanager
> {code:java}
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved
> SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed,
> Message from ResourceManager: NodeManager from ubuntu doesn't satisfy
> minimum allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
> ... 6 more
> 2017-10-03 08:47:49,883 INFO org.apache.hadoop.service.AbstractService:
> Service NodeManager failed in state STARTED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message from
> ResourceManager: NodeManager from ubuntu doesn't satisfy minimum
> allocations, Sending SHUTDOWN signal to the NodeManager.
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message from
> ResourceManager: NodeManager from ubuntu doesn't satisfy minimum
> allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:203)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:272)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:496)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:543)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved
> SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed,
> Message from ResourceManager: NodeManager from ubuntu doesn't satisfy
> minimum allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
> ... 6 more
> {code}
> How i can resolve this ? Please help me, Thanks for reading
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5255) Move ResourceRequest expansion for Delay Scheduling to the Scheduler
[
https://issues.apache.org/jira/browse/YARN-5255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Subru Krishnan updated YARN-5255:
-
Issue Type: Improvement (was: Sub-task)
Parent: (was: YARN-4879)
> Move ResourceRequest expansion for Delay Scheduling to the Scheduler
>
>
> Key: YARN-5255
> URL: https://issues.apache.org/jira/browse/YARN-5255
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Arun Suresh
>Assignee: Arun Suresh
>
> YARN-4879 proposes to enhance the Allocate Request by introducing an id to
> explicitly identify a ResourceRequest.
> Currently, if relaxLocality == true, a Node specific request is expanded to
> rack and *ANY* request by the {{AMRMClient}} before being sent to the
> Scheduler. The requires the 3 copies to perform locality specific delay
> scheduling.
> It would be better to perform the expansion in the Scheduler itself, rather
> than the client, since:
> # The expansion is not really specified in the ApplicationMasterProtocol, a
> non-java client will have to duplicate the expansion logic.
> # Refactor out a lot of unnecessary code in the {{AMRMClientImpl}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-4879) Enhance Allocate Protocol to Identify Requests Explicitly
[ https://issues.apache.org/jira/browse/YARN-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subru Krishnan resolved YARN-4879. -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 2.9.0 > Enhance Allocate Protocol to Identify Requests Explicitly > - > > Key: YARN-4879 > URL: https://issues.apache.org/jira/browse/YARN-4879 > Project: Hadoop YARN > Issue Type: Improvement > Components: applications, resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: SimpleAllocateProtocolProposal-v1.pdf, > SimpleAllocateProtocolProposal-v2.pdf > > > For legacy reasons, the current allocate protocol expects expanded requests > which represent the cumulative request for any change in resource > constraints. This is not only very difficult to comprehend but makes it > impossible for the scheduler to associate container allocations to the > original requests. This problem is amplified by the fact that the expansion > is managed by the AMRMClient which makes it cumbersome for non-Java clients > as they all have to replicate the non-trivial logic. In this JIRA, we are > proposing enhancement to the Allocate Protocol to allow AMs to identify > requests explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5225) Move to a simple clean AMRMClientImpl (v2)
[ https://issues.apache.org/jira/browse/YARN-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Subru Krishnan updated YARN-5225: - Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-4879) > Move to a simple clean AMRMClientImpl (v2) > -- > > Key: YARN-5225 > URL: https://issues.apache.org/jira/browse/YARN-5225 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Subru Krishnan >Assignee: Subru Krishnan > > YARN-4879 puts forward the notion of identifying allocate requests > explicitly. This JIRA proposes to create a first version of a clean > AMRMClient (v2) that replaces the existing convoluted request mapping with > one that directly uses the allocate request id and removing the auto > expansion (rack inference) logic. Please refer to the design doc in the > parent JIRA for details. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6594) [API] Introduce SchedulingRequest object
[
https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190611#comment-16190611
]
Hadoop QA commented on YARN-6594:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-6592 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m
33s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
9s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
53s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
51s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} YARN-6592 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
34s{color} | {color:green} YARN-6592 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 59s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 9 new + 0 unchanged - 0 fixed = 9 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
27s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
24s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
5s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}147m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-6594 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890252/YARN-6594-YARN-6592.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 2cc3fed451cf 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190594#comment-16190594
]
Hadoop QA commented on YARN-7202:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
6s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
50s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 53s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 33s{color} | {color:orange} root: The patch generated 24 new + 10 unchanged
- 1 fixed = 34 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
58s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Updated] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[ https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-2037: --- Attachment: (was: YARN-2037-YARN-branch-2.v1.patch) > Add work preserving restart support for Unmanaged AMs > - > > Key: YARN-2037 > URL: https://issues.apache.org/jira/browse/YARN-2037 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Karthik Kambatla >Assignee: Botong Huang > Attachments: YARN-2037-branch-2.v1.patch, YARN-2037.v1.patch, > YARN-2037.v2.patch, YARN-2037.v3.patch, YARN-2037.v4.patch > > > It would be nice to allow Unmanaged AMs also to restart in a work-preserving > way. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7202) End-to-end UT for api-server
[
https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190567#comment-16190567
]
Hadoop QA commented on YARN-7202:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
3s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} yarn-native-services Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
49s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
58s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} yarn-native-services passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} yarn-native-services passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 7s{color} | {color:orange} root: The patch generated 24 new + 10 unchanged
- 1 fixed = 34 total (was 11) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
26s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 30s{color}
| {color:red} hadoop-yarn-services-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
52s{color} | {color:green} hadoop-yarn-services-api in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}115m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86
[jira] [Commented] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[
https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190554#comment-16190554
]
Hadoop QA commented on YARN-2037:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
29s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 180 unchanged - 3 fixed = 181 total (was 183) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 49s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs |
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| | hadoop.yarn.server.resourcemanager.TestRMHA |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
| |
[jira] [Updated] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7202: Attachment: YARN-7202.yarn-native-services.005.patch - Added ApiServer constructor without parameters. This version of constructor is used by Jersey to initialize REST API. It does not work without the base constructor. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch, > YARN-7202.yarn-native-services.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7219) Make AllocateRequestProto compatible with branch-2/branch-2.8
[
https://issues.apache.org/jira/browse/YARN-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190491#comment-16190491
]
Hudson commented on YARN-7219:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13019 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13019/])
YARN-7219. Make AllocateRequestProto compatible with (rchiang: rev
67e12169b0ae8f3a9edb6511d3ea7ffd57b82e75)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto
> Make AllocateRequestProto compatible with branch-2/branch-2.8
> -
>
> Key: YARN-7219
> URL: https://issues.apache.org/jira/browse/YARN-7219
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0-beta1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: YARN-7219.001.patch
>
>
> For yarn_service_protos.proto, we have the following code in
> (branch-2.8.0, branch-2.8, branch-2)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated ContainerResourceIncreaseRequestProto increase_request = 6;
> repeated UpdateContainerRequestProto update_requests = 7;
> }
> {noformat}
> For yarn_service_protos.proto, we have the following code in
> (trunk)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated UpdateContainerRequestProto update_requests = 6;
> }
> {noformat}
> Notes
> * YARN-3866 was the original JIRA for container resizing.
> * YARN-5221 is what introduced the incompatible change.
> * In branch-2/branch-2.8/branch-2.8.0, this protobuf change was undone by
> "Addendum patch to YARN-3866: fix incompatible API change."
> * There was a similar API fix done in YARN-6071.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190492#comment-16190492
]
Hudson commented on YARN-7211:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13019 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13019/])
YARN-7211. AMSimulator in SLS does't work due to responseId (yufei: rev
79e37dc6dba44e7d1d913593304db7a8516056aa)
* (edit)
hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/appmaster/AMSimulator.java
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6594) [API] Introduce SchedulingRequest object
[
https://issues.apache.org/jira/browse/YARN-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantinos Karanasos updated YARN-6594:
-
Attachment: YARN-6594-YARN-6592.002.patch
Rebasing against latest YARN-6592 branch, taking into account latest version of
placement constraints.
> [API] Introduce SchedulingRequest object
>
>
> Key: YARN-6594
> URL: https://issues.apache.org/jira/browse/YARN-6594
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Konstantinos Karanasos
>Assignee: Konstantinos Karanasos
> Attachments: YARN-6594.001.patch, YARN-6594-YARN-6592.002.patch
>
>
> This JIRA introduces a new SchedulingRequest object.
> It will be part of the {{AllocateRequest}} and will be used to define sizing
> (e.g., number of allocations, size of allocations) and placement constraints
> for allocations.
> Applications can use either this new object (when rich placement constraints
> are required) or the existing {{ResourceRequest}} object.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6943) Update Yarn to YARN in documentation
[ https://issues.apache.org/jira/browse/YARN-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190461#comment-16190461 ] Hudson commented on YARN-6943: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13018 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13018/]) YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna (aajisaka: rev c9b525bd943fe7e098e72fb9be5623862c7ab5ff) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/config/default-config.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/exceptions/YarnRuntimeException.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/java/org/apache/hadoop/registry/server/services/package-info.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/policies/FederationPolicyUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/YarnUncaughtExceptionHandler.java * (edit) hadoop-yarn-project/hadoop-yarn/README * (edit) hadoop-tools/hadoop-sls/src/site/markdown/SchedulerLoadSimulator.md * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/CGroupsHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/YarnVersionInfo.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * (edit) hadoop-tools/hadoop-archive-logs/src/site/markdown/HadoopArchiveLogs.md * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/YarnClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java * (edit) hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/MockResourceManagerFacade.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/NMTokenCache.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/TrackingUriPlugin.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/index.html * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNode.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/test/YarnTestDriver.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/README.md > Update Yarn to YARN in documentation > > > Key: YARN-6943 > URL: https://issues.apache.org/jira/browse/YARN-6943 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Reporter: Miklos
[jira] [Updated] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu updated YARN-7211:
---
Fix Version/s: (was: 3.0.0-beta1)
3.0.0
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190457#comment-16190457
]
Botong Huang commented on YARN-7211:
The unit test failure is irrelevant and I cannot reproduce it locally.
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7219) Make AllocateRequestProto compatible with branch-2/branch-2.8
[
https://issues.apache.org/jira/browse/YARN-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190455#comment-16190455
]
Ray Chiang edited comment on YARN-7219 at 10/3/17 10:16 PM:
Committed to trunk and branch-3.0. Thanks [~asuresh] for reviewing!
was (Author: rchiang):
Committed to trunk. Thanks [~asuresh] for reviewing!
> Make AllocateRequestProto compatible with branch-2/branch-2.8
> -
>
> Key: YARN-7219
> URL: https://issues.apache.org/jira/browse/YARN-7219
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0-beta1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: YARN-7219.001.patch
>
>
> For yarn_service_protos.proto, we have the following code in
> (branch-2.8.0, branch-2.8, branch-2)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated ContainerResourceIncreaseRequestProto increase_request = 6;
> repeated UpdateContainerRequestProto update_requests = 7;
> }
> {noformat}
> For yarn_service_protos.proto, we have the following code in
> (trunk)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated UpdateContainerRequestProto update_requests = 6;
> }
> {noformat}
> Notes
> * YARN-3866 was the original JIRA for container resizing.
> * YARN-5221 is what introduced the incompatible change.
> * In branch-2/branch-2.8/branch-2.8.0, this protobuf change was undone by
> "Addendum patch to YARN-3866: fix incompatible API change."
> * There was a similar API fix done in YARN-6071.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190458#comment-16190458
]
Botong Huang commented on YARN-7211:
Cool, thanks [~yufeigu]!
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Fix For: 3.0.0, 3.1.0
>
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190454#comment-16190454
]
Yufei Gu commented on YARN-7211:
The test failure is unrelated. +1. Committed to trunk and branch-3.0. Thanks
for the patch, [~botong].
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Fix For: 3.0.0-beta1, 3.1.0
>
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7211) AMSimulator in SLS does't work due to refactor of responseId
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu updated YARN-7211:
---
Summary: AMSimulator in SLS does't work due to refactor of responseId
(was: Task in SLS does't work)
> AMSimulator in SLS does't work due to refactor of responseId
>
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang reassigned YARN-7202: --- Assignee: Eric Yang Retrigger precommit test. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He >Assignee: Eric Yang > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7202) End-to-end UT for api-server
[ https://issues.apache.org/jira/browse/YARN-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7202: Attachment: YARN-7202.yarn-native-services.004.patch - Added positive test case for REST API, ServiceClient gets exercised with create and flex. > End-to-end UT for api-server > > > Key: YARN-7202 > URL: https://issues.apache.org/jira/browse/YARN-7202 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jian He > Attachments: YARN-7202.yarn-native-services.001.patch, > YARN-7202.yarn-native-services.002.patch, > YARN-7202.yarn-native-services.003.patch, > YARN-7202.yarn-native-services.004.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6943) Update Yarn to YARN in documentation
[ https://issues.apache.org/jira/browse/YARN-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6943: Component/s: documentation > Update Yarn to YARN in documentation > > > Key: YARN-6943 > URL: https://issues.apache.org/jira/browse/YARN-6943 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Reporter: Miklos Szegedi >Assignee: Chetna Chaudhari >Priority: Minor > Labels: newbie > Attachments: YARN-6943-1.patch, YARN-6943-2.patch, YARN-6943-3.patch, > YARN-6943-4.patch > > > Based on the discussion with [~templedf] in YARN-6757 the official case of > YARN is YARN, not Yarn, so we should update all the md files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6943) Update Yarn to YARN in documentation
[ https://issues.apache.org/jira/browse/YARN-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190425#comment-16190425 ] Akira Ajisaka commented on YARN-6943: - LGTM, +1 TestContainerAllocation: YARN-7044 TestSLSRunner and TestReservationSystemInvariants: YARN-7122 other tests passed on my local environment > Update Yarn to YARN in documentation > > > Key: YARN-6943 > URL: https://issues.apache.org/jira/browse/YARN-6943 > Project: Hadoop YARN > Issue Type: Bug > Components: documentation >Reporter: Miklos Szegedi >Assignee: Chetna Chaudhari >Priority: Minor > Labels: newbie > Attachments: YARN-6943-1.patch, YARN-6943-2.patch, YARN-6943-3.patch, > YARN-6943-4.patch > > > Based on the discussion with [~templedf] in YARN-6757 the official case of > YARN is YARN, not Yarn, so we should update all the md files. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7219) Make AllocateRequestProto compatible with branch-2/branch-2.8
[
https://issues.apache.org/jira/browse/YARN-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ray Chiang updated YARN-7219:
-
Summary: Make AllocateRequestProto compatible with branch-2/branch-2.8
(was: Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk)
> Make AllocateRequestProto compatible with branch-2/branch-2.8
> -
>
> Key: YARN-7219
> URL: https://issues.apache.org/jira/browse/YARN-7219
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0-beta1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Critical
> Attachments: YARN-7219.001.patch
>
>
> For yarn_service_protos.proto, we have the following code in
> (branch-2.8.0, branch-2.8, branch-2)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated ContainerResourceIncreaseRequestProto increase_request = 6;
> repeated UpdateContainerRequestProto update_requests = 7;
> }
> {noformat}
> For yarn_service_protos.proto, we have the following code in
> (trunk)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated UpdateContainerRequestProto update_requests = 6;
> }
> {noformat}
> Notes
> * YARN-3866 was the original JIRA for container resizing.
> * YARN-5221 is what introduced the incompatible change.
> * In branch-2/branch-2.8/branch-2.8.0, this protobuf change was undone by
> "Addendum patch to YARN-3866: fix incompatible API change."
> * There was a similar API fix done in YARN-6071.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7287) Fix typo in ContainerLaunch
[
https://issues.apache.org/jira/browse/YARN-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Kumar updated YARN-7287:
-
Component/s: (was: yarn)
> Fix typo in ContainerLaunch
> ---
>
> Key: YARN-7287
> URL: https://issues.apache.org/jira/browse/YARN-7287
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Ajay Kumar
>Priority: Trivial
>
> Fix typo in ContainerLaunch.
> {code}
> .append(" mapreduce.reduce.e nv\n")
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6916) Moving logging APIs over to slf4j in hadoop-yarn-server-common
[ https://issues.apache.org/jira/browse/YARN-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190381#comment-16190381 ] Hudson commented on YARN-6916: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13016 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13016/]) YARN-6916. Moving logging APIs over to slf4j in (aajisaka: rev 4a877737182808fe3392a116762ba59973a477a6) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/sharedcache/SharedCacheUtil.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppsBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/ErrorsAndWarningsBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/scheduler/OpportunisticContainerAllocator.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppAttemptBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/api/ServerRMProxy.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseContainerTokenSecretManager.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/ContainerBlock.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseNMTokenSecretManager.java > Moving logging APIs over to slf4j in hadoop-yarn-server-common > -- > > Key: YARN-6916 > URL: https://issues.apache.org/jira/browse/YARN-6916 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Fix For: 3.0.0, 3.1.0 > > Attachments: YARN-6712.01.patch, YARN-6916.002.patch, > YARN-6916.003.patch, YARN-6916.004.patch, YARN-6916.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7287) Fix typo in ContainerLaunch
Ajay Kumar created YARN-7287:
Summary: Fix typo in ContainerLaunch
Key: YARN-7287
URL: https://issues.apache.org/jira/browse/YARN-7287
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Reporter: Ajay Kumar
Priority: Trivial
Fix typo in ContainerLaunch.
{code}
.append(" mapreduce.reduce.e nv\n")
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190376#comment-16190376
]
Hadoop QA commented on YARN-7211:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} hadoop-tools/hadoop-sls: The patch generated 0 new +
29 unchanged - 3 fixed = 29 total (was 32) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 36s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.TestSLSRunner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7211 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890233/YARN-7211.v2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0d0d4cfc014a 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 107c177 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17755/artifact/patchprocess/patch-unit-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17755/testReport/ |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17755/console |
| Powered by | Apache Yetus
[jira] [Commented] (YARN-7219) Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk
[
https://issues.apache.org/jira/browse/YARN-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190357#comment-16190357
]
Arun Suresh commented on YARN-7219:
---
+1
> Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk
> -
>
> Key: YARN-7219
> URL: https://issues.apache.org/jira/browse/YARN-7219
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0-beta1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Critical
> Attachments: YARN-7219.001.patch
>
>
> For yarn_service_protos.proto, we have the following code in
> (branch-2.8.0, branch-2.8, branch-2)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated ContainerResourceIncreaseRequestProto increase_request = 6;
> repeated UpdateContainerRequestProto update_requests = 7;
> }
> {noformat}
> For yarn_service_protos.proto, we have the following code in
> (trunk)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated UpdateContainerRequestProto update_requests = 6;
> }
> {noformat}
> Notes
> * YARN-3866 was the original JIRA for container resizing.
> * YARN-5221 is what introduced the incompatible change.
> * In branch-2/branch-2.8/branch-2.8.0, this protobuf change was undone by
> "Addendum patch to YARN-3866: fix incompatible API change."
> * There was a similar API fix done in YARN-6071.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6916) Moving logging APIs over to slf4j in hadoop-yarn-server-common
[ https://issues.apache.org/jira/browse/YARN-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190351#comment-16190351 ] Akira Ajisaka commented on YARN-6916: - Thank you [~bibinchundatt] for reviewing and rebasing the patch. +1, checking this in. > Moving logging APIs over to slf4j in hadoop-yarn-server-common > -- > > Key: YARN-6916 > URL: https://issues.apache.org/jira/browse/YARN-6916 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: YARN-6712.01.patch, YARN-6916.002.patch, > YARN-6916.003.patch, YARN-6916.004.patch, YARN-6916.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6916) Moving logging APIs over to slf4j in hadoop-yarn-server-common
[ https://issues.apache.org/jira/browse/YARN-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-6916: Target Version/s: 3.0.0 (was: 2.9.0, 3.0.0) Removed 2.9.0 from the target versions. > Moving logging APIs over to slf4j in hadoop-yarn-server-common > -- > > Key: YARN-6916 > URL: https://issues.apache.org/jira/browse/YARN-6916 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka > Attachments: YARN-6712.01.patch, YARN-6916.002.patch, > YARN-6916.003.patch, YARN-6916.004.patch, YARN-6916.005.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4163) Audit getQueueInfo and getApplications calls
[
https://issues.apache.org/jira/browse/YARN-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190317#comment-16190317
]
Jason Lowe commented on YARN-4163:
--
Thanks for updating the patch!
It would be good to cleanup the javadoc and checkstyle issues. In addition I
think it would be cleaner and more performant to create an audit log arg
builder class rather than abuse the Map.Entry class. As it is now we're
building a list of entries to pass to the audit logger which then has to loop
over that list passing the entries to a string builder. Instead the caller
could build the args using something like an RMAuditLogger.ArgsBuilder class
that is essentially a wrapper around StringBuilder. The arg builder class
could have an {{add}} method that takes a key, value pair and adds them to the
string with the pair and key/val separator characters just like is done in
RMAuditLogger now. Then when we go to log these extra args we just need to
tack one string builder onto another rather than create a list of objects and
iterate over them.
> Audit getQueueInfo and getApplications calls
>
>
> Key: YARN-4163
> URL: https://issues.apache.org/jira/browse/YARN-4163
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Chang Li
>Assignee: Chang Li
> Attachments: YARN-4163.004.patch, YARN-4163.2.patch,
> YARN-4163.2.patch, YARN-4163.3.patch, YARN-4163.patch
>
>
> getQueueInfo and getApplications seem to sometimes cause spike of load but
> not able to confirm due to they are not audit logged. This patch propose to
> add them to audit log
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang updated YARN-7211:
---
Attachment: YARN-7211.v2.patch
Sure, v2 patch loaded.
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Attachments: YARN-7211.v1.patch, YARN-7211.v2.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7219) Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk
[
https://issues.apache.org/jira/browse/YARN-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190301#comment-16190301
]
Ray Chiang commented on YARN-7219:
--
[~asuresh], let me know if this patch is okay. Thanks.
> Fix AllocateRequestProto difference between branch-2/branch-2.8 and trunk
> -
>
> Key: YARN-7219
> URL: https://issues.apache.org/jira/browse/YARN-7219
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Affects Versions: 3.0.0-beta1
>Reporter: Ray Chiang
>Assignee: Ray Chiang
>Priority: Critical
> Attachments: YARN-7219.001.patch
>
>
> For yarn_service_protos.proto, we have the following code in
> (branch-2.8.0, branch-2.8, branch-2)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated ContainerResourceIncreaseRequestProto increase_request = 6;
> repeated UpdateContainerRequestProto update_requests = 7;
> }
> {noformat}
> For yarn_service_protos.proto, we have the following code in
> (trunk)
> {noformat}
> message AllocateRequestProto {
> repeated ResourceRequestProto ask = 1;
> repeated ContainerIdProto release = 2;
> optional ResourceBlacklistRequestProto blacklist_request = 3;
> optional int32 response_id = 4;
> optional float progress = 5;
> repeated UpdateContainerRequestProto update_requests = 6;
> }
> {noformat}
> Notes
> * YARN-3866 was the original JIRA for container resizing.
> * YARN-5221 is what introduced the incompatible change.
> * In branch-2/branch-2.8/branch-2.8.0, this protobuf change was undone by
> "Addendum patch to YARN-3866: fix incompatible API change."
> * There was a similar API fix done in YARN-6071.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7265) Hadoop Server Log Correlation
[ https://issues.apache.org/jira/browse/YARN-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190285#comment-16190285 ] Eric Yang commented on YARN-7265: - Chukwa has some degree of success in doing this. Chukwa 0.8.0 utilize SOLR to provide indexing feature. This allows user to search for unique text (i.e. Application ID), and all logs contains unique text are categorized and display as results. This includes user's container logs, hdfs (nn/dn), yarn (rm/nm) , and hbase (master/rs) log files. This exists as proof of concept. Chukwa docker image is available on Docker hub for testing. There is a lot work to productize it, but feel free to experiment with it. > Hadoop Server Log Correlation > --- > > Key: YARN-7265 > URL: https://issues.apache.org/jira/browse/YARN-7265 > Project: Hadoop YARN > Issue Type: Wish > Components: log-aggregation >Reporter: Tanping Wang > > Hadoop has many server logs, yarn tasks logs, node manager logs, HDFS logs.. > There are also a lot of different ways can be used to expose the logs, build > relationship horizontally to correlate the logs or search the logs by > keyword. There is a need for a default and yet convenient logging analytics > mechanism in Hadoop itself that at least covers all the server logs of > Hadoop. This log analytics system can correlate the Hadoop server logs by > grouping them by various dimensions including application ID, task ID, job > ID or node ID etc. The raw logs with correlation can be easily accessed by > the application developer or cluster administrator via web page for managing > and debugging. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7286) Add support for docker to have no capabilities
Eric Badger created YARN-7286:
-
Summary: Add support for docker to have no capabilities
Key: YARN-7286
URL: https://issues.apache.org/jira/browse/YARN-7286
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Eric Badger
Assignee: Eric Badger
Support for controlling capabilities was introduced in YARN-4258. However, it
does not allow for the capabilities list to be NULL, since {{getStrings()}}
will treat an empty value the same as it treats an unset property. So, a NULL
list will actually give the default capabilities list.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190265#comment-16190265
]
Jason Lowe commented on YARN-7226:
--
The findbugs warning is unrelated.
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch,
> YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190260#comment-16190260
]
Hadoop QA commented on YARN-7226:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} branch-2.8 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in branch-2.8 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} branch-2.8 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 92 unchanged - 5 fixed = 92 total (was 97) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
0s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c2d96dd |
| JIRA Issue | YARN-7226 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890178/YARN-7226-branch-2.8.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux af4b47797001 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8 / 3629c95 |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17752/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17752/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17752/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Whitelisted variables do not support delayed variable expansion
>
[jira] [Updated] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[ https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-2037: --- Attachment: YARN-2037-branch-2.v1.patch > Add work preserving restart support for Unmanaged AMs > - > > Key: YARN-2037 > URL: https://issues.apache.org/jira/browse/YARN-2037 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Karthik Kambatla >Assignee: Botong Huang > Attachments: YARN-2037-branch-2.v1.patch, YARN-2037.v1.patch, > YARN-2037.v2.patch, YARN-2037.v3.patch, YARN-2037.v4.patch, > YARN-2037-YARN-branch-2.v1.patch > > > It would be nice to allow Unmanaged AMs also to restart in a work-preserving > way. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7285) ContainerExecutor always launches with priorities due to yarn-default property
[ https://issues.apache.org/jira/browse/YARN-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-7285: - Attachment: YARN-7285.001.patch Attaching a patch that removes a specified value for the scheduling priority so the container executor will launch containers using the OS's default priority unless a priority is explicitly specified by the user. > ContainerExecutor always launches with priorities due to yarn-default property > -- > > Key: YARN-7285 > URL: https://issues.apache.org/jira/browse/YARN-7285 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.8.0, 2.9.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Attachments: YARN-7285.001.patch > > > ContainerExecutor will launch containers with a specified priority if a > priority adjustment is specified, otherwise with the OS default priority if > it is unspecified. YARN-3069 added > yarn.nodemanager.container-executor.os.sched.priority.adjustment to > yarn-default.xml, so it is always specified even if the user did not > explicitly set it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7285) ContainerExecutor always launches with priorities due to yarn-default property
[ https://issues.apache.org/jira/browse/YARN-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190238#comment-16190238 ] Jason Lowe commented on YARN-7285: -- YARN-5444 first reported the test failure, but it was misinterpreted as a problem in the unit test rather than a problem introduced in yarn-default.xml. > ContainerExecutor always launches with priorities due to yarn-default property > -- > > Key: YARN-7285 > URL: https://issues.apache.org/jira/browse/YARN-7285 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.8.0, 2.9.0 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > > ContainerExecutor will launch containers with a specified priority if a > priority adjustment is specified, otherwise with the OS default priority if > it is unspecified. YARN-3069 added > yarn.nodemanager.container-executor.os.sched.priority.adjustment to > yarn-default.xml, so it is always specified even if the user did not > explicitly set it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7265) Hadoop Server Log Correlation
[ https://issues.apache.org/jira/browse/YARN-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190236#comment-16190236 ] Tanping Wang edited comment on YARN-7265 at 10/3/17 7:53 PM: - [~jlowe] What the application developers need is a free yet default way of grabbing all the yarn logs, for example. Certainly there could be scripts build to grab the logs, but Hadoop should contain a tool set that provides all the yarn logs related with one job. This does not include any other Hadoop services like Hive, Pig or Oozie, but just Hadoop itself. Thoughts? was (Author: tanping): [~jlowe] What the application developers need is a free yet default way of grabbing all the yarn logs, for example. Certainly there could be scripts build to grab the logs, but Hadoop should contain a tool set that provides all the yarn logs related with one job. This does not include any other Hadoop services like Hive, Pig or Oozie, but just Hadoop itself. > Hadoop Server Log Correlation > --- > > Key: YARN-7265 > URL: https://issues.apache.org/jira/browse/YARN-7265 > Project: Hadoop YARN > Issue Type: Wish > Components: log-aggregation >Reporter: Tanping Wang > > Hadoop has many server logs, yarn tasks logs, node manager logs, HDFS logs.. > There are also a lot of different ways can be used to expose the logs, build > relationship horizontally to correlate the logs or search the logs by > keyword. There is a need for a default and yet convenient logging analytics > mechanism in Hadoop itself that at least covers all the server logs of > Hadoop. This log analytics system can correlate the Hadoop server logs by > grouping them by various dimensions including application ID, task ID, job > ID or node ID etc. The raw logs with correlation can be easily accessed by > the application developer or cluster administrator via web page for managing > and debugging. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7265) Hadoop Server Log Correlation
[ https://issues.apache.org/jira/browse/YARN-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190236#comment-16190236 ] Tanping Wang commented on YARN-7265: [~jlowe] What the application developers need is a free yet default way of grabbing all the yarn logs, for example. Certainly there could be scripts build to grab the logs, but Hadoop should contain a tool set that provides all the yarn logs related with one job. This does not include any other Hadoop services like Hive, Pig or Oozie, but just Hadoop itself. > Hadoop Server Log Correlation > --- > > Key: YARN-7265 > URL: https://issues.apache.org/jira/browse/YARN-7265 > Project: Hadoop YARN > Issue Type: Wish > Components: log-aggregation >Reporter: Tanping Wang > > Hadoop has many server logs, yarn tasks logs, node manager logs, HDFS logs.. > There are also a lot of different ways can be used to expose the logs, build > relationship horizontally to correlate the logs or search the logs by > keyword. There is a need for a default and yet convenient logging analytics > mechanism in Hadoop itself that at least covers all the server logs of > Hadoop. This log analytics system can correlate the Hadoop server logs by > grouping them by various dimensions including application ID, task ID, job > ID or node ID etc. The raw logs with correlation can be easily accessed by > the application developer or cluster administrator via web page for managing > and debugging. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7241) Merge YARN-5734 to trunk/branch-2
[ https://issues.apache.org/jira/browse/YARN-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190229#comment-16190229 ] Yufei Gu commented on YARN-7241: Thanks [~jonathan_huang]. > Merge YARN-5734 to trunk/branch-2 > - > > Key: YARN-7241 > URL: https://issues.apache.org/jira/browse/YARN-7241 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-7241.001.patch, YARN-7241.002.patch, > YARN-7241.003.patch, YARN-7241.004.patch, YARN-7241.005.patch, > YARN-7241.006.patch, YARN-7241-branch-2.001.patch, > YARN-7241-branch-2.002.patch > > > Ticket for jenkins pre-commit for full diff. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190214#comment-16190214
]
Jason Lowe commented on YARN-7226:
--
The unit test failure is interesting. It's a latent bug caused by YARN-3069
that was masked by YARN-5444 in 2.9 and 3.x. I filed YARN-7285 to fix that bug
properly, and I pulled YARN-5444 into branch-2.8 in the interim. I kicked
Jenkins for another run.
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch,
> YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5444) Fix failing unit tests in TestLinuxContainerExecutorWithMocks
[
https://issues.apache.org/jira/browse/YARN-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-5444:
-
Fix Version/s: 2.8.3
Thanks, Yufei! I pulled this into branch-2.8 as well.
> Fix failing unit tests in TestLinuxContainerExecutorWithMocks
> -
>
> Key: YARN-5444
> URL: https://issues.apache.org/jira/browse/YARN-5444
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Fix For: 2.9.0, 3.0.0-alpha1, 2.8.3
>
> Attachments: YARN-5444.001.patch
>
>
> Test case {{testLaunchCommandWithoutPriority}} and {{testStartLocalizer}} are
> based on the assumption that Yarn configuration files won't be loaded, which
> is not true in some situations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7285) ContainerExecutor always launches with priorities due to yarn-default property
Jason Lowe created YARN-7285: Summary: ContainerExecutor always launches with priorities due to yarn-default property Key: YARN-7285 URL: https://issues.apache.org/jira/browse/YARN-7285 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.8.0, 2.9.0 Reporter: Jason Lowe Assignee: Jason Lowe Priority: Minor ContainerExecutor will launch containers with a specified priority if a priority adjustment is specified, otherwise with the OS default priority if it is unspecified. YARN-3069 added yarn.nodemanager.container-executor.os.sched.priority.adjustment to yarn-default.xml, so it is always specified even if the user did not explicitly set it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7265) Hadoop Server Log Correlation
[ https://issues.apache.org/jira/browse/YARN-7265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanping Wang resolved YARN-7265. Resolution: Duplicate > Hadoop Server Log Correlation > --- > > Key: YARN-7265 > URL: https://issues.apache.org/jira/browse/YARN-7265 > Project: Hadoop YARN > Issue Type: Wish > Components: log-aggregation >Reporter: Tanping Wang > > Hadoop has many server logs, yarn tasks logs, node manager logs, HDFS logs.. > There are also a lot of different ways can be used to expose the logs, build > relationship horizontally to correlate the logs or search the logs by > keyword. There is a need for a default and yet convenient logging analytics > mechanism in Hadoop itself that at least covers all the server logs of > Hadoop. This log analytics system can correlate the Hadoop server logs by > grouping them by various dimensions including application ID, task ID, job > ID or node ID etc. The raw logs with correlation can be easily accessed by > the application developer or cluster administrator via web page for managing > and debugging. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7241) Merge YARN-5734 to trunk/branch-2
[ https://issues.apache.org/jira/browse/YARN-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190176#comment-16190176 ] Jonathan Hung edited comment on YARN-7241 at 10/3/17 7:23 PM: -- Hi [~yufeigu], I just compiled and ran SLS with this feature applied - it compiled fine, and started fine, but I ran into YARN-7211 shortly after startup. After applying the patch containers seem to be allocating and running. So it seems everything should be working OK. was (Author: jhung): Hi [~yufeigu], I just compiled and ran SLS with this feature applied - it compiled fine, and started fine, but I ran into YARN-7211 shortly after startup. So it seems everything should be working OK. > Merge YARN-5734 to trunk/branch-2 > - > > Key: YARN-7241 > URL: https://issues.apache.org/jira/browse/YARN-7241 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-7241.001.patch, YARN-7241.002.patch, > YARN-7241.003.patch, YARN-7241.004.patch, YARN-7241.005.patch, > YARN-7241.006.patch, YARN-7241-branch-2.001.patch, > YARN-7241-branch-2.002.patch > > > Ticket for jenkins pre-commit for full diff. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7241) Merge YARN-5734 to trunk/branch-2
[ https://issues.apache.org/jira/browse/YARN-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190176#comment-16190176 ] Jonathan Hung commented on YARN-7241: - Hi [~yufeigu], I just compiled and ran SLS with this feature applied - it compiled fine, and started fine, but I ran into YARN-7211 shortly after startup. So it seems everything should be working OK. > Merge YARN-5734 to trunk/branch-2 > - > > Key: YARN-7241 > URL: https://issues.apache.org/jira/browse/YARN-7241 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-7241.001.patch, YARN-7241.002.patch, > YARN-7241.003.patch, YARN-7241.004.patch, YARN-7241.005.patch, > YARN-7241.006.patch, YARN-7241-branch-2.001.patch, > YARN-7241-branch-2.002.patch > > > Ticket for jenkins pre-commit for full diff. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[ https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6625: - Fix Version/s: 2.9.0 > yarn application -list returns a tracking URL for AM that doesn't work in > secured and HA environment > > > Key: YARN-6625 > URL: https://issues.apache.org/jira/browse/YARN-6625 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > Fix For: 2.9.0, 3.0.0-beta1 > > Attachments: YARN-6625.001.patch, YARN-6625.002.patch, > YARN-6625.003.patch, YARN-6625.004.patch, YARN-6625.branch-2.005.patch > > > The tracking URL given at the command line should work secured or not. The > tracking URLs are like http://node-2.abc.com:47014 and AM web server supposed > to redirect it to a RM address like this > http://node-1.abc.com:8088/proxy/application_1494544954891_0002/, but it > fails to do that because the connection is rejected when AM is talking to RM > admin service to get HA status. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190158#comment-16190158
]
Yufei Gu commented on YARN-7211:
Thanks [~botong] for the patch. Looks good to me. Would you mind fix the style
issue through it is not introduced by your patch? Thanks.
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Attachments: YARN-7211.v1.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[
https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190145#comment-16190145
]
Hadoop QA commented on YARN-6625:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-yarn-server-web-proxy in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:eaf5c66 |
| JIRA Issue | YARN-6625 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890179/YARN-6625.branch-2.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 174db11a6e39 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / 598329e |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17751/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17751/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> yarn application -list returns a tracking URL for AM that doesn't work in
> secured and HA environment
>
>
> Key: YARN-6625
> URL: https://issues.apache.org/jira/browse/YARN-6625
> Project: Hadoop YARN
> Issue Type: Bug
> Components: amrmproxy
>Affects Versions: 3.0.0-alpha2
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Fix For: 3.0.0-beta1
>
[jira] [Commented] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190125#comment-16190125
]
Hadoop QA commented on YARN-7211:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 17s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 11s{color} | {color:orange} hadoop-tools/hadoop-sls: The patch generated 2
new + 30 unchanged - 2 fixed = 32 total (was 32) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
48s{color} | {color:green} hadoop-sls in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | YARN-7211 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890193/YARN-7211.v1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 3b5dd4ee83b9 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 4111e6c |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17750/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17750/testReport/ |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17750/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was
[jira] [Commented] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[ https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190117#comment-16190117 ] Wangda Tan commented on YARN-6625: -- Thanks [~yufeigu] for the review, I will commit the patch once Jenkins give +1. > yarn application -list returns a tracking URL for AM that doesn't work in > secured and HA environment > > > Key: YARN-6625 > URL: https://issues.apache.org/jira/browse/YARN-6625 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > Fix For: 3.0.0-beta1 > > Attachments: YARN-6625.001.patch, YARN-6625.002.patch, > YARN-6625.003.patch, YARN-6625.004.patch, YARN-6625.branch-2.005.patch > > > The tracking URL given at the command line should work secured or not. The > tracking URLs are like http://node-2.abc.com:47014 and AM web server supposed > to redirect it to a RM address like this > http://node-1.abc.com:8088/proxy/application_1494544954891_0002/, but it > fails to do that because the connection is rejected when AM is talking to RM > admin service to get HA status. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[ https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190114#comment-16190114 ] Yufei Gu commented on YARN-6625: Thanks for the patch, [~wangda]. +1 for patch v5. > yarn application -list returns a tracking URL for AM that doesn't work in > secured and HA environment > > > Key: YARN-6625 > URL: https://issues.apache.org/jira/browse/YARN-6625 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > Fix For: 3.0.0-beta1 > > Attachments: YARN-6625.001.patch, YARN-6625.002.patch, > YARN-6625.003.patch, YARN-6625.004.patch, YARN-6625.branch-2.005.patch > > > The tracking URL given at the command line should work secured or not. The > tracking URLs are like http://node-2.abc.com:47014 and AM web server supposed > to redirect it to a RM address like this > http://node-1.abc.com:8088/proxy/application_1494544954891_0002/, but it > fails to do that because the connection is rejected when AM is talking to RM > admin service to get HA status. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-7283) Nodemanager can't start
[
https://issues.apache.org/jira/browse/YARN-7283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved YARN-7283.
Resolution: Invalid
Hi Nguyen! JIRA is for reporting bugs. Please use the user mailing list for
these questions.
Please set {{yarn.nodemanager.resource.memory-mb}},
{{yarn.nodemanager.resource.cpu-vcores}} in your yarn-site.xml
> Nodemanager can't start
> ---
>
> Key: YARN-7283
> URL: https://issues.apache.org/jira/browse/YARN-7283
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.4
>Reporter: Nguyen Xuan Tinh
>
> i installed hadoop with psedou mode follow
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html
> . Then when i start-all.sh i got
> 26177 SecondaryNameNode
> 26355 ResourceManager
> 12211 Jps
> 25814 NameNode
> 25976 DataNode
> so i saw log of nodemanager
> {code:java}
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved
> SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed,
> Message from ResourceManager: NodeManager from ubuntu doesn't satisfy
> minimum allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
> ... 6 more
> 2017-10-03 08:47:49,883 INFO org.apache.hadoop.service.AbstractService:
> Service NodeManager failed in state STARTED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message from
> ResourceManager: NodeManager from ubuntu doesn't satisfy minimum
> allocations, Sending SHUTDOWN signal to the NodeManager.
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN
> signal from Resourcemanager ,Registration of NodeManager failed, Message from
> ResourceManager: NodeManager from ubuntu doesn't satisfy minimum
> allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:203)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:272)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:496)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:543)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved
> SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed,
> Message from ResourceManager: NodeManager from ubuntu doesn't satisfy
> minimum allocations, Sending SHUTDOWN signal to the NodeManager.
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
> ... 6 more
> {code}
> How i can resolve this ? Please help me, Thanks for reading
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-7269:
-
Fix Version/s: 3.1.0
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190111#comment-16190111
]
Wangda Tan commented on YARN-7269:
--
Will backport this to branch-3.0/branch-2 once YARN-6625 gets in.
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190108#comment-16190108
]
Wangda Tan commented on YARN-7269:
--
Thanks [~jianhe]/[~yufeigu] for reviewing the issue, thanks
[~ssath...@hortonworks.com] for reporting and helping on this issue.
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels
[ https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190106#comment-16190106 ] Wangda Tan commented on YARN-2497: -- Thanks [~templedf]. I'm +1 to common scheduler / node label manager code changes, please get someone to help with fair scheduler related logics review. > Changes for fair scheduler to support allocate resource respect labels > -- > > Key: YARN-2497 > URL: https://issues.apache.org/jira/browse/YARN-2497 > Project: Hadoop YARN > Issue Type: Sub-task > Components: fairscheduler >Reporter: Wangda Tan >Assignee: Daniel Templeton > Attachments: YARN-2497.001.patch, YARN-2497.002.patch, > YARN-2497.003.patch, YARN-2497.004.patch, YARN-2497.005.patch, > YARN-2497.006.patch, YARN-2497.007.patch, YARN-2497.008.patch, > YARN-2497.009.patch, YARN-2497.010.patch, YARN-2499.WIP01.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7241) Merge YARN-5734 to trunk/branch-2
[ https://issues.apache.org/jira/browse/YARN-7241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190103#comment-16190103 ] Yufei Gu commented on YARN-7241: Hi [~jhung], have you ever checked SLS? Hadoop QA don't compile and test against SLS. We usually found SLS is broken after a patch is in. Better to do a smoke test for SLS. > Merge YARN-5734 to trunk/branch-2 > - > > Key: YARN-7241 > URL: https://issues.apache.org/jira/browse/YARN-7241 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung > Attachments: YARN-7241.001.patch, YARN-7241.002.patch, > YARN-7241.003.patch, YARN-7241.004.patch, YARN-7241.005.patch, > YARN-7241.006.patch, YARN-7241-branch-2.001.patch, > YARN-7241-branch-2.002.patch > > > Ticket for jenkins pre-commit for full diff. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5762) Summarize ApplicationNotFoundException in the RM log
[
https://issues.apache.org/jira/browse/YARN-5762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated YARN-5762:
---
Target Version/s: 2.10.0 (was: 2.9.0)
> Summarize ApplicationNotFoundException in the RM log
>
>
> Key: YARN-5762
> URL: https://issues.apache.org/jira/browse/YARN-5762
> Project: Hadoop YARN
> Issue Type: Task
>Affects Versions: 2.7.2
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
>Priority: Minor
> Attachments: YARN-5762.01.patch
>
>
> We found a lot of {{ApplicationNotFoundException}} in the RM logs. These were
> most likely caused by the {{AggregatedLogDeletionService}} [which
> checks|https://github.com/apache/hadoop/blob/262827cf75bf9c48cd95335eb04fd8ff1d64c538/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogDeletionService.java#L156]
> that the application is not running anymore. e.g.
> {code}2016-10-17 15:25:26,542 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 20 on 8032, call
> org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport
> from :12205 Call#35401 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application
> with id 'application_1473396553140_1451' doesn't exist in RM.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:327)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:175)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:417)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> 2016-10-17 15:25:26,633 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 47 on 8032, call
> org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplicationReport
> from :12205 Call#35404 Retry#0
> org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application
> with id 'application_1473396553140_1452' doesn't exist in RM.
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationReport(ClientRMService.java:327)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationReport(ApplicationClientProtocolPBServiceImpl.java:175)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:417)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1679)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7254) UI and metrics changes related to absolute resource configuration
[ https://issues.apache.org/jira/browse/YARN-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7254: -- Attachment: YARN-7254.001.patch v1 patch > UI and metrics changes related to absolute resource configuration > - > > Key: YARN-7254 > URL: https://issues.apache.org/jira/browse/YARN-7254 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Reporter: Sunil G >Assignee: Sunil G > Attachments: YARN-7254.001.patch > > > Impact on UI and metrics related to absolute resource configuration on CS -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang reassigned YARN-7211:
--
Assignee: Botong Huang (was: Yufei Gu)
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Botong Huang
>Priority: Blocker
> Attachments: YARN-7211.v1.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7211) Task in SLS does't work
[
https://issues.apache.org/jira/browse/YARN-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang updated YARN-7211:
---
Attachment: YARN-7211.v1.patch
Thanks [~yufeigu] for reporting the issue. I've loaded the simple fix. We are
enforcing a more strict reponseId check in AM to RM heartbeat in YARN-6640. Not
sure why we didn't hit these test failure earlier though.
> Task in SLS does't work
> ---
>
> Key: YARN-7211
> URL: https://issues.apache.org/jira/browse/YARN-7211
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
>Affects Versions: 3.0.0-beta1, 3.1.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
>Priority: Blocker
> Attachments: YARN-7211.v1.patch
>
>
> {code}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1980)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator.sendContainerRequest(MRAMSimulator.java:339)
> at
> org.apache.hadoop.yarn.sls.appmaster.AMSimulator.middleStep(AMSimulator.java:201)
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:94)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.yarn.exceptions.InvalidApplicationMasterRequestException:
> Invalid responseId in AllocateRequest from application attempt:
> appattempt_1505762809623_0001_01, expect responseId to be 0, but get 1
> at
> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService.allocate(ApplicationMasterService.java:377)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:343)
> at
> org.apache.hadoop.yarn.sls.appmaster.MRAMSimulator$1.run(MRAMSimulator.java:340)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> ... 6 more
> Exception in thread "pool-4-thread-8" java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.sls.scheduler.TaskRunner$Task.run(TaskRunner.java:103)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems like it is broken by YARN-6640. SLS works after revert YARN-6640 as I
> tested.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels
[
https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189991#comment-16189991
]
Hadoop QA commented on YARN-2497:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 22 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 18s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 12s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 7 new + 1020 unchanged - 30 fixed = 1027 total (was 1050) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 45s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 48m 10s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit
[jira] [Commented] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189985#comment-16189985
]
Hudson commented on YARN-7269:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13012 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13012/])
YARN-7269. Tracking URL in the app state does not get redirected to (jianhe:
rev 4111e6c7819d66e17e0932302c589777a355dbae)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmFilterInitializer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilterInitializer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/test/java/org/apache/hadoop/yarn/server/webproxy/amfilter/TestAmFilter.java
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6856) Support CLI for Node Attributes Mapping
[
https://issues.apache.org/jira/browse/YARN-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189982#comment-16189982
]
Hadoop QA commented on YARN-6856:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} yarn-3409 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 6m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
13s{color} | {color:green} yarn-3409 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
54s{color} | {color:green} yarn-3409 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
12s{color} | {color:green} yarn-3409 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
50s{color} | {color:green} yarn-3409 passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 10m
42s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
4s{color} | {color:green} yarn-3409 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
1s{color} | {color:green} yarn-3409 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 12s{color} | {color:orange} root: The patch generated 28 new + 26 unchanged
- 0 fixed = 54 total (was 26) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
23s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
10s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 12m
18s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
11s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 23s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
56s{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black}
[jira] [Commented] (YARN-7269) Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications
[
https://issues.apache.org/jira/browse/YARN-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189962#comment-16189962
]
Jian He commented on YARN-7269:
---
committed the patch to trunk, thanks Wangda !
Thanks [~ssath...@hortonworks.com] for reporting the issue !
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications
> ---
>
> Key: YARN-7269
> URL: https://issues.apache.org/jira/browse/YARN-7269
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Sumana Sathish
>Assignee: Tan, Wangda
>Priority: Critical
> Attachments: YARN-7269.001.patch, YARN-7269.002.patch,
> YARN-7269.003.patch
>
>
> Tracking URL in the app state does not get redirected to ApplicationMaster
> for Running applications. It gives following exception
> {code}
> org.mortbay.log: /ws/v1/mapreduce/info
> javax.servlet.ServletException: Could not determine the proxy server for
> redirection
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.findRedirectUrl(AmIpFilter.java:199)
> at
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter.doFilter(AmIpFilter.java:141)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1426)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[
https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189957#comment-16189957
]
Robert Kanter commented on YARN-7207:
-
[~aw], did you have any more comments? Otherwise, I think [~yufeigu] can
commit this.
> Cache the local host name when getting application list in RM
> -
>
> Key: YARN-7207
> URL: https://issues.apache.org/jira/browse/YARN-7207
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: RM
>Affects Versions: 3.1.0
>Reporter: Yufei Gu
>Assignee: Yufei Gu
> Attachments: YARN-7207.001.patch, YARN-7207.002.patch
>
>
> {{getLocalHostName()}} is invoked for generating the report for each
> application, which means it is called 1000 times for each
> {{getApplications()}} if there are 1000 apps in RM. Some user got a
> performance issue when {{getLocalHostName()}} is slow under some network envs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6550) Capture launch_container.sh logs to a separate log file
[
https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-6550:
-
Fix Version/s: 2.9.0
> Capture launch_container.sh logs to a separate log file
> ---
>
> Key: YARN-6550
> URL: https://issues.apache.org/jira/browse/YARN-6550
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0-beta1
>Reporter: Wangda Tan
>Assignee: Suma Shivaprasad
> Fix For: 2.9.0, 3.0.0
>
> Attachments: YARN-6550.002.patch, YARN-6550.003.patch,
> YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch,
> YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch,
> YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.012.patch,
> YARN-6550.branch-2.001.patch, YARN-6550.patch
>
>
> launch_container.sh which generated by NM will do a bunch of things (like
> create link, etc.) while launch a process. No logs captured until {{exec}} is
> called. We need capture all failures of launch_container.sh for easier
> troubleshooting.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189941#comment-16189941
]
Hadoop QA commented on YARN-7226:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} branch-2.8 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
56s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in branch-2.8 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} branch-2.8 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 90 unchanged - 5 fixed = 90 total (was 95) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 49s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.TestLinuxContainerExecutorWithMocks |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c2d96dd |
| JIRA Issue | YARN-7226 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890178/YARN-7226-branch-2.8.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 6e1d1a8668dc 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8 / 4540cbb |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/17748/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17748/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17748/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
[jira] [Updated] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[ https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6625: - Attachment: (was: YARN-6625.branch-2.005.patch) > yarn application -list returns a tracking URL for AM that doesn't work in > secured and HA environment > > > Key: YARN-6625 > URL: https://issues.apache.org/jira/browse/YARN-6625 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > Fix For: 3.0.0-beta1 > > Attachments: YARN-6625.001.patch, YARN-6625.002.patch, > YARN-6625.003.patch, YARN-6625.004.patch, YARN-6625.branch-2.005.patch > > > The tracking URL given at the command line should work secured or not. The > tracking URLs are like http://node-2.abc.com:47014 and AM web server supposed > to redirect it to a RM address like this > http://node-1.abc.com:8088/proxy/application_1494544954891_0002/, but it > fails to do that because the connection is rejected when AM is talking to RM > admin service to get HA status. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6625) yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment
[ https://issues.apache.org/jira/browse/YARN-6625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6625: - Attachment: YARN-6625.branch-2.005.patch [~yufeigu], apologize for uploading a wrong patch, now this is the correct one, please let me know your thoughts. > yarn application -list returns a tracking URL for AM that doesn't work in > secured and HA environment > > > Key: YARN-6625 > URL: https://issues.apache.org/jira/browse/YARN-6625 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy >Affects Versions: 3.0.0-alpha2 >Reporter: Yufei Gu >Assignee: Yufei Gu > Fix For: 3.0.0-beta1 > > Attachments: YARN-6625.001.patch, YARN-6625.002.patch, > YARN-6625.003.patch, YARN-6625.004.patch, YARN-6625.branch-2.005.patch > > > The tracking URL given at the command line should work secured or not. The > tracking URLs are like http://node-2.abc.com:47014 and AM web server supposed > to redirect it to a RM address like this > http://node-1.abc.com:8088/proxy/application_1494544954891_0002/, but it > fails to do that because the connection is rejected when AM is talking to RM > admin service to get HA status. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-7226:
-
Attachment: YARN-7226-branch-2.8.006.patch
The unit test failures are unrelated and are all caused by a bind exception due
to an address being in use. I'm guessing some process leaked from a previous
Jenkins run on the box and is sitting on the desired port. Those unit tests
pass for me locally with the patch applied.
Attaching the branch-2.8 version of the patch.
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch,
> YARN-7226-branch-2.8.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7246) Fix the default docker binary path
[
https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189878#comment-16189878
]
Eric Badger commented on YARN-7246:
---
[~shaneku...@gmail.com], thanks for the updated patch. Instead of removing the
instances of {{const}}, could we instead change the function signature of
{{get_docker_binary_path()}} to return a {{const}}?
> Fix the default docker binary path
> --
>
> Key: YARN-7246
> URL: https://issues.apache.org/jira/browse/YARN-7246
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Blocker
> Attachments: YARN-7246-branch-2.8.2.001.patch,
> YARN-7246-branch-2.8.2.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[
https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189874#comment-16189874
]
Hadoop QA commented on YARN-2037:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} YARN-2037 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-2037 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890173/YARN-2037-YARN-branch-2.v1.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17747/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add work preserving restart support for Unmanaged AMs
> -
>
> Key: YARN-2037
> URL: https://issues.apache.org/jira/browse/YARN-2037
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Karthik Kambatla
>Assignee: Botong Huang
> Attachments: YARN-2037.v1.patch, YARN-2037.v2.patch,
> YARN-2037.v3.patch, YARN-2037.v4.patch, YARN-2037-YARN-branch-2.v1.patch
>
>
> It would be nice to allow Unmanaged AMs also to restart in a work-preserving
> way.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189866#comment-16189866
]
Hadoop QA commented on YARN-7226:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 157 unchanged - 5 fixed = 157 total (was 162)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 30m 43s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.nodemanager.webapp.TestNMWebServer |
| | hadoop.yarn.server.nodemanager.TestNodeStatusUpdater |
| | hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels |
| | hadoop.yarn.server.nodemanager.TestNodeManagerReboot |
| Timed out junit tests |
org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerResync |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:eaf5c66 |
| JIRA Issue | YARN-7226 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890166/YARN-7226-branch-2.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 4fd6582a7b1d 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / 8beae14 |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/17745/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17745/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
[jira] [Updated] (YARN-2037) Add work preserving restart support for Unmanaged AMs
[
https://issues.apache.org/jira/browse/YARN-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Botong Huang updated YARN-2037:
---
Attachment: YARN-2037-YARN-branch-2.v1.patch
Patch for branch-2 uploaded. Changed the constructor of
{{TestWorkPreservingUnmanagedAM}} a bit to make it compile.
> Add work preserving restart support for Unmanaged AMs
> -
>
> Key: YARN-2037
> URL: https://issues.apache.org/jira/browse/YARN-2037
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: 2.4.0
>Reporter: Karthik Kambatla
>Assignee: Botong Huang
> Attachments: YARN-2037.v1.patch, YARN-2037.v2.patch,
> YARN-2037.v3.patch, YARN-2037.v4.patch, YARN-2037-YARN-branch-2.v1.patch
>
>
> It would be nice to allow Unmanaged AMs also to restart in a work-preserving
> way.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2497) Changes for fair scheduler to support allocate resource respect labels
[
https://issues.apache.org/jira/browse/YARN-2497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated YARN-2497:
---
Attachment: YARN-2497.010.patch
Renamed the two getter methods, deleted the unused {{Resources}} method, and
made its sister method unused so I could delete it, too.
> Changes for fair scheduler to support allocate resource respect labels
> --
>
> Key: YARN-2497
> URL: https://issues.apache.org/jira/browse/YARN-2497
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
>Reporter: Wangda Tan
>Assignee: Daniel Templeton
> Attachments: YARN-2497.001.patch, YARN-2497.002.patch,
> YARN-2497.003.patch, YARN-2497.004.patch, YARN-2497.005.patch,
> YARN-2497.006.patch, YARN-2497.007.patch, YARN-2497.008.patch,
> YARN-2497.009.patch, YARN-2497.010.patch, YARN-2499.WIP01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7226) Whitelisted variables do not support delayed variable expansion
[
https://issues.apache.org/jira/browse/YARN-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-7226:
-
Attachment: YARN-7226-branch-2.006.patch
Thanks for the review and commit! Here's the patch for branch-2. I'll put up
the patch for branch-2.8 after Jenkins has had a chance to comment on this one.
> Whitelisted variables do not support delayed variable expansion
> ---
>
> Key: YARN-7226
> URL: https://issues.apache.org/jira/browse/YARN-7226
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.9.0, 2.8.1, 3.0.0-alpha4
>Reporter: Jason Lowe
>Assignee: Jason Lowe
> Attachments: YARN-7226.001.patch, YARN-7226.002.patch,
> YARN-7226.003.patch, YARN-7226.004.patch, YARN-7226.005.patch,
> YARN-7226.006.patch, YARN-7226-branch-2.006.patch
>
>
> The nodemanager supports a configurable list of environment variables, via
> yarn.nodemanager.env-whitelist, that will be propagated to the container's
> environment unless those variables were specified in the container launch
> context. Unfortunately the handling of these whitelisted variables prevents
> using delayed variable expansion. For example, if a user shipped their own
> version of hadoop with their job via the distributed cache and specified:
> {noformat}
> HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
> {noformat}
> as part of their job, the variable will be set as the *literal* string:
> {noformat}
> $PWD/my-private-hadoop/
> {noformat}
> rather than having $PWD expand to the container's current directory as it
> does for any other, non-whitelisted variable being set to the same value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7246) Fix the default docker binary path
[ https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189716#comment-16189716 ] Shane Kumpf commented on YARN-7246: --- Thanks [~ebadger] - new patch attached that addresses the compiler warnings. > Fix the default docker binary path > -- > > Key: YARN-7246 > URL: https://issues.apache.org/jira/browse/YARN-7246 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Shane Kumpf >Assignee: Shane Kumpf >Priority: Blocker > Attachments: YARN-7246-branch-2.8.2.001.patch, > YARN-7246-branch-2.8.2.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7246) Fix the default docker binary path
[
https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189710#comment-16189710
]
Hadoop QA commented on YARN-7246:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.8.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
28s{color} | {color:green} branch-2.8.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-2.8.2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} branch-2.8.2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
44s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:d0a0f24 |
| JIRA Issue | YARN-7246 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890148/YARN-7246-branch-2.8.2.002.patch
|
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 46bd1a5060c2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2.8.2 / 76e053a |
| Default Java | 1.7.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17743/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17743/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix the default docker binary path
> --
>
> Key: YARN-7246
> URL: https://issues.apache.org/jira/browse/YARN-7246
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Shane Kumpf
>Assignee: Shane Kumpf
>Priority: Blocker
> Attachments: YARN-7246-branch-2.8.2.001.patch,
> YARN-7246-branch-2.8.2.002.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6856) Support CLI for Node Attributes Mapping
[ https://issues.apache.org/jira/browse/YARN-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-6856: Attachment: YARN-6856-yarn-3409.003.patch Uploading new patch which fixes the name and as well some modifications for one of the files to apply successfully ! > Support CLI for Node Attributes Mapping > --- > > Key: YARN-6856 > URL: https://issues.apache.org/jira/browse/YARN-6856 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, capacityscheduler, client >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6856-YARN-3409.001.patch, > YARN-6856-YARN-3409.002.patch, YARN-6856-yarn-3409.003.patch > > > This focuses on the new CLI for the mapping of Node Attributes -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7246) Fix the default docker binary path
[ https://issues.apache.org/jira/browse/YARN-7246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Kumpf updated YARN-7246: -- Attachment: YARN-7246-branch-2.8.2.002.patch > Fix the default docker binary path > -- > > Key: YARN-7246 > URL: https://issues.apache.org/jira/browse/YARN-7246 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Shane Kumpf >Assignee: Shane Kumpf >Priority: Blocker > Attachments: YARN-7246-branch-2.8.2.001.patch, > YARN-7246-branch-2.8.2.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6856) Support CLI for Node Attributes Mapping
[ https://issues.apache.org/jira/browse/YARN-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189668#comment-16189668 ] Naganarasimha G R commented on YARN-6856: - Yes [~sunilg], working on it. Will upload shortly. > Support CLI for Node Attributes Mapping > --- > > Key: YARN-6856 > URL: https://issues.apache.org/jira/browse/YARN-6856 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, capacityscheduler, client >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6856-YARN-3409.001.patch, > YARN-6856-YARN-3409.002.patch > > > This focuses on the new CLI for the mapping of Node Attributes -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6856) Support CLI for Node Attributes Mapping
[ https://issues.apache.org/jira/browse/YARN-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189662#comment-16189662 ] Sunil G commented on YARN-6856: --- I think patch needs a change in name as we use yarn-3409 and a rebase. > Support CLI for Node Attributes Mapping > --- > > Key: YARN-6856 > URL: https://issues.apache.org/jira/browse/YARN-6856 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, capacityscheduler, client >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6856-YARN-3409.001.patch, > YARN-6856-YARN-3409.002.patch > > > This focuses on the new CLI for the mapping of Node Attributes -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6855) CLI Proto Modifications to support Node Attributes
[ https://issues.apache.org/jira/browse/YARN-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189601#comment-16189601 ] Naganarasimha G R commented on YARN-6855: - Thanks for the review and commit ! I have changed the status of YARN-6856 to patch available ! > CLI Proto Modifications to support Node Attributes > -- > > Key: YARN-6855 > URL: https://issues.apache.org/jira/browse/YARN-6855 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, capacityscheduler, client >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Fix For: yarn-3409 > > Attachments: YARN-6855-YARN-3409.001.patch, > YARN-6855-YARN-3409.002.patch, YARN-6855-YARN-3409.003.patch, > YARN-6855-YARN-3409.004.patch, YARN-6855-YARN-3409.005.patch, > YARN-6855-yarn-3409.006.patch, YARN-6855-YARN-3409.006.patch > > > This jira focuses only on the proto modifications required for the CLI -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
