[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[
https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101122#comment-16101122
]
Hadoop QA commented on YARN-3931:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 8 new + 39 unchanged - 0 fixed = 47 total (was 39) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 6s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-3931 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878938/YARN-3931.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux bdc06bf4831a 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a92bf39 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16552/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/16552/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16552/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[
https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252916#comment-15252916
]
Hadoop QA commented on YARN-3931:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} YARN-3931 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12745729/YARN-3931.001.patch |
| JIRA Issue | YARN-3931 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/11167/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> default-node-label-expression doesn’t apply when an application is submitted
> by RM rest api
> ---
>
> Key: YARN-3931
> URL: https://issues.apache.org/jira/browse/YARN-3931
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.6.0
> Environment: hadoop-2.6.0
>Reporter: kyungwan nam
>Assignee: kyungwan nam
> Labels: patch
> Attachments: YARN-3931.001.patch
>
>
> *
> yarn.scheduler.capacity..default-node-label-expression=large_disk
> * submit an application using rest api without "app-node-label-expression”,
> "am-container-node-label-expression”
> * RM doesn’t allocate containers to the hosts associated with large_disk node
> label
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252745#comment-15252745 ] Raul Gutierrez Segales commented on YARN-3931: -- [~wangda], [~kyungwan nam] - ping, can we apply this? > default-node-label-expression doesn’t apply when an application is submitted > by RM rest api > --- > > Key: YARN-3931 > URL: https://issues.apache.org/jira/browse/YARN-3931 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.6.0 > Environment: hadoop-2.6.0 >Reporter: kyungwan nam >Assignee: kyungwan nam > Labels: patch > Attachments: YARN-3931.001.patch > > > * > yarn.scheduler.capacity..default-node-label-expression=large_disk > * submit an application using rest api without "app-node-label-expression”, > "am-container-node-label-expression” > * RM doesn’t allocate containers to the hosts associated with large_disk node > label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680795#comment-14680795 ] Wangda Tan commented on YARN-3931: -- Sorry for response late. Fix looks good, thanks [~kyungwan nam]. +1 to fix application-name / application-type together in the patch, we need to make behavior consistent when using REST API / Java API to submit application. default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.6.0 Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: kyungwan nam Labels: patch Attachments: YARN-3931.001.patch * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14634406#comment-14634406 ] kyungwan nam commented on YARN-3931: application-name, application-type” are empty string, not default value when i submit an application by rest api without those. that's caused by same reason… can i fix it here? if you want i’ll create another issue. default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.6.0 Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: kyungwan nam Labels: patch Attachments: YARN-3931.001.patch * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629491#comment-14629491 ] Naganarasimha G R commented on YARN-3931: - Hi [~kyungwan nam], Thanks for raising the issue ... i have assigned this jira to my name but if you are interested to further look into this jira and solve it . please reassign. default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: Naganarasimha G R * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[
https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629489#comment-14629489
]
kyungwan nam commented on YARN-3931:
node-label-expression is initialized to empty string
{code}
...
public ApplicationSubmissionContextInfo() {
applicationId = ;
applicationName = ;
containerInfo = new ContainerLaunchContextInfo();
resource = new ResourceInfo();
priority = Priority.UNDEFINED.getPriority();
isUnmanagedAM = false;
cancelTokensWhenComplete = true;
keepContainers = false;
applicationType = ;
tags = new HashSetString();
appNodeLabelExpression = ;
amContainerNodeLabelExpression = ;
}
{code}
but, check whether node-label-expression is null or not
{code}
// check labels in the resource request.
String labelExp = resReq.getNodeLabelExpression();
// if queue has default label expression, and RR doesn't have, use the
// default label expression of queue
if (labelExp == null queueInfo != null) {
labelExp = queueInfo.getDefaultNodeLabelExpression();
resReq.setNodeLabelExpression(labelExp);
}
{code}
default-node-label-expression doesn’t apply when an application is submitted
by RM rest api
---
Key: YARN-3931
URL: https://issues.apache.org/jira/browse/YARN-3931
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Environment: hadoop-2.6.0
Reporter: kyungwan nam
Assignee: Naganarasimha G R
*
yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk
* submit an application using rest api without app-node-label-expression”,
am-container-node-label-expression”
* RM doesn’t allocate containers to the hosts associated with large_disk node
label
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629790#comment-14629790 ] Naganarasimha G R commented on YARN-3931: - [~kyungwan nam], Good that you are trying to contribute :), we need to request some committer to add you to the list of contributors but in the mean time you can upload the patch with test case i can help you in reviewing [~wangda tan], Can you please add [~kyungwan nam] to the contributor list and assign him this jira ? default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: Naganarasimha G R * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630010#comment-14630010 ] Wangda Tan commented on YARN-3931: -- Thanks for raising the issue [~kyungwan nam], I just added you to contributor list and assigned the JIRA to you. default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: kyungwan nam * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629701#comment-14629701 ] kyungwan nam commented on YARN-3931: hi, i couldn't reassign it to me. i think i don't have the privilege to assign issue default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: Naganarasimha G R * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[ https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630659#comment-14630659 ] Xianyin Xin commented on YARN-3931: --- This reminds me an earlier trouble i have met. Hi [~Naganarasimha], can we consider to remove the node label expression in the code? It seems not make sense we set a node label as . For node label expression, it should be some_label or null. Just an unrigorous thoughts, what do you think? default-node-label-expression doesn’t apply when an application is submitted by RM rest api --- Key: YARN-3931 URL: https://issues.apache.org/jira/browse/YARN-3931 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Environment: hadoop-2.6.0 Reporter: kyungwan nam Assignee: kyungwan nam Attachments: YARN-3931.001.patch * yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk * submit an application using rest api without app-node-label-expression”, am-container-node-label-expression” * RM doesn’t allocate containers to the hosts associated with large_disk node label -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3931) default-node-label-expression doesn’t apply when an application is submitted by RM rest api
[
https://issues.apache.org/jira/browse/YARN-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630747#comment-14630747
]
Hadoop QA commented on YARN-3931:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 32s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 46s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 22s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 25s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 51m 13s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 39s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12745729/YARN-3931.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / ee36f4f |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8565/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8565/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8565/console |
This message was automatically generated.
default-node-label-expression doesn’t apply when an application is submitted
by RM rest api
---
Key: YARN-3931
URL: https://issues.apache.org/jira/browse/YARN-3931
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.6.0
Environment: hadoop-2.6.0
Reporter: kyungwan nam
Assignee: kyungwan nam
Labels: patch
Attachments: YARN-3931.001.patch
*
yarn.scheduler.capacity.queue-path.default-node-label-expression=large_disk
* submit an application using rest api without app-node-label-expression”,
am-container-node-label-expression”
* RM doesn’t allocate containers to the hosts associated with large_disk node
label
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
