[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150527#comment-16150527
]
Sunil G commented on YARN-6933:
---
Yes +1 from me as well. I ll commit shortly
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch, YARN-6933-YARN-3926.009.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147646#comment-16147646
]
Hadoop QA commented on YARN-6933:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
28s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
11s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884504/YARN-6933-YARN-3926.009.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 4ecef5628fbb 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 38d04b8 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17209/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17209/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was autom
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147579#comment-16147579
]
Daniel Templeton commented on YARN-6933:
Fine by me. I'll let [~sunilg] have the final say.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch, YARN-6933-YARN-3926.009.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16147542#comment-16147542
]
Manikandan R commented on YARN-6933:
[~sunilg], [~dan...@cloudera.com] Thanks for your review. Modified the comments
and attached a new patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch, YARN-6933-YARN-3926.009.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145670#comment-16145670
]
Daniel Templeton commented on YARN-6933:
BTW, I'm otherwise fine with the patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145669#comment-16145669
]
Daniel Templeton commented on YARN-6933:
Interesting point, [~sunilg]. I kinda think "valid" is correct, in that we're
considering _memory_ to be a valid resource, just like _memory-mb_. Maybe we
should rephrase entirely. How about "reserved" instead of "valid"?
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145121#comment-16145121
]
Sunil G commented on YARN-6933:
---
Thanks [~maniraj...@gmail.com] I think message in latest patch seems fine. But
comment in {{checkMandatoryResources}} is not correct. {{as an valid}} should
be {{as invalid}}.
I think you can wait for [~templedf] comment before updating next patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145079#comment-16145079
]
Hadoop QA commented on YARN-6933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
53s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
18s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
11s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 0s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12884213/YARN-6933-YARN-3926.008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 5e5cf101b10f 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 38d04b8 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/17174/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/17174/artifact/patchprocess/whitespace-
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144977#comment-16144977
]
Manikandan R commented on YARN-6933:
[~dan...@cloudera.com] Attaching a new patch with the changes. New message
would be slightly different from others as 'memory' check is different from
'memory-mb' & 'vcores' checks. I think throwing the same exception message
would make some confusions to end users?. Thoughts?
In addition, logging a warn message as well
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch,
> YARN-6933-YARN-3926.008.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142851#comment-16142851
]
Daniel Templeton commented on YARN-6933:
The message used for CPUs and the other memory check is: "Attempt to re-define
mandatory resource 'memory-mb'. It can only be of type 'COUNTABLE' and have
units 'Mi'." Can we use that same message here?
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138690#comment-16138690
]
Hadoop QA commented on YARN-6933:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
7s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
8s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 54m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883358/YARN-6933-YARN-3926.007.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux a34eae277603 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 2144629 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17091/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17091/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was autom
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138568#comment-16138568
]
Manikandan R commented on YARN-6933:
[~dan...@cloudera.com] Thanks for taking a look. Incorporated your suggestions
and attached new patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch, YARN-6933-YARN-3926.007.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16137136#comment-16137136
]
Daniel Templeton commented on YARN-6933:
Thanks for the patches, [~maniraj...@gmail.com]. I just have two tiny
quibbles. First, can we add a comment to explain why we're checking for
{{"memory"}} in addition to {{MEMORY}}, i.e. for historical reasons we still
support "memory" as a valid resource name. Second, for the exception error
message, can we mirror the messages for the {{MEMORY}} and {{VCORES}}
exceptions?
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136452#comment-16136452
]
Sunil G commented on YARN-6933:
---
Latest patch seems fine. Committing later today if there are no objections.
cc/ [~templedf]
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136391#comment-16136391
]
Hadoop QA commented on YARN-6933:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
33s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
3s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
35s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 67m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12883048/YARN-6933-YARN-3926.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux b2d3d13f0ad6 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 7d9f92c |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/17056/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17056/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was auto
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136309#comment-16136309
]
Manikandan R commented on YARN-6933:
Rebasing patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch,
> YARN-6933-YARN-3926.006.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135165#comment-16135165
]
Hadoop QA commented on YARN-6933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-6933 does not apply to YARN-3926. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882871/YARN-6933-YARN-3926.005.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/17030/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135143#comment-16135143
]
Manikandan R commented on YARN-6933:
Modified the exception message. Also fixed checkstyle issues and attached the
new patch.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch, YARN-6933-YARN-3926.005.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134127#comment-16134127
]
Sunil G commented on YARN-6933:
---
[~templedf], could you also please take a look.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133145#comment-16133145
]
Sunil G commented on YARN-6933:
---
Latest patch seems fine to me. May be exception message in
{{checkMandatoryResources}} could be improved given there is a new patch. I
will wait for [~templedf] also to take a look.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131916#comment-16131916
]
Hadoop QA commented on YARN-6933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
51s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
52s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 56s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 6 unchanged - 0 fixed = 8 total (was 6) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 31s{color}
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.api.TestPBImplRecords |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882528/YARN-6933-YARN-3926.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 6835c17afe67 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 210548b |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16985/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/16985/artifact/patchprocess/patch-unit-hadoop-ya
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131872#comment-16131872
]
Manikandan R commented on YARN-6933:
[~sunilg] Thanks for very detailed explanation.
Uploaded a new patch which will not allow "memory" resource from both users and
admin configuration. Since {{checkMandatoryResources}} being called from
{{getNodeResourceInformation}} method, even if admin configures "memory" as
part of node resources, it will not be allowed.
1. Taken care
2. I was wrong here in my last response. It has been kept intentionally to
ensure below block is working fine
{code}if (resourceInformationMap.containsKey(resourceName)) {
throw new YarnRuntimeException(
"Error in config, key '" + resourceName + "' specified twice");
}
{code}
3. vcores default unit is empty string (""). Passing as "G" is intentional to
produce the error case.
4. Taken care
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch,
> YARN-6933-YARN-3926.004.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129242#comment-16129242
]
Sunil G commented on YARN-6933:
---
I think i ll try to rephrase my comment
# Ideally we do not want any user to provide a resource named "memory-mb",
"memory", "vcores". These are mandatory resources and we do not want user to
configure same as new resource types
# Since we removed DISALLOWED_NAMES in this patch, *if* conditions in
{{ResourceUtils#checkMandatatoryResources}} will be hitting.
{code}
if (resourceInformationMap.containsKey(MEMORY)) {
ResourceInformation memInfo = resourceInformationMap.get(MEMORY);
String memUnits = ResourceInformation.MEMORY_MB.getUnits();
ResourceTypes memType = ResourceInformation.MEMORY_MB.getResourceType();
if (!memInfo.getUnits().equals(memUnits) || !memInfo.getResourceType()
.equals(memType)) {
throw new YarnRuntimeException(
"Attempt to re-define mandatory resource 'memory-mb'. It can only"
+ " be of type 'COUNTABLE' and have units 'Mi'.");
}
}
if (resourceInformationMap.containsKey(VCORES)) {
...
}
{code}
so if any resource is added w/o units as "Mi" or type as "COUNTABLE", those
will be thrown as exception.
Hence i think we just need to make add if condition in
checkMandatatoryResources method to throw an exception.
{code}
if (resourceName.equals("memory")) {
throw new YarnRuntimeException(
"Resource type cannot be named '" + resourceName
+ "'. That name is disallowed.");
}
{code}
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129145#comment-16129145
]
Manikandan R commented on YARN-6933:
[~sunilg] Currently {{ResourceUtils#checkMandatoryResources}} does 2 checks -
units and resource type (for ex, countable) only if {{resourceInformationMap}}
has MEMORY_MB and VCORES. Otherwise, not. Given that we don't want to define
the constants, Is doing resource type check for "memory" good enough only when
map has this key? Inaddition, do we need to add "memory" in
{{addMandatoryResources()}} method if it is not there like other params? In
general, Are we treating "memory" as mandatory resources?
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129071#comment-16129071
]
Sunil G commented on YARN-6933:
---
bq.Shall I define "M" as default for "memory"?
I think lets not define any default other than existing MEMORY_MB and VCORE
objects in ResourceUtils.
[~maniraj...@gmail.com] Could you please give a patch as per above comments.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126094#comment-16126094
]
Manikandan R commented on YARN-6933:
[~sunilg] Thanks for the review.
1. I was under the impression only memory-mb and vcores are mandatory
resources. Will do check even for "memory". Need to define constant in
{{ResourceInformation}}. Shall I define "M" as default for "memory"?
2. Yes, will need to fix.
3. vcores default unit is empty string (""). It is intentional pass to produce
the error case.
4. Yes, we will need to do changes in junits accordingly.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125515#comment-16125515
]
Sunil G commented on YARN-6933:
---
[~maniraj...@gmail.com]
Thanks for the patch.
Quick comments:
# In {{checkMandatoryResources}}, you could also check for *memory* along with
memory-mb
# In {{resource-types-error-4.xml}}, resource1 is repeated 2 times
# In {{resource-types-error-2.xml}}, why vcores unit is *G* ?
# {{TestResourceUtils.java}} could be updated with memory as well post comment
#1
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116995#comment-16116995
]
Hadoop QA commented on YARN-6933:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
14s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
41s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 47s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 3 new + 30 unchanged - 1 fixed = 33 total (was 31) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
21s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880666/YARN-6933-YARN-3926.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 5cfca8819b70 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 1b586d7 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16743/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16743/testReport/ |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16116909#comment-16116909
]
Manikandan R commented on YARN-6933:
[~dan...@cloudera.com]
Fixed junit failures by changing the test inputs. Earlier,
{{testGetResourceTypesConfigErrors()}} method had passed because
{{DISALLOWED_NAMES}} block was throwing exception. After removing the same
block, it requires a change in test inputs. Can you confirm this?
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch, YARN-6933-YARN-3926.003.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115820#comment-16115820
]
Hadoop QA commented on YARN-6933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3926 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
1s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
53s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} YARN-3926 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} YARN-3926 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
52s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 2s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 3 new + 30 unchanged - 1 fixed = 33 total (was 31) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 26s{color}
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.util.resource.TestResourceUtils |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880563/YARN-6933-YARN-3926.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5ead59482f9c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-3926 / 1b586d7 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16725/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/16725/artifact/patchprocess/whitespace-eol.txt
|
| unit |
https://builds.apache.org/job/PreCommit-Y
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115805#comment-16115805
]
Manikandan R commented on YARN-6933:
[~dan...@cloudera.com] Thanks.
{quote}You should also be able to delete DISALLOWED_NAMES, since that was the
only place it was used{quote}
Ok, will do.
{quote}Why the change in the test file? I should expect that setting the units
to nothing for memory would be an error, since memory expects "Mi" as the
units.{quote}
{{checkMandatoryResources}} does this check only for "memory-mb" and "vcores"
resources. I thought purpose of {code}String[] mem3 = { "memory", "" };{code}
test case is to ensure DISALLOWED_NAMES check functionality because it was
failing there only. However, I modified the test case input to "memory-mb" to
cover your case. or Am i missing anything?
Rebased my working copy and attached patch for review.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch,
> YARN-6933-YARN-3926.002.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115458#comment-16115458
]
Hadoop QA commented on YARN-6933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-6933 does not apply to YARN-3926. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6933 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880525/YARN-6933-YARN-3926.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16720/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115428#comment-16115428
]
Daniel Templeton commented on YARN-6933:
Thanks, [~maniraj...@gmail.com]. You should also be able to delete
{{DISALLOWED_NAMES}}, since that was the only place it was used. Why the
change in the test file? I should expect that setting the units to nothing for
memory would be an error, since memory expects "Mi" as the units.
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6933) ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources() are duplicating work
[
https://issues.apache.org/jira/browse/YARN-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115362#comment-16115362
]
Manikandan R commented on YARN-6933:
Attached patch for review.
Also addressed few more changes -
1. Changes discussed in
https://issues.apache.org/jira/browse/YARN-6934?focusedCommentId=16114520&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16114520
2. Fixed typo issues in method names. for ex, addManadtoryResources()
> ResourceUtils.DISALLOWED_NAMES and ResourceUtils.checkMandatoryResources()
> are duplicating work
> ---
>
> Key: YARN-6933
> URL: https://issues.apache.org/jira/browse/YARN-6933
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Assignee: Manikandan R
> Labels: newbie++
> Attachments: YARN-6933-YARN-3926.001.patch
>
>
> Both are used to check that the mandatory resources were not redefined. Only
> one check is needed. I would recommend dropping {{DISALLOWED_RESOURCES}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
