[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289199#comment-16289199
]
Sunil G commented on YARN-7383:
---
Jenkins came clean. Committing shortly.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
> Attachments: YARN-7383.001.patch, YARN-7383.002.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289191#comment-16289191
]
genericqa commented on YARN-7383:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 24s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
15s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 47s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
3s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7383 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12900164/YARN-7383.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux cb3b0ba15570 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / fa4ecd8b |
| maven | version: Apache Maven 3.3.9 |
| Default
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289097#comment-16289097
]
Sunil G commented on YARN-7383:
---
TestPBImplRecords is passing locally. I ll still rerun jenkins to cross check.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
> Attachments: YARN-7383.001.patch, YARN-7383.002.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274172#comment-16274172
]
genericqa commented on YARN-7383:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 44s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
21s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 56s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 6s{color}
| {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 74m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.api.TestPBImplRecords |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7383 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12900164/YARN-7383.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux c265c0c4311c 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274105#comment-16274105
]
Gergely Novák commented on YARN-7383:
-
Rebased.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
> Attachments: YARN-7383.001.patch, YARN-7383.002.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272874#comment-16272874
]
Sunil G commented on YARN-7383:
---
I think this patch need a rebase. [~GergelyNovak], could you please help to do
that, I ll commit post that.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
> Attachments: YARN-7383.001.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265326#comment-16265326
]
Sunil G commented on YARN-7383:
---
+1 from me. Committing later today if no objections.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
> Attachments: YARN-7383.001.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16239571#comment-16239571
]
Hadoop QA commented on YARN-7383:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
41s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 35s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 40s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
37s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
43s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | YARN-7383 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12895665/YARN-7383.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 2d28fee70823 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / b6d8f87 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238065#comment-16238065
]
Daniel Templeton commented on YARN-7383:
The patch looks fine to me.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>Assignee: Gergely Novák
>Priority: Major
> Attachments: YARN-7383.001.patch
>
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217215#comment-16217215
]
Wangda Tan commented on YARN-7383:
--
[~templedf], I think supporting domain name for resource type is a clean
design, this can clearly tell what is supported by YARN and what is supported
by customer.
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216431#comment-16216431
]
Sunil G commented on YARN-7383:
---
Yes. Since we can have more devices like fpga etc, this naming model was
making sense {{yarn.io/gpu}}
Considering we could have a "dot" in names, I also think we need to change that
logic while loading node resources. And we could update doc also ?
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216108#comment-16216108
]
Jonathan Hung commented on YARN-7383:
-
Currently the resource name for GPU is {{yarn.io/gpu}}. (If people use
GpuResourcePlugin for GPU resource discovery, this ticket doesn't affect GPUs,
but if we stick to using "." convention for namespace, this ticket will affect
specifying node resources for them via {{node-resources.xml}}.)
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7383) Node resource not parsed correctly for resource names containing dot
[
https://issues.apache.org/jira/browse/YARN-7383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216104#comment-16216104
]
Daniel Templeton commented on YARN-7383:
The docs don't say a whole lot, but I would assume that '.' is not a valid
character for a resource name. CC: [~leftnoteasy], [~sunilg]
> Node resource not parsed correctly for resource names containing dot
>
>
> Key: YARN-7383
> URL: https://issues.apache.org/jira/browse/YARN-7383
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Jonathan Hung
>
> Currently when getting resource capabilities in {{node-resources.xml}}, the
> following parsing code is used: {noformat}String[] parts =
> prop.split("\\.");
> LOG.info("Found resource entry " + prop);
> if (parts.length == 4) {
> String resourceType = parts[3];
> if (!nodeResources.containsKey(resourceType)) {
> nodeResources
> .put(resourceType, ResourceInformation.newInstance(resourceType));
> }
> String units = getUnits(value);
> Long resourceValue =
> Long.valueOf(value.substring(0, value.length() - units.length()));
> nodeResources.get(resourceType).setValue(resourceValue);
> nodeResources.get(resourceType).setUnits(units);
> if (LOG.isDebugEnabled()) {
> LOG.debug("Setting value for resource type " + resourceType + " to "
> + resourceValue + " with units " + units);
> }
> }{noformat}
> If resource names contain dots though (e.g. for resource names in a
> namespace) this will not parse correctly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
