[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14538890#comment-14538890 ] Wangda Tan commented on YARN-3521: -- Thanks for updating, [~sunilg], Latest patch LGTM, +1. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch, 0006-YARN-3521.patch, 0007-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534632#comment-14534632
]
Hadoop QA commented on YARN-3521:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated
17 new checkstyle issues (total was 61, now 66). |
| {color:red}-1{color} | whitespace | 0m 5s | The patch has 18 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 53m 26s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 90m 6s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731449/0006-YARN-3521.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7b1ea9c |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7802/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7802/console |
This message was automatically generated.
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch,
0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch,
0006-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14535158#comment-14535158
]
Wangda Tan commented on YARN-3521:
--
[~sunilg],
Patch generally looks good, I think it's fine if adding a wrapper class is
unavoidable.
One minor comment is, could you update {{NodeToLabelsName}} to
{{NodeToLabelsEntryList}}?
Thanks,
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch,
0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch,
0006-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532966#comment-14532966 ] Sunil G commented on YARN-3521: --- Hi [~leftnoteasy] List of NodeToLabelsEntry will also have issues. By default queryParams can have primitive types. If we have objects as list, then these objects need to have constructor for string. In here, we need String and ListString for ctor. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530612#comment-14530612
]
Hadoop QA commented on YARN-3521:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 0s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 51s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated
14 new checkstyle issues (total was 58, now 69). |
| {color:red}-1{color} | whitespace | 0m 3s | The patch has 19 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 17s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 57m 30s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 94m 47s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12730808/0005-YARN-3521.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a583a40 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7734/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7734/console |
This message was automatically generated.
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch,
0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14530999#comment-14530999 ] Vinod Kumar Vavilapalli commented on YARN-3521: --- bq. I'm fine with dropping NodeLabelNames as well, if it can keep the REST returned structure clean bq. 1. NodeLabelsName and NodeToLabelsName classes are needed for replace and remove apis. Else as you mentioned, REST apis wont be clean enough. I hope this is fine. We can directly pass a String or a set of Strings as arguments? See getLabelsToNodes() web-service for example. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531176#comment-14531176
]
Wangda Tan commented on YARN-3521:
--
Just tried to deploy and see the REST API result, some comments:
1. label-mappings
{code}
entry
key
namey/name
exclusivitytrue/exclusivity
/key
value
nodeIDsListhost2:0/nodeIDsList
/value
/entry
{code}
I think it's better to rename {{nodeIDsList}} to {{node}}, so the result will
be
{code}
...
value
nodehost1:1234/node
nodehost2:5678/node
/value
{code}
2. nodeLabelsInfo:
Now it is:
{code}
nodeLabelsInfo
nodeLabels
namex/name
exclusivitytrue/exclusivity
/nodeLabels
nodeLabels
namey/name
exclusivitytrue/exclusivity
/nodeLabels
/nodeLabelsInfo
{code}
I suggest to change
{code}
@XmlElement(name = nodeLabelsInfo)
protected ArrayListNodeLabelInfo nodeLabelsInfo =
new ArrayListNodeLabelInfo();
{code}
To
{{name=nodeLabelInfo}}
So the result will be:
{code}
nodeLabelsInfo
nodeLabelInfo
namex/name
exclusivitytrue/exclusivity
/nodeLabelInfo
nodeLabelInfo
namey/name
exclusivitytrue/exclusivity
/nodeLabelInfo
/nodeLabelsInfo
{code}
3. nodeToLabelsName:
I suggest to add a {{@XmlRootElement(name = nodeLabelName)}} to
NodeLabelsName.nodeLabels to make the result to be:
{code}
nodeToLabelsName
nodeToLabels
entry
keynode1/key
value
nodeLabelNamex/nodeLabelName
/value
/entry
entry
keynode2/key
value
nodeLabelNamey/nodeLabelName
/value
/entry
entry
keynode3/key
value
nodeLabelNamey/nodeLabelName
nodeLabelNamez/nodeLabelName
/value
/entry
/nodeToLabels
/nodeToLabelsName
{code}
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch,
0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531212#comment-14531212 ] Wangda Tan commented on YARN-3521: -- Some more comments: - NodeLabelsName is not necessary for removeFromCluserNodeLabels/replaceLabelsOnNode, use ArrayList seems enough and clean to me - replaceLabelsOnNodes can take List of NodeToLabelsEntry, which contains String nodeId and ArrayListString labels, this will be simpler than maintaining two objects: NodeToLabelsName and NodeLabelsName to me. Sounds good? [~sunilg]/[~vinodkv]? Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531103#comment-14531103 ] Wangda Tan commented on YARN-3521: -- [~vinodkv], I just checked, we still need NodeLabelsName/NodeToLabelsName, since JAXB doesn't support nested Java data structure, for example: MapString, ListString is not supported. Instead, you have to use MapString, WrappedList, NodeLabelsName is the WrappedList here. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14529052#comment-14529052 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Thanks for updating, I have a offline sync with Vinod about using object or string in API, some suggestions: - addToClusterNodeLabel should be object, (you've done this in your patch) - getLabelsOnNode, getNodeToLabels, getLabelsToNodes should use object, this will make user can easily understand attributes of labels on nodes without calling getClusterNodeLabels. (You have done some of them, but getLabelsToNodes should be updated as well) - replace/remove should use list of label name only, label name is unique key of node labels, using NodeLabelInfo object here is unnecessary. - I found in your patch, when calling getNodeToLabels, it returns NodeLabelInfo with default attributes, we can fix this in separated patch (we need make changes to NodeLabelsManager too) - RPC API should be consistent with this, should be addressed in a separated JIRA. I'm fine with dropping NodeLabelNames as well, if it can keep the REST returned structure clean :). Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14526747#comment-14526747 ] Sunil G commented on YARN-3521: --- 1. bq.Should be exclusivity. Yes. I updated the same 2. bq.Did we ever call these APIs stable? No. I have changed to a NodeLabelsInfo object and added new getter which can supply list/set of string names. 3. Why are we not dropping the name-only records? I have removed *NodeLabelsName*. And instead use *NodeLabelsInfo*, also added a new getter which can give back String of label names. NodeToLabelsName is renamed as NodeToLabelsInfo and internally it also uses NodeLabelInfo. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527697#comment-14527697 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Make sense to me, bq. IMO I also feel that NodeLabelManager apis can use Object rather than Strings. Admin interface can take this conversion logic. Sorry I didn't get this, currently addToCluserNodeLabels is already takes object instead of String and you're using it in your patch. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527904#comment-14527904 ] Sunil G commented on YARN-3521: --- [~leftnoteasy] Yes, Its not a valid point. replaceLabelsOnNode and removeFromClusterNodeLabels doesn't need node label object, name is enough. Pls discard my earlier comment. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch, 0004-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523947#comment-14523947 ] Wangda Tan commented on YARN-3521: -- Thanks for update, [~sunilg], Latest patch LGTM, one minor thing is changes of TestNodeLabelContainerAllocation could be reverted, I can do this while committing. +1, will wait for few days to see if there's any other comments. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, 0003-YARN-3521.patch In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523144#comment-14523144
]
Hadoop QA commented on YARN-3521:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 49s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 44s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 34s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated
42 new checkstyle issues (total was 320, now 348). |
| {color:red}-1{color} | whitespace | 0m 2s | The patch has 15 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 52m 18s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 88m 58s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729744/0003-YARN-3521.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1b3b9e5 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7570/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7570/console |
This message was automatically generated.
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch,
0003-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522180#comment-14522180
]
Wangda Tan commented on YARN-3521:
--
[~sunilg], Thanks for updating, just tried it locally, some comments:
1) it seems the structure of REST response is not correct for NodeLabelsInfo:
{code}
nodeLabelsInfo
nodeLabelsInfo
namex/name
exclusitytrue/exclusity
/nodeLabelsInfo
nodeLabelsInfo
namey/name
exclusitytrue/exclusity
/nodeLabelsInfo
/nodeLabelsInfo
{code}
It should be {{nodeLabelInfo}} instead of {{nodeLabelsInfo}}, could you solve
this issue?
2) It's better to add a test for specifying exclusivity when adding node
labels. (Verify exclusivity added to NodeLabelsManager).
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522141#comment-14522141
]
Hadoop QA commented on YARN-3521:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 19 line(s) that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 29s | The applied patch generated 3
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 63m 37s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 104m 45s | |
\\
\\
|| Reason || Tests ||
| Timed out tests |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729530/0002-YARN-3521.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e2e8f77 |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/whitespace.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7557/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7557/console |
This message was automatically generated.
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[
https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520226#comment-14520226
]
Wangda Tan commented on YARN-3521:
--
Hi Sunil,
Thanks for working on this, some comments:
NodelabelsInfo: (It should be NodeLabelInfo, right?)
- nodeLabelName: don't need call {{new String()}} since it will be always
initialized, and I prefer to call it name
- nodeLabelExclusivity - exclusivity
- Also getter
- Setters are not used by anybody, could be removed
- I'm not sure if you need add an empty constructure to make {{// JAXB needs
this}} like other infos?
- Could add a constructor of NodeLabelsInfo receives NodeLabel which will be
used by RMWebServices
- We may need to add a separated NodeLabelsInfo and it contains ArrayList of
NodeLabelInfo
NodeToLabelsInfo - NodeToLabelNames
addToClusterNodeLabels now receives Set as parameter, I'm not sure if it works,
could you add test to verify add/get node labels? Now
TestRMWebServicesNodeLabels will fail
Support return structured NodeLabel objects in REST API when call
getClusterNodeLabels
--
Key: YARN-3521
URL: https://issues.apache.org/jira/browse/YARN-3521
Project: Hadoop YARN
Issue Type: Sub-task
Components: api, client, resourcemanager
Reporter: Wangda Tan
Assignee: Sunil G
Attachments: 0001-YARN-3521.patch
In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should
make the same change in REST API side to make them consistency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516214#comment-14516214 ] Sunil G commented on YARN-3521: --- Yes. [~leftnoteasy] This change suggestion looks fine for me. I will update patch on same. Also i will rename the Ticket name based on the last update. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514642#comment-14514642 ] Wangda Tan commented on YARN-3521: -- To make REST API compatible with CLI as well as YarnClient, I suggest to make following changes: 1) NodeLabelsInfo - NodeLabelsName 2) Create a new NodeLabelsInfo contains String name and boolean exclusivity, potentially more fields in the future. 3) Only change add/getClusterNodeLabels to use the new NodeLabelsInfo, and other uses NodeLabelsName. Thoughts? Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505437#comment-14505437 ] Sunil G commented on YARN-3521: --- Recently have done few work in Rest. I wud like to take over, pls reassign otherwise. Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505503#comment-14505503 ] Wangda Tan commented on YARN-3521: -- [~sunilg], thanks for taking this, it's yours :) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels -- Key: YARN-3521 URL: https://issues.apache.org/jira/browse/YARN-3521 Project: Hadoop YARN Issue Type: Sub-task Components: api, client, resourcemanager Reporter: Wangda Tan Assignee: Sunil G In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
