[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538890#comment-14538890 ] Wangda Tan commented on YARN-3521: -- Thanks for updating, [~sunilg], Latest patch LGTM, +1. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch, > 0006-YARN-3521.patch, 0007-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536241#comment-14536241 ] Hadoop QA commented on YARN-3521: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 49s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 32s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 43s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 50s | The applied patch generated 5 new checkstyle issues (total was 61, now 54). | | {color:red}-1{color} | whitespace | 0m 6s | The patch has 22 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:green}+1{color} | yarn tests | 52m 25s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 89m 15s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12731679/0007-YARN-3521.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 6471d18 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7844/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/7844/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7844/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7844/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7844/console | This message was automatically generated. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch, > 0006-YARN-3521.patch, 0007-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535158#comment-14535158 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Patch generally looks good, I think it's fine if adding a wrapper class is unavoidable. One minor comment is, could you update {{NodeToLabelsName}} to {{NodeToLabelsEntryList}}? Thanks, > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch, > 0006-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534632#comment-14534632 ] Hadoop QA commented on YARN-3521: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 34s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated 17 new checkstyle issues (total was 61, now 66). | | {color:red}-1{color} | whitespace | 0m 5s | The patch has 18 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 34s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:green}+1{color} | yarn tests | 53m 26s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 90m 6s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12731449/0006-YARN-3521.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 7b1ea9c | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7802/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7802/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7802/console | This message was automatically generated. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch, > 0006-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14532966#comment-14532966 ] Sunil G commented on YARN-3521: --- Hi [~leftnoteasy] List of NodeToLabelsEntry will also have issues. By default queryParams can have primitive types. If we have objects as list, then these objects need to have constructor for string. In here, we need String and List for ctor. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531212#comment-14531212 ] Wangda Tan commented on YARN-3521: -- Some more comments: - NodeLabelsName is not necessary for removeFromCluserNodeLabels/replaceLabelsOnNode, use ArrayList seems enough and clean to me - replaceLabelsOnNodes can take List of NodeToLabelsEntry, which contains String nodeId and ArrayList labels, this will be simpler than maintaining two objects: NodeToLabelsName and NodeLabelsName to me. Sounds good? [~sunilg]/[~vinodkv]? > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531176#comment-14531176 ] Wangda Tan commented on YARN-3521: -- Just tried to deploy and see the REST API result, some comments: 1. label-mappings {code} y true host2:0 {code} I think it's better to rename {{nodeIDsList}} to {{node}}, so the result will be {code} ... host1:1234 host2:5678 {code} 2. nodeLabelsInfo: Now it is: {code} x true y true {code} I suggest to change {code} @XmlElement(name = "nodeLabelsInfo") protected ArrayList nodeLabelsInfo = new ArrayList(); {code} To {{name="nodeLabelInfo"}} So the result will be: {code} x true y true {code} 3. nodeToLabelsName: I suggest to add a {{@XmlRootElement(name = "nodeLabelName")}} to NodeLabelsName.nodeLabels to make the result to be: {code} node1 x node2 y node3 y z {code} > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531103#comment-14531103 ] Wangda Tan commented on YARN-3521: -- [~vinodkv], I just checked, we still need NodeLabelsName/NodeToLabelsName, since JAXB doesn't support nested Java data structure, for example: Map> is not supported. Instead, you have to use Map, NodeLabelsName is the WrappedList here. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530999#comment-14530999 ] Vinod Kumar Vavilapalli commented on YARN-3521: --- bq. I'm fine with dropping NodeLabelNames as well, if it can keep the REST returned structure clean bq. 1. NodeLabelsName and NodeToLabelsName classes are needed for replace and remove apis. Else as you mentioned, REST apis wont be clean enough. I hope this is fine. We can directly pass a String or a set of Strings as arguments? See getLabelsToNodes() web-service for example. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14530612#comment-14530612 ] Hadoop QA commented on YARN-3521: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 15m 0s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 41s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 51s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated 14 new checkstyle issues (total was 58, now 69). | | {color:red}-1{color} | whitespace | 0m 3s | The patch has 19 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 37s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 17s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:green}+1{color} | yarn tests | 57m 30s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 94m 47s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12730808/0005-YARN-3521.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / a583a40 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7734/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7734/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7734/console | This message was automatically generated. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch, 0005-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529052#comment-14529052 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Thanks for updating, I have a offline sync with Vinod about using object or string in API, some suggestions: - addToClusterNodeLabel should be object, (you've done this in your patch) - getLabelsOnNode, getNodeToLabels, getLabelsToNodes should use object, this will make user can easily understand attributes of labels on nodes without calling getClusterNodeLabels. (You have done some of them, but getLabelsToNodes should be updated as well) - replace/remove should use list of label name only, label name is unique key of node labels, using NodeLabelInfo object here is unnecessary. - I found in your patch, when calling getNodeToLabels, it returns NodeLabelInfo with default attributes, we can fix this in separated patch (we need make changes to NodeLabelsManager too) - RPC API should be consistent with this, should be addressed in a separated JIRA. I'm fine with dropping NodeLabelNames as well, if it can keep the REST returned structure clean :). > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527904#comment-14527904 ] Sunil G commented on YARN-3521: --- [~leftnoteasy] Yes, Its not a valid point. replaceLabelsOnNode and removeFromClusterNodeLabels doesn't need node label object, name is enough. Pls discard my earlier comment. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527697#comment-14527697 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Make sense to me, bq. IMO I also feel that NodeLabelManager apis can use Object rather than Strings. Admin interface can take this conversion logic. Sorry I didn't get this, currently addToCluserNodeLabels is already takes object instead of String and you're using it in your patch. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch, 0004-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526747#comment-14526747 ] Sunil G commented on YARN-3521: --- 1. bq.Should be exclusivity. Yes. I updated the same 2. bq.Did we ever call these APIs stable? No. I have changed to a NodeLabelsInfo object and added new getter which can supply list/set of string names. 3. Why are we not dropping the name-only records? I have removed *NodeLabelsName*. And instead use *NodeLabelsInfo*, also added a new getter which can give back String of label names. NodeToLabelsName is renamed as NodeToLabelsInfo and internally it also uses NodeLabelInfo. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523947#comment-14523947 ] Wangda Tan commented on YARN-3521: -- Thanks for update, [~sunilg], Latest patch LGTM, one minor thing is changes of TestNodeLabelContainerAllocation could be reverted, I can do this while committing. +1, will wait for few days to see if there's any other comments. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523144#comment-14523144 ] Hadoop QA commented on YARN-3521: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 49s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 2 new or modified test files. | | {color:green}+1{color} | javac | 7m 44s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 34s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 46s | The applied patch generated 42 new checkstyle issues (total was 320, now 348). | | {color:red}-1{color} | whitespace | 0m 2s | The patch has 15 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 33s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:green}+1{color} | yarn tests | 52m 18s | Tests passed in hadoop-yarn-server-resourcemanager. | | | | 88m 58s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12729744/0003-YARN-3521.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 1b3b9e5 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/whitespace.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7570/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7570/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7570/console | This message was automatically generated. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch, > 0003-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522180#comment-14522180 ] Wangda Tan commented on YARN-3521: -- [~sunilg], Thanks for updating, just tried it locally, some comments: 1) it seems the structure of REST response is not correct for NodeLabelsInfo: {code} x true y true {code} It should be {{nodeLabelInfo}} instead of {{nodeLabelsInfo}}, could you solve this issue? 2) It's better to add a test for specifying exclusivity when adding node labels. (Verify exclusivity added to NodeLabelsManager). > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522141#comment-14522141 ] Hadoop QA commented on YARN-3521: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:red}-1{color} | whitespace | 0m 0s | The patch has 19 line(s) that end in whitespace. | | {color:green}+1{color} | javac | 7m 34s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 5m 29s | The applied patch generated 3 additional checkstyle issues. | | {color:green}+1{color} | install | 1m 34s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce any new Findbugs (version 2.0.3) warnings. | | {color:red}-1{color} | yarn tests | 63m 37s | Tests failed in hadoop-yarn-server-resourcemanager. | | | | 104m 45s | | \\ \\ || Reason || Tests || | Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12729530/0002-YARN-3521.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / e2e8f77 | | whitespace | https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/whitespace.txt | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/checkstyle-result-diff.txt | | hadoop-yarn-server-resourcemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/7557/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/7557/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/7557/console | This message was automatically generated. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch, 0002-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520226#comment-14520226 ] Wangda Tan commented on YARN-3521: -- Hi Sunil, Thanks for working on this, some comments: NodelabelsInfo: (It should be NodeLabelInfo, right?) - nodeLabelName: don't need call {{new String()}} since it will be always initialized, and I prefer to call it "name" - nodeLabelExclusivity -> exclusivity - Also getter - Setters are not used by anybody, could be removed - I'm not sure if you need add an empty constructure to make {{// JAXB needs this}} like other infos? - Could add a constructor of NodeLabelsInfo receives NodeLabel which will be used by RMWebServices - We may need to add a separated NodeLabelsInfo and it contains ArrayList of NodeLabelInfo NodeToLabelsInfo -> NodeToLabelNames addToClusterNodeLabels now receives Set as parameter, I'm not sure if it works, could you add test to verify add/get node labels? Now TestRMWebServicesNodeLabels will fail > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > Attachments: 0001-YARN-3521.patch > > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516214#comment-14516214 ] Sunil G commented on YARN-3521: --- Yes. [~leftnoteasy] This change suggestion looks fine for me. I will update patch on same. Also i will rename the Ticket name based on the last update. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14514642#comment-14514642 ] Wangda Tan commented on YARN-3521: -- To make REST API compatible with CLI as well as YarnClient, I suggest to make following changes: 1) NodeLabelsInfo -> NodeLabelsName 2) Create a new NodeLabelsInfo contains String name and boolean exclusivity, potentially more fields in the future. 3) Only change add/getClusterNodeLabels to use the new NodeLabelsInfo, and other uses NodeLabelsName. Thoughts? > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505503#comment-14505503 ] Wangda Tan commented on YARN-3521: -- [~sunilg], thanks for taking this, it's yours :) > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3521) Support return structured NodeLabel objects in REST API when call getClusterNodeLabels
[ https://issues.apache.org/jira/browse/YARN-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505437#comment-14505437 ] Sunil G commented on YARN-3521: --- Recently have done few work in Rest. I wud like to take over, pls reassign otherwise. > Support return structured NodeLabel objects in REST API when call > getClusterNodeLabels > -- > > Key: YARN-3521 > URL: https://issues.apache.org/jira/browse/YARN-3521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Sunil G > > In YARN-3413, yarn cluster CLI returns NodeLabel instead of String, we should > make the same change in REST API side to make them consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)