[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[
https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-2800:
-
Attachment: YARN-2800-20150122-1.patch
Updated patch.
I just checked code, only RMNodeLabelsManager possibly has multi-thread access
to NodeLabelsManager, {{nodeLabelsEnabled}} will be protected by write lock of
CommonNodeLabelsManager. So I think we don't need add volatile to it. And in
addition, it is only used in CommonsNodeLabelsManager, so make it private.
Please kindly review.
Thanks,
> Remove MemoryNodeLabelsStore and add a way to enable/disable node labels
> feature
>
>
> Key: YARN-2800
> URL: https://issues.apache.org/jira/browse/YARN-2800
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: client, resourcemanager
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch,
> YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch,
> YARN-2800-20141119-1.patch, YARN-2800-20141203-1.patch,
> YARN-2800-20141205-1.patch, YARN-2800-20141205-1.patch,
> YARN-2800-20150122-1.patch
>
>
> In the past, we have a MemoryNodeLabelStore, mostly for user to try this
> feature without configuring where to store node labels on file system. It
> seems convenient for user to try this, but actually it causes some bad use
> experience. User may add/remove labels, and edit capacity-scheduler.xml.
> After RM restart, labels will gone, (we store it in mem). And RM cannot get
> started if we have some queue uses labels, and the labels don't exist in
> cluster.
> As what we discussed, we should have an explicitly way to let user specify if
> he/she wants this feature or not. If node label is disabled, any operations
> trying to modify/use node labels will throw exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated YARN-2800: - Attachment: YARN-2800-20141205-1.patch Reattaching the patch by Wangda to kick Jenkins. > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch, > YARN-2800-20141119-1.patch, YARN-2800-20141203-1.patch, > YARN-2800-20141205-1.patch, YARN-2800-20141205-1.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Attachment: YARN-2800-20141205-1.patch Fixed UT failures. > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch, > YARN-2800-20141119-1.patch, YARN-2800-20141203-1.patch, > YARN-2800-20141205-1.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Attachment: YARN-2800-20141203-1.patch [~ozawa], thanks for your comment, make sense to me, updated. [~jianhe], could you take a look please? > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch, > YARN-2800-20141119-1.patch, YARN-2800-20141203-1.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Attachment: YARN-2800-20141119-1.patch > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch, > YARN-2800-20141119-1.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Attachment: YARN-2800-20141118-2.patch Updated patch, fixed UT > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch, YARN-2800-20141118-2.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Attachment: YARN-2800-20141118-1.patch Uploaded patch and kick Jenkins > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch, > YARN-2800-20141118-1.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Description: In the past, we have a MemoryNodeLabelStore, mostly for user to try this feature without configuring where to store node labels on file system. It seems convenient for user to try this, but actually it causes some bad experiecne. User may add/remove labels, and edit capacity-scheduler.xml. After RM restart, labels will gone, (we store it in mem). And RM cannot start if we have some queue uses labels, and the labels don't exist in cluster. As what we discussed, we should have an explicitly way to let user specify if he/she wants this feature or not. If node label is disabled, all operations trying to modify/use node labels will throw exception. was:Even though we have documented this, but it will be better to explicitly print a message in both RM/RMAdminCLI side to explicitly say that the node label being added will be lost across RM restart. > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad > experiecne. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot start > if we have some queue uses labels, and the labels don't exist in cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, all operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Description: In the past, we have a MemoryNodeLabelStore, mostly for user to try this feature without configuring where to store node labels on file system. It seems convenient for user to try this, but actually it causes some bad use experience. User may add/remove labels, and edit capacity-scheduler.xml. After RM restart, labels will gone, (we store it in mem). And RM cannot get started if we have some queue uses labels, and the labels don't exist in cluster. As what we discussed, we should have an explicitly way to let user specify if he/she wants this feature or not. If node label is disabled, any operations trying to modify/use node labels will throw exception. was: In the past, we have a MemoryNodeLabelStore, mostly for user to try this feature without configuring where to store node labels on file system. It seems convenient for user to try this, but actually it causes some bad experiecne. User may add/remove labels, and edit capacity-scheduler.xml. After RM restart, labels will gone, (we store it in mem). And RM cannot start if we have some queue uses labels, and the labels don't exist in cluster. As what we discussed, we should have an explicitly way to let user specify if he/she wants this feature or not. If node label is disabled, all operations trying to modify/use node labels will throw exception. > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch > > > In the past, we have a MemoryNodeLabelStore, mostly for user to try this > feature without configuring where to store node labels on file system. It > seems convenient for user to try this, but actually it causes some bad use > experience. User may add/remove labels, and edit capacity-scheduler.xml. > After RM restart, labels will gone, (we store it in mem). And RM cannot get > started if we have some queue uses labels, and the labels don't exist in > cluster. > As what we discussed, we should have an explicitly way to let user specify if > he/she wants this feature or not. If node label is disabled, any operations > trying to modify/use node labels will throw exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2800) Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature
[ https://issues.apache.org/jira/browse/YARN-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-2800: - Summary: Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature (was: Remove MemoryNodeLabelsStore and add a way to disable node labels feature) > Remove MemoryNodeLabelsStore and add a way to enable/disable node labels > feature > > > Key: YARN-2800 > URL: https://issues.apache.org/jira/browse/YARN-2800 > Project: Hadoop YARN > Issue Type: Sub-task > Components: client, resourcemanager >Reporter: Wangda Tan >Assignee: Wangda Tan > Attachments: YARN-2800-20141102-1.patch, YARN-2800-20141102-2.patch > > > Even though we have documented this, but it will be better to explicitly > print a message in both RM/RMAdminCLI side to explicitly say that the node > label being added will be lost across RM restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
