[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[
https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-2729:
Attachment: YARN-2729.20151023-1.patch
Thanks [~rohithsharma] & [~sunilg], for the comments . I have updated the patch
with the approach specified. but i have a query.
bq. If you call super.serviceStop() first, D1 clean up will not happen first
rather, D1(D,C,B,A)-cleanup D1 which is at the end.
I could not get the above statement clearly, IIUC the order would be either
*D1,D,C,B,A* when {{super.serviceStop()}} is called first and *D,D1,C,B,A* when
{{super.serviceStop()}} is called at the last. Latter is what we are trying to
achieve earlier, correct me if i am wrong.
> Support script based NodeLabelsProvider Interface in Distributed Node Label
> Configuration Setup
> ---
>
> Key: YARN-2729
> URL: https://issues.apache.org/jira/browse/YARN-2729
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch,
> YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch,
> YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch,
> YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch,
> YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch,
> YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch,
> YARN-2729.20150925-1.patch, YARN-2729.20151015-1.patch,
> YARN-2729.20151019.patch, YARN-2729.20151023-1.patch,
> YARN-2729.20151310-1.patch, YARN-2729.20151310-2.patch
>
>
> Support script based NodeLabelsProvider Interface in Distributed Node Label
> Configuration Setup .
> Miscellaneous Issues :
> # In configurationNodeLabelsProvider instead of taking multiple labels from
> single configuration, we need to support exclusive configuration for
> partition (single label).
> # Proper logging when registration of Node Fails
> # Classloader was not getting reset from custom class loader in
> TestConfigurationNodeLabelsProvider.java which could make test cases fail in
> certain conditions
> # In ResourceTrackerService we need to consider distributed configuration
> only when node labels are enabled. leads to lots of logs in certain conditions
> # NodeLabelsProvider needs to be a interface rather than abstract class
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20151019.patch Thanks [~sunilg] for the comments and better late than never ! bq. We can do super.stop() first from ScriptBasedNodeLabelsProvider. But I feel it may not be better, so could we reset shexec to null after attached process is destroyed (may also need a null check in timetask if we do this). IMHO i feel the former approach is better than the latter as i am not inclined towards approach of having checks in the parts of code which gets frequently executed just to avoid corner case . Hence i am reverting the fix for the comment which was given by [~rohithsharma]. IIUC there are no impacts if we do super.serviceStop though theoritically its not the right way to do . Thoughts ? bq. But if we have multiple lines from input string which starts with NODE_LABEL_PARTITION_PATTERN, then only last line will be set to nodePartitionLabel. I didnt understand this part. As discussed offline i have done these modifications as a node can belong to only one Partition Label and when we support Constraints we can support more patterns later. Other issues i have fixed. If better approach is possible for first will rework. > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151015-1.patch, > YARN-2729.20151019.patch, YARN-2729.20151310-1.patch, > YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . > Miscellaneous Issues : > # In configurationNodeLabelsProvider instead of taking multiple labels from > single configuration, we need to support exclusive configuration for > partition (single label). > # Proper logging when registration of Node Fails > # Classloader was not getting reset from custom class loader in > TestConfigurationNodeLabelsProvider.java which could make test cases fail in > certain conditions > # In ResourceTrackerService we need to consider distributed configuration > only when node labels are enabled. leads to lots of logs in certain conditions > # NodeLabelsProvider needs to be a interface rather than abstract class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-2729: -- Assignee: Naganarasimha G R (was: Sunil G) > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151015-1.patch, > YARN-2729.20151310-1.patch, YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . > Miscellaneous Issues : > # In configurationNodeLabelsProvider instead of taking multiple labels from > single configuration, we need to support exclusive configuration for > partition (single label). > # Proper logging when registration of Node Fails > # Classloader was not getting reset from custom class loader in > TestConfigurationNodeLabelsProvider.java which could make test cases fail in > certain conditions > # In ResourceTrackerService we need to consider distributed configuration > only when node labels are enabled. leads to lots of logs in certain conditions > # NodeLabelsProvider needs to be a interface rather than abstract class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Description: Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . Miscellaneous Issues : # In configurationNodeLabelsProvider instead of taking multiple labels from single configuration, we need to support exclusive configuration for partition (single label). # Proper logging when registration of Node Fails # Classloader was not getting reset from custom class loader in TestConfigurationNodeLabelsProvider.java which could make test cases fail in certain conditions # In ResourceTrackerService we need to consider distributed configuration only when node labels are enabled. leads to lots of logs in certain conditions # NodeLabelsProvider needs to be a interface rather than abstract class was: Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . Miscellaneous Issues : # In configurationNodeLabelsProvider instead of taking multiple labels from single configuration, we need to support configuration for partition (single label). # > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151310-1.patch, > YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . > Miscellaneous Issues : > # In configurationNodeLabelsProvider instead of taking multiple labels from > single configuration, we need to support exclusive configuration for > partition (single label). > # Proper logging when registration of Node Fails > # Classloader was not getting reset from custom class loader in > TestConfigurationNodeLabelsProvider.java which could make test cases fail in > certain conditions > # In ResourceTrackerService we need to consider distributed configuration > only when node labels are enabled. leads to lots of logs in certain conditions > # NodeLabelsProvider needs to be a interface rather than abstract class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Description: Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . Miscellaneous Issues : # In configurationNodeLabelsProvider instead of taking multiple labels from single configuration, we need to support configuration for partition (single label). # was:Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151310-1.patch, > YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . > Miscellaneous Issues : > # In configurationNodeLabelsProvider instead of taking multiple labels from > single configuration, we need to support configuration for partition (single > label). > # -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20151015-1.patch Attaching a patch to make NodeLabelsProvider as a interface instead of abstract class > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151015-1.patch, > YARN-2729.20151310-1.patch, YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . > Miscellaneous Issues : > # In configurationNodeLabelsProvider instead of taking multiple labels from > single configuration, we need to support exclusive configuration for > partition (single label). > # Proper logging when registration of Node Fails > # Classloader was not getting reset from custom class loader in > TestConfigurationNodeLabelsProvider.java which could make test cases fail in > certain conditions > # In ResourceTrackerService we need to consider distributed configuration > only when node labels are enabled. leads to lots of logs in certain conditions > # NodeLabelsProvider needs to be a interface rather than abstract class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20151310-2.patch Hi [~rohithsharma] bq. When I run tests locally at package level i.e org.apache.hadoop.yarn.server.nodemanager.nodelabels, all the tests cases in TestScriptBasedNodeLabelsProvider are failing with below error Thanks for pointing out this issue, Its happening only when the order of execution is ConfigNLP and ScriptNLP and the temp dir containing Yarn-Site.xml is getting deleted in \@AfterClass > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151310-1.patch, > YARN-2729.20151310-2.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20151310-1.patch Thanks for the review [~rohithsharma], bq. Does ScriptBasedNodeLabelsProvider is periodically runs similar to ConfigurationNodeLabelsProvider? Yes as it extends AbstractNodeLabelsProvider it supports periodic execution. bq. Is there anywher documented how the script output should be? yes its documented in *yarn-default.xml* and javadocs for the class *ScriptBasedNodeLabelsProvider.java*. Do you have any other place in mind where it can be updated? bq. About the test TestNodeStatusUpdaterForLabels failing randomly.May be some race condition is exist. Yes this is issue has been already raised as YARN-4169, but nutting to do in specific to this jira and was not actually reproducible in my local setup. bq. When I run tests locally at package level i.e org.apache.hadoop.yarn.server.nodemanager.nodelabels, all the tests cases in TestScriptBasedNodeLabelsProvider are failing with below error Not facing this issue, will sync offline with you on this. Other issues have been handled in the patch along with one more minor issue which was got during testing, In ResourceTrackerService we are checking for distributed or delegated-centralized without checking for node labels enabled which caused unnecessary logs on RM side. > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup > --- > > Key: YARN-2729 > URL: https://issues.apache.org/jira/browse/YARN-2729 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, > YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, > YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, > YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, > YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, > YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch, > YARN-2729.20150925-1.patch, YARN-2729.20151310-1.patch > > > Support script based NodeLabelsProvider Interface in Distributed Node Label > Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[
https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-2729:
Attachment: YARN-2729.20150925-1.patch
Hi [~wangda] & [~rohithsharma],
Please find the attached patch with the offline discussed modifications
{quote}
*For Config :*
Currently we are supporting configuration param :
"yarn.nodemanager.node-labels.provider.configured-node-labels"
so instead can we have = >
yarn.nodemanager.node-labels.provider.configured-node-partition which accepts
only single partition and in future we can add
yarn.nodemanager.node-labels.provider.configured-node-constraints which can
support multiple constraints
*For Script :*
Currently in the patch i am searching for lines starting with "NODE_LABELS:" in
script output and labels separated by "," though we currently support only a
single Partition per node.
So my suggestion we can currently support for parsing lines starting with
"NODE_PARTITION:" and send the string following this pattern as Node Label.
Later in future we can support for "NODE_CONSTRAINTS:" and accept multiple
constraints for it.
{quote}
> Support script based NodeLabelsProvider Interface in Distributed Node Label
> Configuration Setup
> ---
>
> Key: YARN-2729
> URL: https://issues.apache.org/jira/browse/YARN-2729
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch,
> YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch,
> YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch,
> YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch,
> YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch,
> YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch,
> YARN-2729.20150925-1.patch
>
>
> Support script based NodeLabelsProvider Interface in Distributed Node Label
> Configuration Setup .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150830-1.patch Attaching a patch to sync with the changes of YARN-2923. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, YARN-2729.20150517-1.patch, YARN-2729.20150830-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150517-1.patch Hi [~wangda] # rebased the patch on top of 3565 # Moved common code which was earlier here to YARN-2923, as 2923 jira will be going first Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch, YARN-2729.20150517-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150404-1.patch Hi [~wangda], Uploading the patch with fixing the review comments given earlier... Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, YARN-2729.20150402-1.patch, YARN-2729.20150404-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150402-1.patch Attaching a patch to disable the timer and include the missing files Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch, YARN-2729.20150402-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150401-1.patch Hi [~wangda], Not sure the reason for failure in applying the patch ...I am able to do this in local setup (succeded after fuzz offset 65 lines), so i am rebasing it and uploading it. Also as per [~john.jian.fang] he wanted some kind of configuration such that the timer task doesn't run, as their usecase requires updating of node labels only during registration and do not want timer to continuously running for this. So do we need to support this (as unnecessary additional timer thread will be running) ? and if so can we check for interval-ms and decide based on its value (0 or -1 to not to launch the timer task) ? Also provide your feedback on earlier discussion : whether to do the changes required to make HadoopCommon's NodeHealthScriptRunner reuseable (In seperate jira) and rework on ScriptBasedNodeLabelsProvider ? Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch, YARN-2729.20150401-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150322-1.patch Hi [~wangda], Have rebased the patch based on changes in yarn-2495, one open point, whether to do the changes required to make HadoopCommon's NodeHealthScriptRunner reuseable (In seperate jira) and rework on ScriptBasedNodeLabelsProvider ? Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch, YARN-2729.20150322-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20150309-1.patch Hi [~wangda] Rebasing the patch Removing dependency on yarn-2923, Changed configuration names to suit current conf suggestions Have made it to fail fast on invalid configurations. If the above modifications are fine then will start looking into changes required to make HadoopCommon's NodeHealthScriptRunner to make it reuseable. (In seperate jira) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch, YARN-2729.20150309-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20141210-1.patch Re basing the patch and incorporating the changes based on yarn-2923 (changes in yarn configuration) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch, YARN-2729.20141210-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20141120-1.patch Updating patch with [~wangda]'s review comments Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch, YARN-2729.20141120-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20141031-1.patch Updated with review comments and test cases Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch, YARN-2729.20141031-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20141024-1.patch Hi [~wangda] As per our discussion I have done the scriptNodeLabelsPRovider similar to the NodeHealthCheckerService. And moved the yarn configuration changes related to the script into this jira itself. Please review Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch, YARN-2729.20141024-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2729) Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup
[ https://issues.apache.org/jira/browse/YARN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-2729: Attachment: YARN-2729.20141023-1.patch Attaching the WIP patch for this part of the issue... Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup --- Key: YARN-2729 URL: https://issues.apache.org/jira/browse/YARN-2729 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Naganarasimha G R Assignee: Naganarasimha G R Attachments: YARN-2729.20141023-1.patch Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
