[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated YARN-8256: - Target Version/s: 3.5.0 (was: 3.4.0) > Pluggable provider for node membership management > - > > Key: YARN-8256 > URL: https://issues.apache.org/jira/browse/YARN-8256 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager >Affects Versions: 2.8.3, 3.0.2 >Reporter: Dagang Wei >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > h1. Background > HDFS-7541 introduced a pluggable provider framework for node membership > management, which gives HDFS the flexibility to have different ways to manage > node membership for different needs. > [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] > is the class which provides the abstraction. Currently, there are 2 > implementations in the HDFS codebase: > 1) > [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] > which uses 2 config files which are defined by the properties dfs.hosts and > dfs.hosts.exclude. > 2) > [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] > which uses a single JSON file defined by the property dfs.hosts. > dfs.namenode.hosts.provider.classname is the property determining which > implementation is used > h1. Problem > YARN should be consistent with HDFS in terms of pluggable provider for node > membership management. The absence of it makes YARN impossible to have other > config sources, e.g., ZooKeeper, database, other config file formats, etc. > h1. Proposed solution > [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] > is the class for managing YARN node membership today. It uses > [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] > to read config files specified by the property > yarn.resourcemanager.nodes.include-path for nodes to include and > yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. > The proposed solution is to > 1) introduce a new interface {color:#008000}HostsConfigManager{color} which > provides the abstraction for node membership management. Update > {color:#008000}NodeListManager{color} to depend on > {color:#008000}HostsConfigManager{color} instead of > {color:#008000}HostsFileReader{color}. Then create a wrapper class for > {color:#008000}HostsFileReader{color} which implements the interface. > 2) introduce a new config property > {color:#008000}yarn.resourcemanager.hosts-config.manager.class{color} for > specifying the implementation class. Set the default value to the wrapper > class of {color:#008000}HostsFileReader{color} for backward compatibility > between new code and old config. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated YARN-8256: --- Target Version/s: 3.4.0 (was: 3.3.0) Bulk update: moved all 3.3.0 non-blocker issues, please move back if it is a blocker. > Pluggable provider for node membership management > - > > Key: YARN-8256 > URL: https://issues.apache.org/jira/browse/YARN-8256 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager >Affects Versions: 2.8.3, 3.0.2 >Reporter: Dagang Wei >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > h1. Background > HDFS-7541 introduced a pluggable provider framework for node membership > management, which gives HDFS the flexibility to have different ways to manage > node membership for different needs. > [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] > is the class which provides the abstraction. Currently, there are 2 > implementations in the HDFS codebase: > 1) > [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] > which uses 2 config files which are defined by the properties dfs.hosts and > dfs.hosts.exclude. > 2) > [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] > which uses a single JSON file defined by the property dfs.hosts. > dfs.namenode.hosts.provider.classname is the property determining which > implementation is used > h1. Problem > YARN should be consistent with HDFS in terms of pluggable provider for node > membership management. The absence of it makes YARN impossible to have other > config sources, e.g., ZooKeeper, database, other config file formats, etc. > h1. Proposed solution > [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] > is the class for managing YARN node membership today. It uses > [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] > to read config files specified by the property > yarn.resourcemanager.nodes.include-path for nodes to include and > yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. > The proposed solution is to > 1) introduce a new interface {color:#008000}HostsConfigManager{color} which > provides the abstraction for node membership management. Update > {color:#008000}NodeListManager{color} to depend on > {color:#008000}HostsConfigManager{color} instead of > {color:#008000}HostsFileReader{color}. Then create a wrapper class for > {color:#008000}HostsFileReader{color} which implements the interface. > 2) introduce a new config property > {color:#008000}yarn.resourcemanager.hosts-config.manager.class{color} for > specifying the implementation class. Set the default value to the wrapper > class of {color:#008000}HostsFileReader{color} for backward compatibility > between new code and old config. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8256: - Target Version/s: 3.3.0 (was: 3.2.0) Bulk update: moved all 3.2.0 non-blocker issues, please move back if it is a blocker. > Pluggable provider for node membership management > - > > Key: YARN-8256 > URL: https://issues.apache.org/jira/browse/YARN-8256 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager >Affects Versions: 2.8.3, 3.0.2 >Reporter: Dagang Wei >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > h1. Background > HDFS-7541 introduced a pluggable provider framework for node membership > management, which gives HDFS the flexibility to have different ways to manage > node membership for different needs. > [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] > is the class which provides the abstraction. Currently, there are 2 > implementations in the HDFS codebase: > 1) > [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] > which uses 2 config files which are defined by the properties dfs.hosts and > dfs.hosts.exclude. > 2) > [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] > which uses a single JSON file defined by the property dfs.hosts. > dfs.namenode.hosts.provider.classname is the property determining which > implementation is used > h1. Problem > YARN should be consistent with HDFS in terms of pluggable provider for node > membership management. The absence of it makes YARN impossible to have other > config sources, e.g., ZooKeeper, database, other config file formats, etc. > h1. Proposed solution > [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] > is the class for managing YARN node membership today. It uses > [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] > to read config files specified by the property > yarn.resourcemanager.nodes.include-path for nodes to include and > yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. > The proposed solution is to > 1) introduce a new interface {color:#008000}HostsConfigManager{color} which > provides the abstraction for node membership management. Update > {color:#008000}NodeListManager{color} to depend on > {color:#008000}HostsConfigManager{color} instead of > {color:#008000}HostsFileReader{color}. Then create a wrapper class for > {color:#008000}HostsFileReader{color} which implements the interface. > 2) introduce a new config property > {color:#008000}yarn.resourcemanager.hosts-config.manager.class{color} for > specifying the implementation class. Set the default value to the wrapper > class of {color:#008000}HostsFileReader{color} for backward compatibility > between new code and old config. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated YARN-8256: - Description: h1. Background HDFS-7541 introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARN impossible to have other config sources, e.g., ZooKeeper, database, other config file formats, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership today. It uses [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] to read config files specified by the property yarn.resourcemanager.nodes.include-path for nodes to include and yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. The proposed solution is to 1) introduce a new interface {color:#008000}HostsConfigManager{color} which provides the abstraction for node membership management. Update {color:#008000}NodeListManager{color} to depend on {color:#008000}HostsConfigManager{color} instead of {color:#008000}HostsFileReader{color}. Then create a wrapper class for {color:#008000}HostsFileReader{color} which implements the interface. 2) introduce a new config property {color:#008000}yarn.resourcemanager.hosts-config.manager.class{color} for specifying the implementation class. Set the default value to the wrapper class of {color:#008000}HostsFileReader{color} for backward compatibility between new code and old config. was: h1. Background HDFS-7541 introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARN impossible to have other config sources, e.g., ZooKeeper, database, other config file formats, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership
[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated YARN-8256: - Description: h1. Background HDFS-7541 introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARN impossible to have other config sources, e.g., ZooKeeper, database, other config file formats, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership today. It uses [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] to read config files specified by the property yarn.resourcemanager.nodes.include-path for nodes to include and yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. The proposed solution is to 1) introduce a new interface {color:#008000}HostsConfigManager{color} which provides the abstraction for node membership management. Update {color:#008000}NodeListManager{color} to depend on {color:#008000}HostsConfigManager{color} instead of {color:#008000}HostsFileReader{color}. Then create a wrapper class for {color:#008000}HostsFileReader{color} which implements the interface. 2) introduce a new config property {color:#008000}yarn.resourcemanager.hosts.provider.classname{color} for specifying the implementation class. Set the default value to the wrapper class of {color:#008000}HostsFileReader{color} for backward compatibility between new code and old config. was: h1. Background HDFS-7541 introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARN impossible to have other config sources, e.g., ZooKeeper, database, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership today. It uses
[jira] [Updated] (YARN-8256) Pluggable provider for node membership management
[ https://issues.apache.org/jira/browse/YARN-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei updated YARN-8256: - Description: h1. Background HDFS-7541 introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARN impossible to have other config sources, e.g., ZooKeeper, database, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership today. It uses [HostsFileReader|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java] to read config files specified by the property yarn.resourcemanager.nodes.include-path for nodes to include and yarn.resourcemanager.nodes.nodes.exclude-path for nodes to exclude. The proposed solution is to 1) introduce a new interface {color:#008000}HostsConfigManager{color} which provides the abstraction for node membership management. Update {color:#008000}NodeListManager{color} to depend on {color:#008000}HostsConfigManager{color} instead of {color:#008000}HostsFileReader{color}. Then create a wrapper class for {color:#008000}HostsFileReader{color} which implements the interface. 2) introduce a new config property {color:#008000}yarn.resourcemanager.hosts.provider.classname{color} for specifying the implementation class. Set the default value to the wrapper class of {color:#008000}HostsFileReader{color} for backward compatibility between new code and old config. was: h1. Background [HDFS-7541|https://issues.apache.org/jira/browse/HDFS-7541] introduced a pluggable provider framework for node membership management, which gives HDFS the flexibility to have different ways to manage node membership for different needs. [org.apache.hadoop.hdfs.server.blockmanagement.HostConfigManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostConfigManager.java] is the class which provides the abstraction. Currently, there are 2 implementations in the HDFS codebase: 1) [org.apache.hadoop.hdfs.server.blockmanagement.HostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HostFileManager.java] which uses 2 config files which are defined by the properties dfs.hosts and dfs.hosts.exclude. 2) [org.apache.hadoop.hdfs.server.blockmanagement.CombinedHostFileManager|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java] which uses a single JSON file defined by the property dfs.hosts. dfs.namenode.hosts.provider.classname is the property determining which implementation is used h1. Problem YARN should be consistent with HDFS in terms of pluggable provider for node membership management. The absence of it makes YARM impossible to have other config sources, e.g., ZooKeeper, database, etc. h1. Proposed solution [org.apache.hadoop.yarn.server.resourcemanager.NodesListManager|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java] is the class for managing YARN node membership