[
https://issues.apache.org/jira/browse/HADOOP-7359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059096#comment-13059096
]
Travis Crawford commented on HADOOP-7359:
-----------------------------------------
@dhruba - Correct, by default there's no change from today; files are used, and
they do not refresh.
The version I'll use internally gets data from ZooKeeper, so its not
particularly reusable (hierarchy, data format, etc). I could see about posting
to github or something as an example. You're right though, a jdbc-based version
would likely be reusable by other sites.
> Pluggable interface for cluster membership
> ------------------------------------------
>
> Key: HADOOP-7359
> URL: https://issues.apache.org/jira/browse/HADOOP-7359
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Travis Crawford
> Attachments: HADOOP-7359.diff
>
>
> Currently Hadoop uses local files to determine cluster membership. With HDFS
> for example, dfs.hosts and dfs.hosts.exclude are used.
> To enable tighter integrations cluster membership should be an interface,
> with the current file-based functionality provided as the default
> implementation. The common case would be no functional change, however, sites
> could plug an alternative implementation in, such as pulling the machine
> lists from a machine database.
> DETAILS:
> Two machine lists, includes and excludes, are used to define cluster
> membership and state. HostsFileReader currently handles reading these lists
> from files, who's names are passed in by FSNamesystem for HDFS and JobTracker
> for MR.
> The proposed change is adding a HostsReader interface to common, and changing
> HostsFileReader to an abstract class that functions the same as today.
> Two new classes, DFSHostsFileReader and MRHostsFileReader, extend
> HostsFileReader and simply pass the appropriate file names in. These new
> classes are needed because config key names live outside common.
> Two new conf keys, defaulting to the file-based readers, would be added to
> choose a different hosts reader: dfs.namenode.hosts.reader.class
> mapreduce.jobtracker.hosts.reader.class
> Comments/suggestions? I have most of this written already but would love some
> feedback on the general idea before posting the diff.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
