[
https://issues.apache.org/jira/browse/HDFS-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001749#comment-13001749
]
Jitendra Nath Pandey commented on HDFS-1689:
--------------------------------------------
Design goals of this change:
1. Existing single namenode configuration must work with federation without any
change.
2. Configuration for the all the nodes should be same, with no need to set the
same parameter names to a different
values on different nodes.
Design:
1. A new logical ID called "NameServiceID" is being added. This identifies the
namenodes and
secondary/backup/checkpointer namenodes belonging together under it. Only the
namenode and
secondary/backup/checkpointer node related parameters are expected to be
different for different nodes in the cluster.
In order to support a single configuration file, the existing parameter name
that are different for different namenodes are suffixed with nameServiceID. The
configuration parameters that need this suffix (depending on the installation)
are:
Namenode params:
- dfs.namenode.rpc-address
- dfs.namenode.servicerpc-address
- dfs.namenode.http-address
- dfs.namenode.https-address
- dfs.namenode.keytab.file
Secondary Namenode params:
- dfs.namenode.secondary.http-address
- dfs.secondary.namenode.keytab.file
Backup node params:
- dfs.namenode.backup.address
- dfs.namenode.backup.http-address
- dfs.namenode.backup.dnrpc-address
2. Two new configuration items are added:
- dfs.federation.nameservices: is configured with list of comma separated
nameserviceIDs. This will be used by
Datanodes to determine all the namenodes in the cluster.
- dfs.federation.nameservice.id: This is a parameter that needs to be set
only on namenodes/secondary
namenodes/backup/checkpointer. On datanodes this parameter is not used. This is
set on each namnode/secondary namenode
to the corresponding nameServiceID. This can be set by XML include of another
file that just defines this parameter on namenodes and empty file on datanodes.
Alternatively this can be set by passing
-Ddfs.federation.nameservice=<nameServiceID> to JVM.
3. When a namenode starts up, it reads the corresponding nameServiceID. It then
loads the namenode specific parameters and sets them in the generic keys
(without namesapceID suffix), in runtime configuration. Only generic params are
used from then on. The same mechanism is done on secondary/backup/checkpointers
also.
> Hdfs Federation: Configuration for namenodes
> --------------------------------------------
>
> Key: HDFS-1689
> URL: https://issues.apache.org/jira/browse/HDFS-1689
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: Federation Branch
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Fix For: Federation Branch
>
> Attachments: HDFS-1689.1.patch, HDFS-1689.2.patch
>
>
> We need ability to specify some configuration parameters which are namenode
> specific.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
