Yiqun Lin commented on HDFS-13214:

Hi [~Tao Jie], 
So I think we should set dfs.nameservice.id to ns1 or ns2 rather than ns-fed.
Yes, this should be right.

Today property dfs.nameservice.id is not a necessary one in a federation 
cluster (HA or non-HA), right?
>From the function {{DFSUtil.getNameServiceId(Configuration conf, String 
>addressKey)}} which used in the Router, {{dfs.nameservice.id}} doesn't must be 
>specified. If this key isn't set, then it find the nameservice Id by matching 
>the addressKey with the the address of the local node. If multiple addresses 
>that matched, then the error will be thrown.

To avoid some potential problems happening in setting up Router, we can 
recommend users to configure the {{dfs.nameservice.id}}. If the local node is 
in a HA mode, {{dfs.ha.namenode.id}} is also recommended to configure. 
Otherwise, it will also found this by matching way. This can be documented in 
the doc of RBF. Any other thoughts?

> RBF: Configuration on Router conflicts with client side configuration
> ---------------------------------------------------------------------
>                 Key: HDFS-13214
>                 URL: https://issues.apache.org/jira/browse/HDFS-13214
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 2.9.0
>            Reporter: Tao Jie
>            Priority: Major
> In a typical router-based federation cluster, hdfs-site.xml is supposed to be:
> {code}
> <property>
>     <name>dfs.nameservices</name>
>     <value>ns1,ns2,ns-fed</value>
>   </property>
>   <property>
>     <name>dfs.ha.namenodes.ns-fed</name>
>     <value>r1,r2</value>
>   </property>
>   <property>
>     <name>dfs.namenode.rpc-address.ns1</name>
>     <value>host1:8020</value>
>   </property>
>   <property>
>     <name>dfs.namenode.rpc-address.ns2</name>
>     <value>host2:8020</value>
>   </property>
>   <property>
>     <name>dfs.namenode.rpc-address.ns-fed.r1</name>
>     <value>host1:8888</value>
>   </property>
>   <property>
>     <name>dfs.namenode.rpc-address.ns-fed.r2</name>
>     <value>host2:8888</value>
>   </property>
> {code}
> {{dfs.ha.namenodes.ns-fed}} here is used for client to access the Router. 
> However with this configuration on server node, Router fails to start with 
> error:
> {code}
> org.apache.hadoop.HadoopIllegalArgumentException: Configuration has multiple 
> addresses that match local node's address. Please configure the system with 
> dfs.nameservice.id and dfs.ha.namenode.id
>         at org.apache.hadoop.hdfs.DFSUtil.getSuffixIDs(DFSUtil.java:1198)
>         at org.apache.hadoop.hdfs.DFSUtil.getNameServiceId(DFSUtil.java:1131)
>         at 
> org.apache.hadoop.hdfs.DFSUtil.getNamenodeNameServiceId(DFSUtil.java:1086)
>         at 
> org.apache.hadoop.hdfs.server.federation.router.Router.createLocalNamenodeHearbeatService(Router.java:466)
>         at 
> org.apache.hadoop.hdfs.server.federation.router.Router.createNamenodeHearbeatServices(Router.java:423)
>         at 
> org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:199)
>         at 
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
>         at 
> org.apache.hadoop.hdfs.server.federation.router.DFSRouter.main(DFSRouter.java:69)
> 2018-03-01 18:05:56,208 ERROR 
> org.apache.hadoop.hdfs.server.federation.router.DFSRouter: Failed to start 
> router
> {code}
> Then the router tries to find the local namenode, multiple properties: 
> {{dfs.namenode.rpc-address.ns1}}, {{dfs.namenode.rpc-address.ns-fed.r1}} 
> match the local address.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to