[
https://issues.apache.org/jira/browse/HDFS-7684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299229#comment-14299229
]
Xiaoyu Yao commented on HDFS-7684:
----------------------------------
Thanks [~tianyin] for reporting this. The one that you hit can be fixed by
changing the conf.get to conf.getTrimmed.
{code}
final String httpsAddrString = conf.get(
DFSConfigKeys.DFS_NAMENODE_SECONDARY_HTTPS_ADDRESS_KEY,
DFSConfigKeys.DFS_NAMENODE_SECONDARY_HTTPS_ADDRESS_DEFAULT);
InetSocketAddress httpsAddr = NetUtils.createSocketAddr(httpsAddrString);
{code}
Searched the call of NetUtils.createSocketAddr() in HDFS code, I found many
other places with similar untrimmed host:port issues. For example in
DataNodeManager#DataNodeManager() below. I think we should fix them as well
with this JIRA.
{code}
this.defaultXferPort = NetUtils.createSocketAddr(
conf.get(DFSConfigKeys.DFS_DATANODE_ADDRESS_KEY,
DFSConfigKeys.DFS_DATANODE_ADDRESS_DEFAULT)).getPort();
this.defaultInfoPort = NetUtils.createSocketAddr(
conf.get(DFSConfigKeys.DFS_DATANODE_HTTP_ADDRESS_KEY,
DFSConfigKeys.DFS_DATANODE_HTTP_ADDRESS_DEFAULT)).getPort();
this.defaultInfoSecurePort = NetUtils.createSocketAddr(
conf.get(DFSConfigKeys.DFS_DATANODE_HTTPS_ADDRESS_KEY,
DFSConfigKeys.DFS_DATANODE_HTTPS_ADDRESS_DEFAULT)).getPort();
this.defaultIpcPort = NetUtils.createSocketAddr(
conf.get(DFSConfigKeys.DFS_DATANODE_IPC_ADDRESS_KEY,
DFSConfigKeys.DFS_DATANODE_IPC_ADDRESS_DEFAULT)).getPort();
{code}
> The host:port settings of dfs.namenode.secondary.http-address should be
> trimmed before use
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-7684
> URL: https://issues.apache.org/jira/browse/HDFS-7684
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.4.1, 2.5.1
> Reporter: Tianyin Xu
> Assignee: Anu Engineer
>
> With the following setting,
> <property>
> <name>dfs.namenode.secondary.http-address</name>
> <value>myhostname:50090 </value>
> </property>
> The secondary NameNode could not be started
> $ hadoop-daemon.sh start secondarynamenode
> starting secondarynamenode, logging to
> /home/hadoop/hadoop-2.4.1/logs/hadoop-hadoop-secondarynamenode-xxx.out
> /home/hadoop/hadoop-2.4.1/bin/hdfs
> Exception in thread "main" java.lang.IllegalArgumentException: Does not
> contain a valid host:port authority: myhostname:50090
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:196)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.getHttpAddress(SecondaryNameNode.java:203)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:214)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:192)
> at
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:651)
> We were really confused and misled by the log message: we thought about the
> DNS problems (changed to IP address but no success) and the network problem
> (tried to test the connections with no success...)
> It turned out to be that the setting is not trimmed and the additional space
> character in the end of the setting caused the problem... OMG!!!...
> Searching on the Internet, we find we are really not alone. So many users
> encountered similar trim problems! The following lists a few:
> http://solaimurugan.blogspot.com/2013/10/hadoop-multi-node-cluster-configuration.html
> http://stackoverflow.com/questions/11263664/error-while-starting-the-hadoop-using-strat-all-sh
> https://issues.apache.org/jira/browse/HDFS-2799
> https://issues.apache.org/jira/browse/HBASE-6973
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
