Hi Aneela,

1. Looks like you have attached the hdfs-site.xml from 'hadoop-master' node. 
For this node datanode connection is successfull as mentioned in below logs.

         2016-06-29 10:01:35,700 INFO 
SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for 
nn/hadoop-master@platalyticsrealm (auth:KERBEROS)
2016-06-29 10:01:35,744 INFO 
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
 Authorization successful for nn/hadoop-master@platalyticsrealm (auth:KERBEROS) 
for protocol=interface org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol

         2016-06-29 10:01:36,845 INFO org.apache.hadoop.net.NetworkTopology: 
Adding a new node: /default-rack/192.168.23.206:1004


2. For the other node, 'hadoop-slave' kerberos athentication is successfull, 
but ServiceAuthorizationManager check failed.

2016-06-29 10:01:37,474 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth 
successful for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
2016-06-29 10:01:37,512 WARN 
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
 Authorization failed for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS) for 
protocol=interface org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol, 
expected client Kerberos principal is nn/hadoop-slave@platalyticsrealm
2016-06-29 10:01:37,514 INFO org.apache.hadoop.ipc.Server: Connection from 
192.168.23.207:32807 for protocol 
org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol is unauthorized for 
user dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)

reason could be mostly, "dfs.datanode.kerberos.principal" configuration in both 
nodes differ. I can see that this configuration in hadoop-master's 
hdfs-site.xml set to 'nn/_HOST@platalyticsrealm' but it might have been set to 
'dn/_HOST@platalyticsrealm' in hadoop-slave node's configurations.

Please change this configuration in all nodes to 'dn/_HOST@platalyticsrealm' 
and restart all NNs and DNs, and check again.

If this does not help, then please share the hdfs-site.xml of hadoop-slave node 
too.

-Vinay

From: Aneela Saleem [mailto:ane...@platalytics.com]
Sent: 29 June 2016 21:35
To: user@hadoop.apache.org
Subject: Fwd: datanode is unable to connect to namenode



Sent from my iPhone

Begin forwarded message:
From: Aneela Saleem <ane...@platalytics.com<mailto:ane...@platalytics.com>>
Date: 29 June 2016 at 10:16:36 GMT+5
To: "sreebalineni ." <sreebalin...@gmail.com<mailto:sreebalin...@gmail.com>>
Subject: Re: datanode is unable to connect to namenode
Attached are the log files for datanode and namenode. Also i have attached 
hdfs-site.xml for namenode please check if there are any issues in 
configuration file.

I have following two Kerberos Principals:

nn/hadoop-master
dn/hadoop-slave

i have copied kdc.conf and krb5.conf on both nodes. Also i copied keytab file 
on datanode. And i have starting services with principal nn/hadoop-master.

On Wed, Jun 29, 2016 at 9:35 AM, sreebalineni . 
<sreebalin...@gmail.com<mailto:sreebalin...@gmail.com>> wrote:
Probably sharing both Name node and datanode logs may help.

On Wed, Jun 29, 2016 at 10:02 AM, Aneela Saleem 
<ane...@platalytics.com<mailto:ane...@platalytics.com>> wrote:
Following is the result of telnet

Trying 192.168.23.206...
Connected to hadoop-master.
Escape character is '^]'.

On Wed, Jun 29, 2016 at 3:57 AM, Aneela Saleem 
<ane...@platalytics.com<mailto:ane...@platalytics.com>> wrote:
Thanks Sreebalineni for the response.

This is the result of the netstat -a | grep 8020 command
tcp        0      0 hadoop-master:8020      *:*                     LISTEN
tcp        0      0 hadoop-master:33356     hadoop-master:8020      ESTABLISHED
tcp        0      0 hadoop-master:8020      hadoop-master:33356     ESTABLISHED
tcp        0      0 hadoop-master:55135     hadoop-master:8020      TIME_WAIT

And this is my /etc/hosts file
#127.0.0.1      localhost
#127.0.1.1      vm6-VirtualBox
192.168.23.206  hadoop-master platalytics.com<http://platalytics.com> 
vm6-VirtualBox
192.168.23.207  hadoop-slave
# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters


Can you please tell me what's wrong with above configuration and how can i 
check whether it is firewall issue?

Thanks

On Wed, Jun 29, 2016 at 12:11 AM, sreebalineni . 
<sreebalin...@gmail.com<mailto:sreebalin...@gmail.com>> wrote:

Are you able to telnet ping. Check the firewalls as well
On Jun 29, 2016 12:39 AM, "Aneela Saleem" 
<ane...@platalytics.com<mailto:ane...@platalytics.com>> wrote:
Hi all,

I have setup two nodes cluster with security enabled. I have everything running 
successful like namenode, datanode, resourcemanager, nodemanager, 
jobhistoryserver etc. But datanode is unable to connect to namenode, as i can 
see only one node on the web UI. checking logs of datanode gives following 
warning:

WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to 
server: hadoop-master/192.168.23.206:8020<http://192.168.23.206:8020>

Rest of the things look fine. Please help me in this regard, what could be the 
issue?




Reply via email to