On one of the namenodes I have found the following warning: 2015-09-24 18:40:17,639 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/192.168.51.4:54310
On my master node I see that the process is running and has bound that port vagrant@hadoop-master:~/src$ sudo lsof -i :54310 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 7480 hadoop 202u IPv4 26931 0t0 TCP hadoop-master:54310 (LISTEN) java 7480 hadoop 212u IPv4 28758 0t0 TCP hadoop-master:54310->localhost:47226 (ESTABLISHED) java 7651 hadoop 238u IPv4 28247 0t0 TCP localhost:47226->hadoop-master:54310 (ESTABLISHED) hadoop@hadoop-master:~$ jps 7856 SecondaryNameNode 7651 DataNode 7480 NameNode 8106 Jps I don't appear to have any firewall rules interfering with traffic vagrant@hadoop-master:~/src$ sudo iptables --list Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination The iptables --list output is identical on hadoop-data1. I also show a process attempting to connect to hadoop-master vagrant@hadoop-data1:~$ sudo lsof -i :54310 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 3823 hadoop 238u IPv4 19304 0t0 TCP hadoop-data1:45600->hadoop-master:54310 (SYN_SENT) I am confused by the notation of hostname/IP:port. All help appreciated. Daniel On Thu, Sep 24, 2015 at 12:51 PM, Daniel Watrous <[email protected]> wrote: > I have a multi-node cluster with two datanodes. After running > start-dfs.sh, I show the following processes running > > hadoop@hadoop-master:~$ $HADOOP_HOME/sbin/slaves.sh jps > hadoop-master: 10933 DataNode > hadoop-master: 10759 NameNode > hadoop-master: 11145 SecondaryNameNode > hadoop-master: 11567 Jps > hadoop-data1: 5186 Jps > hadoop-data1: 5059 DataNode > hadoop-data2: 5180 Jps > hadoop-data2: 5053 DataNode > > > However, the other two DataNodes aren't visible. > http://screencast.com/t/icsLnXXDk > > Where can I look for clues? >
