If it's an HA cluster, is it possible the client doesn't have the proper HA configuration so it doesn't know what host name to connect to?
Otherwise, the usual suspect is the firewall configuration between the client and the NameNode. On Tue, Oct 24, 2023 at 9:05 AM Harry Jamison <harryjamiso...@yahoo.com.invalid> wrote: > I feel like I am doing something really dumb here, but my namenode is > having a connection refused on port 8020. > > There is nothing in the logs that seems to indicate an error as far as I > can tell > > ps aux shows the namenode is running > > root 13169 10196 9 21:18 pts/1 00:00:02 > /usr/lib/jvm/java-11-openjdk-amd64//bin/java -Dproc_namenode > -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender > -Dhadoop.security.logger=INFO,RFAS > -Dyarn.log.dir=/hadoop/hadoop/hadoop-3.3.6/logs -Dyarn.log.file=hadoop.log > -Dyarn.home.dir=/hadoop/hadoop/hadoop-3.3.6 -Dyarn.root.logger=INFO,console > -Djava.library.path=/hadoop/hadoop/hadoop-3.3.6/lib/native > -Dhadoop.log.dir=/hadoop/hadoop/hadoop-3.3.6/logs > -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/hadoop/hadoop/hadoop-3.3.6 > -Dhadoop.id.str=root -Dhadoop.root.logger=TRACE,console > -Dhadoop.policy.file=hadoop-policy.xml > org.apache.hadoop.hdfs.server.namenode.NameNode > > Netstat shows that this port is not open but others are > root@vmnode1:/hadoop/hadoop/hadoop# netstat -tulapn|grep 802 > tcp 0 0 192.168.1.159:8023 0.0.0.0:* > LISTEN 16347/java > tcp 0 0 192.168.1.159:8022 0.0.0.0:* > LISTEN 16347/java > tcp 0 0 192.168.1.159:8022 192.168.1.159:56830 > ESTABLISHED 16347/java > tcp 0 0 192.168.1.159:56830 192.168.1.159:8022 > ESTABLISHED 13889/java > tcp 0 0 192.168.1.159:8022 192.168.1.104:58264 > ESTABLISHED 16347/java > > > From the namenode logs I see that it has 8020 as the expected port > [2023-10-23 21:18:21,739] INFO fs.defaultFS is hdfs://vmnode1:8020/ > (org.apache.hadoop.hdfs.server.namenode.NameNodeUtils) > [2023-10-23 21:18:21,739] INFO Clients should use vmnode1:8020 to access > this namenode/service. (org.apache.hadoop.hdfs.server.namenode.NameNode) > > My datanodes seem to be connecting, because I see that information about 0 > invalid blocks in the logs > [2023-10-24 09:03:21,255] INFO BLOCK* registerDatanode: from > DatanodeRegistration(192.168.1.159:9866, > datanodeUuid=fbefce35-15f7-43df-a666-ecc90f4bef0f, infoPort=9864, > infoSecurePort=0, ipcPort=9867, > storageInfo=lv=-57;cid=CID-0b66d2f6-6c6a-4f3f-bdb1-b1ab0c947d00;nsid=2036303633;c=1697774550786) > storage fbefce35-15f7-43df-a666-ecc90f4bef0f > (org.apache.hadoop.hdfs.StateChange) > [2023-10-24 09:03:21,255] INFO Removing a node: /default-rack/ > 192.168.1.159:9866 (org.apache.hadoop.net.NetworkTopology) > [2023-10-24 09:03:21,255] INFO Adding a new node: /default-rack/ > 192.168.1.159:9866 (org.apache.hadoop.net.NetworkTopology) > [2023-10-24 09:03:21,281] INFO BLOCK* processReport 0x746ca82e1993dcbb > with lease ID 0xa39c5071fd7ca21f: Processing first storage report for > DS-ab8f27ed-6129-492c-9b8a-3800c46703fb from datanode DatanodeRegistration( > 192.168.1.159:9866, datanodeUuid=fbefce35-15f7-43df-a666-ecc90f4bef0f, > infoPort=9864, infoSecurePort=0, ipcPort=9867, > storageInfo=lv=-57;cid=CID-0b66d2f6-6c6a-4f3f-bdb1-b1ab0c947d00;nsid=2036303633;c=1697774550786) > (BlockStateChange) > [2023-10-24 09:03:21,281] INFO BLOCK* processReport 0x746ca82e1993dcbb > with lease ID 0xa39c5071fd7ca21f: from storage > DS-ab8f27ed-6129-492c-9b8a-3800c46703fb node DatanodeRegistration( > 192.168.1.159:9866, datanodeUuid=fbefce35-15f7-43df-a666-ecc90f4bef0f, > infoPort=9864, infoSecurePort=0, ipcPort=9867, > storageInfo=lv=-57;cid=CID-0b66d2f6-6c6a-4f3f-bdb1-b1ab0c947d00;nsid=2036303633;c=1697774550786), > blocks: 0, hasStaleStorage: false, processing time: 0 msecs, > invalidatedBlocks: 0 (BlockStateChange) > > > Is there anything else that I should look at? > I am not sure how to debug why it is not starting up on this port > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > For additional commands, e-mail: user-h...@hadoop.apache.org > >
