Hi All, I'm trying to run hadoop(0.20.2) examples in Pseudo-Distributed Mode following the hadoop user guide. After I run the 'start-all.sh', it seems the namenode can't connect to datanode.
'SSH localhost' is OK on my server. Someone advises to rm '/tmp/hadoop-XXXX' and format namenode again, but it doesn't work. And 'iptables -L' shows there is no firewall rules in my server: test:/home/liyun2010# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Is there anyone can give me more advice? Thanks! Bellow is my namenode and datanode log files: liyun2010@test:~/hadoop-0.20.2/logs$ cat hadoop-liyun2010-namenode-test.puppet.com.log 2011-05-19 10:58:25,938 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = test.puppet.com/127.0.0.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 2011-05-19 10:58:26,197 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=9000 2011-05-19 10:58:26,212 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: test.puppet.com/127.0.0.1:9000 2011-05-19 10:58:26,220 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null 2011-05-19 10:58:26,224 INFO org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext 2011-05-19 10:58:26,405 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=liyun2010,users 2011-05-19 10:58:26,406 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup 2011-05-19 10:58:26,406 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true 2011-05-19 10:58:26,429 INFO org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics: Initializing FSNamesystemMetrics using context object:org.apache.hadoop.metrics.spi.NullContext 2011-05-19 10:58:26,434 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStatusMBean 2011-05-19 10:58:26,511 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files = 9 2011-05-19 10:58:26,524 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files under construction = 1 2011-05-19 10:58:26,530 INFO org.apache.hadoop.hdfs.server.common.Storage: Image file of size 920 loaded in 0 seconds. 2011-05-19 10:58:26,606 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Invalid opcode, reached end of edit log Number of transactions found 99 2011-05-19 10:58:26,606 INFO org.apache.hadoop.hdfs.server.common.Storage: Edits file /tmp/hadoop-liyun2010/dfs/name/current/edits of size 1049092 edits # 99 loaded in 0 seconds. 2011-05-19 10:58:26,660 INFO org.apache.hadoop.hdfs.server.common.Storage: Image file of size 920 saved in 0 seconds. 2011-05-19 10:58:26,810 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading FSImage in 505 msecs 2011-05-19 10:58:26,825 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of blocks = 0 2011-05-19 10:58:26,825 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid blocks = 0 2011-05-19 10:58:26,825 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of under-replicated blocks = 0 2011-05-19 10:58:26,825 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of over-replicated blocks = 0 2011-05-19 10:58:26,825 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 0 secs. 2011-05-19 10:58:26,826 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes 2011-05-19 10:58:26,826 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks 2011-05-19 10:58:27,025 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2011-05-19 10:58:27,174 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070 2011-05-19 10:58:27,178 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50070 webServer.getConnectors()[0].getLocalPort() returned 50070 2011-05-19 10:58:27,178 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50070 2011-05-19 10:58:27,179 INFO org.mortbay.log: jetty-6.1.14 2011-05-19 10:58:27,269 WARN org.mortbay.log: Can't reuse /tmp/Jetty_0_0_0_0_50070_hdfs____w2cu08, using /tmp/Jetty_0_0_0_0_50070_hdfs____w2cu08_740365192444258489 2011-05-19 10:58:28,610 INFO org.mortbay.log: Started [email protected]:50070 2011-05-19 10:58:28,611 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070 2011-05-19 10:58:28,612 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2011-05-19 10:58:28,613 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9000: starting 2011-05-19 10:58:28,617 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9000: starting 2011-05-19 10:58:28,618 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9000: starting 2011-05-19 10:58:28,621 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9000: starting 2011-05-19 10:58:28,625 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9000: starting 2011-05-19 10:58:28,625 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9000: starting 2011-05-19 10:58:28,626 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9000: starting 2011-05-19 10:58:28,627 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 9000: starting 2011-05-19 10:58:28,629 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 9000: starting 2011-05-19 10:58:28,630 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9000: starting 2011-05-19 10:58:28,630 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9000: starting 2011-05-19 10:58:30,680 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 127.0.0.1:50010 storage DS-173493047-127.0.0.1-50010-1305278767521 2011-05-19 10:58:30,687 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/127.0.0.1:50010 2011-05-19 10:58:39,361 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-liyun2010/mapred/system dst=nullperm=null 2011-05-19 10:58:39,393 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-liyun2010/mapred/system dst=nullperm=null 2011-05-19 10:58:39,405 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-liyun2010/mapred/system dst=nullperm=liyun2010:supergroup:rwxr-xr-x 2011-05-19 10:58:39,417 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-liyun2010/mapred/system dst=nullperm=liyun2010:supergroup:rwx-wx-wx 2011-05-19 10:58:39,507 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=create src=/tmp/hadoop-liyun2010/mapred/system/jobtracker.infodst=null perm=liyun2010:supergroup:rw-r--r-- 2011-05-19 10:58:39,530 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-liyun2010/mapred/system/jobtracker.info dst=null perm=liyun2010:supergroup:rw------- 2011-05-19 10:58:39,538 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Not able to place enough replicas, still in need of 1 2011-05-19 10:58:39,541 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9000, call addBlock(/tmp/hadoop-liyun2010/mapred/system/jobtracker.info, DFSClient_1143649887) from 127.0.0.1:56940: error: java.io.IOException: File /tmp/hadoop-liyun2010/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 java.io.IOException: File /tmp/hadoop-liyun2010/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1271) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) 2011-05-19 10:58:39,554 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=liyun2010,users ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-liyun2010/mapred/system/jobtracker.infodst=null perm=null liyun2010@test:~/hadoop-0.20.2/logs$ cat hadoop-liyun2010-datanode-test.puppet.com.log 2011-05-19 10:58:27,372 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = test.puppet.com/127.0.0.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 2011-05-19 10:58:28,932 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Registered FSDatasetStatusMBean 2011-05-19 10:58:28,938 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 50010 2011-05-19 10:58:28,942 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s 2011-05-19 10:58:29,137 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2011-05-19 10:58:29,341 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50075 2011-05-19 10:58:29,342 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50075 webServer.getConnectors()[0].getLocalPort() returned 50075 2011-05-19 10:58:29,342 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075 2011-05-19 10:58:29,342 INFO org.mortbay.log: jetty-6.1.14 2011-05-19 10:58:30,600 INFO org.mortbay.log: Started [email protected]:50075 2011-05-19 10:58:30,620 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=DataNode, sessionId=null 2011-05-19 10:58:30,659 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=DataNode, port=50020 2011-05-19 10:58:30,670 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2011-05-19 10:58:30,672 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50020: starting 2011-05-19 10:58:30,672 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(test.puppet.com:50010, storageID=DS-173493047-127.0.0.1-50010-1305278767521, infoPort=50075, ipcPort=50020) 2011-05-19 10:58:30,673 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 50020: starting 2011-05-19 10:58:30,689 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 50020: starting 2011-05-19 10:58:30,689 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting 2011-05-19 10:58:30,690 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1:50010, storageID=DS-173493047-127.0.0.1-50010-1305278767521, infoPort=50075, ipcPort=50020)In DataNode.run, data = FSDataset{dirpath='/tmp/hadoop-liyun2010/dfs/data/current'} 2011-05-19 10:58:30,691 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: using BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec 2011-05-19 10:58:30,774 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0 blocks got processed in 23 msecs 2011-05-19 10:58:30,776 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic block scanner 2011-05-19 李�S
