Hi Chamara, Pl try to start the name node without all the other services and attach the name node log. It will help to isolate the issue in Name Node.
You can start the name node by executing "bin/hadoop namenode" Thanks, Deependra. On Wed, Aug 8, 2012 at 9:56 AM, Chamara Ariyarathne <[email protected]>wrote: > I get the following [1] when starting hadoop namenode in the distributed > hadoop cluster and it prevents from starting all the hadoop nodes. > > /etc/hosts is: > 10.100.3.221 bamhadoop01 > 10.100.3.228 bamhadoop02 > 10.100.3.224 bamhadoop03 > 10.100.3.244 appserver.cloud-test.wso2.com > > # The following lines are desirable for IPv6 capable hosts > ::1 ip6-localhost ip6-loopback > fe00::0 ip6-localnet > ff00::0 ip6-mcastprefix > ff02::1 ip6-allnodes > ff02::2 ip6-allrouters > > $ hostname > bamhadoop01 > > Property in core-site.xml > <property> > <name>fs.default.name</name> > <value>hdfs://bamhadoop01:9000</value> > </property> > <property> > > Can ssh to any other hadoop datanode machines without password. > > [1] > 2012-08-08 09:45:05,289 INFO > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2012-08-08 09:45:05,297 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > MetricsSystem,sub=Stats registered. > 2012-08-08 09:45:05,297 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot > period at 10 second(s). > 2012-08-08 09:45:05,297 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics > system started > 2012-08-08 09:45:05,478 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi > registered. > 2012-08-08 09:45:05,481 WARN > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already > exists! > 2012-08-08 09:45:05,482 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-08-08 09:45:05,483 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Starting expired delegation token remover thread, > tokenRemoverScanInterval=60 min(s) > 2012-08-08 09:45:05,483 INFO > org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: > Updating the current master key for generating delegation tokens > 2012-08-08 09:45:05,483 INFO org.apache.hadoop.mapred.JobTracker: > Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, > limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) > 2012-08-08 09:45:05,484 INFO org.apache.hadoop.util.HostsFileReader: > Refreshing hosts (include/exclude) list > 2012-08-08 09:45:05,517 INFO org.apache.hadoop.mapred.JobTracker: Starting > jobtracker with owner as bamtesttmp > 2012-08-08 09:45:05,531 INFO org.apache.hadoop.ipc.Server: Starting > SocketReader > 2012-08-08 09:45:05,533 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > RpcDetailedActivityForPort9001 registered. > 2012-08-08 09:45:05,534 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > RpcActivityForPort9001 registered. > 2012-08-08 09:45:10,598 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-08-08 09:45:10,634 INFO org.apache.hadoop.http.HttpServer: Added > global filtersafety > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > 2012-08-08 09:45:10,655 INFO org.apache.hadoop.http.HttpServer: Port > returned by webServer.getConnectors()[0].getLocalPort() before open() is > -1. Opening the listener on 50030 > 2012-08-08 09:45:10,656 INFO org.apache.hadoop.http.HttpServer: > listener.getLocalPort() returned 50030 > webServer.getConnectors()[0].getLocalPort() returned 50030 > 2012-08-08 09:45:10,656 INFO org.apache.hadoop.http.HttpServer: Jetty > bound to port 50030 > 2012-08-08 09:45:10,656 INFO org.mortbay.log: jetty-6.1.26 > 2012-08-08 09:45:10,676 WARN org.mortbay.log: Can't reuse > /tmp/Jetty_0_0_0_0_50030_job____yn7qmk, using > /tmp/Jetty_0_0_0_0_50030_job____yn7qmk_2518359383055609936 > 2012-08-08 09:45:10,811 INFO org.mortbay.log: Started > [email protected]:50030 > 2012-08-08 09:45:10,815 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm > registered. > 2012-08-08 09:45:10,815 INFO > org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source > JobTrackerMetrics registered. > 2012-08-08 09:45:10,815 INFO org.apache.hadoop.mapred.JobTracker: > JobTracker up at: 9001 > 2012-08-08 09:45:10,815 INFO org.apache.hadoop.mapred.JobTracker: > JobTracker webserver: 50030 > 2012-08-08 09:45:11,867 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 0 time(s). > 2012-08-08 09:45:12,868 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 1 time(s). > 2012-08-08 09:45:13,868 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 2 time(s). > 2012-08-08 09:45:14,869 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 3 time(s). > 2012-08-08 09:45:15,869 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 4 time(s). > 2012-08-08 09:45:16,870 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 5 time(s). > 2012-08-08 09:45:17,870 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 6 time(s). > 2012-08-08 09:45:18,871 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 7 time(s). > 2012-08-08 09:45:19,871 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 8 time(s). > 2012-08-08 09:45:20,872 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 9 time(s). > 2012-08-08 09:45:20,874 INFO org.apache.hadoop.mapred.JobTracker: problem > cleaning system directory: null > java.net.ConnectException: Call to bamhadoop01/10.100.3.221:9000 failed > on connection exception: java.net.ConnectException: Connection refused > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057) > at org.apache.hadoop.ipc.Client.call(Client.java:1033) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224) > at $Proxy5.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364) > at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208) > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:175) > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89) > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1310) > at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:65) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1328) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:109) > at org.apache.hadoop.mapred.JobTracker$3.run(JobTracker.java:2349) > at org.apache.hadoop.mapred.JobTracker$3.run(JobTracker.java:2347) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2347) > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2171) > at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300) > at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:291) > at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4956) > Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) > at > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406) > at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414) > at > org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527) > at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187) > at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164) > at org.apache.hadoop.ipc.Client.call(Client.java:1010) > ... 22 more > 2012-08-08 09:45:31,877 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 0 time(s). > 2012-08-08 09:45:32,878 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 1 time(s). > 2012-08-08 09:45:33,878 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 2 time(s). > 2012-08-08 09:45:34,879 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 3 time(s). > 2012-08-08 09:45:35,879 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: bamhadoop01/10.100.3.221:9000. Already tried 4 time(s). > > > > -- > *Chamara Ariyarathne* > Senior Software Engineer - QA; > WSO2 Inc; http://www.wso2.com/. > Mobile; *+94772786766* > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Deependra Ariyadewa WSO2, Inc. http://wso2.com/ http://wso2.org email [email protected]; cell +94 71 403 5996 ; Blog http://risenfall.wordpress.com/ PGP info: KeyID: 'DC627E6F'
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
