Hi
I have copy the conf files on the three virtual machine,
I am confused about it
Cloud you help me
-----邮件原件-----
发件人: common-user-return-32877-guanhua.tian=ia.ac...@hadoop.apache.org
[mailto:common-user-return-32877-guanhua.tian=ia.ac...@hadoop.apache.org] 代表 tgh
发送时间: 2012年2月23日 18:10
收件人: common-user@hadoop.apache.org
主题: 答复: TaskTracker Error
Hi
I use ubuntu , the firewall seems off , for three virtual machine, and
how to solve ERROR , cloud you help me ?
root@ubuntu:/home/hadoop-0.20.2# service iptables status
iptables: unrecognized service
root@ubuntu:/home/hadoop-0.20.2# ufw disable
Firewall stopped and disabled on system startup
root@ubuntu:/home/hadoop-0.20.2# ufw status
Status: inactive
root@ubuntu:/home/hadoop-0.20.2#
this is port by Java on master 192.168.164.128
root@ubuntu:~#
root@ubuntu:~# netstat -nap|grep java
tcp6 0 0 :::41095 :::* LISTEN
4000/java
tcp6 0 0 :::50090 :::* LISTEN
4222/java
tcp6 0 0 :::50060 :::* LISTEN
4492/java
tcp6 0 0 :::42316 :::* LISTEN
4297/java
tcp6 0 0 192.168.164.136:9100 :::* LISTEN
3800/java
tcp6 0 0 192.168.164.136:9101 :::* LISTEN
4297/java
tcp6 0 0 :::50030 :::* LISTEN
4297/java
tcp6 0 0 :::33297 :::* LISTEN
3800/java
tcp6 0 0 127.0.0.1:60722 :::* LISTEN
4492/java
tcp6 0 0 :::50070 :::* LISTEN
3800/java
tcp6 0 0 :::50010 :::* LISTEN
4000/java
tcp6 0 0 :::50075 :::* LISTEN
4000/java
tcp6 0 0 :::35262 :::* LISTEN
4222/java
tcp6 0 0 :::50020 :::* LISTEN
4000/java
tcp6 0 0 192.168.164.136:58531 192.168.164.136:9101 ESTABLISHED
4492/java
tcp6 0 0 192.168.164.136:9100 192.168.164.136:37493 ESTABLISHED
3800/java
tcp6 0 0 192.168.164.136:37490 192.168.164.136:9100 ESTABLISHED
4297/java
tcp6 0 0 192.168.164.136:9100 192.168.164.137:53796 ESTABLISHED
3800/java
tcp6 0 0 192.168.164.136:9100 192.168.164.136:37490 ESTABLISHED
3800/java
tcp6 0 0 192.168.164.136:9100 192.168.164.138:40077 ESTABLISHED
3800/java
tcp6 0 0 192.168.164.136:37493 192.168.164.136:9100 ESTABLISHED
4000/java
unix 2 [ ] STREAM CONNECTED 21015 4492/java
unix 2 [ ] STREAM CONNECTED 20907 4297/java
unix 2 [ ] STREAM CONNECTED 20204 4222/java
unix 2 [ ] STREAM CONNECTED 19574 4000/java
unix 2 [ ] STREAM CONNECTED 19293 3800/java
root@ubuntu:~#
this is on slaves 192.168.164.137
root@ubuntu:/home/hadoop-0.20.2#
root@ubuntu:/home/hadoop-0.20.2# netstat -nap|grep java
tcp6 0 0 :::50060 :::* LISTEN
13130/java
tcp6 0 0 127.0.0.1:40112 :::* LISTEN
13130/java
tcp6 0 0 :::35703 :::* LISTEN
12949/java
tcp6 0 0 :::50010 :::* LISTEN
12949/java
tcp6 0 0 :::50075 :::* LISTEN
12949/java
tcp6 0 0 :::50020 :::* LISTEN
12949/java
tcp6 0 0 192.168.164.137:53796 192.168.164.136:9100 ESTABLISHED
12949/java
tcp6 0 0 192.168.164.137:43216 192.168.164.136:9101 ESTABLISHED
13130/java
unix 2 [ ] STREAM CONNECTED 51464 13130/java
unix 2 [ ] STREAM CONNECTED 49229 12949/java
root@ubuntu:/home/hadoop-0.20.2#
-----邮件原件-----
发件人: common-user-return-32874-guanhua.tian=ia.ac...@hadoop.apache.org
[mailto:common-user-return-32874-guanhua.tian=ia.ac...@hadoop.apache.org] 代表
Harsh J
发送时间: 2012年2月23日 17:31
收件人: common-user@hadoop.apache.org
主题: Re: TaskTracker Error
Have you ensured your firewall is off on all instances, or appropriately
configured if you need them?
$ service iptables stop
It is turned on by default on most distributions. I know CentOS6 turns it on by
default, with some rules.
On Thu, Feb 23, 2012 at 2:33 PM, tgh <guanhua.t...@ia.ac.cn> wrote:
> Hi
>
> I setup hadoop with hadoop 0.20.2
>
>
>
> I use three virtual machines on vmware,
>
> The three virtual machine could ssh with each other,
>
> ERROR rise , the tasktracker on slave 192.168.164.137 and
> 192.168.164.138 cloud not connect to master, while the tasktracker on
> 192.168.164.136 seems no error,
>
>
>
> Cloud you help me
>
>
>
> The conf file is set as follows,
>
> root@ubuntu:/home/hadoop-0.20.2/conf# cat masters
>
> 192.168.164.136
>
> root@ubuntu:/home/hadoop-0.20.2/conf# cat slaves
>
> 192.168.164.136
>
> 192.168.164.137
>
> 192.168.164.138
>
> root@ubuntu:/home/hadoop-0.20.2/conf#
>
> root@ubuntu:/home/hadoop-0.20.2/conf# cat core-site.xml
>
> <?xml version="1.0"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
>
>
> <!-- Put site-specific property overrides in this file. -->
>
>
>
> <configuration>
>
> <property>
>
> <name>fs.default.name</name>
>
> <value>hdfs://192.168.164.136:9100</value>
>
> </property>
>
> <property>
>
> <name>hadoop.tmp.dir</name>
>
> <value>/home/hadoop-0.20.2/tmp/</value>
>
> </property>
>
> <property>
>
> <name>dfs.replication</name>
>
> <value>1</value>
>
> </property>
>
> <!-- property>
>
> <name>mapred.child.java.opts</name>
>
> <value>-Xmx128m</value>
>
> </property>
>
> <property>
>
> <name>dfs.block.size</name>
>
> <value>5120000</value>
>
> <description>The default block size for new files.</description>
>
> </property -->
>
> </configuration>
>
>
>
> root@ubuntu:/home/hadoop-0.20.2/conf# cat mapred-site.xml
>
> <?xml version="1.0"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
>
>
> <!-- Put site-specific property overrides in this file. -->
>
>
>
> <configuration>
>
> <property>
>
> <name>mapred.job.tracker</name>
>
> <value>192.168.164.136:9101</value>
>
> </property>
>
> </configuration>
>
>
>
>
>
>
>
> now ERROR rise , the tasktracker on slave 192.168.164.137 and
> 192.168.164.138 cloud not connect to master, while the tasktracker on
> 192.168.164.136 seems no error,
>
>
>
> this is the log on 192.168.164.138,
>
> root@ubuntu:/home/hadoop-0.20.2/logs#
>
> root@ubuntu:/home/hadoop-0.20.2/logs# cat
> hadoop-root-tasktracker-ubuntu.log
>
>
> 2012-02-23 00:44:10,851 INFO org.apache.hadoop.mapred.TaskTracker:
> STARTUP_MSG:
>
> /************************************************************
>
> STARTUP_MSG: Starting TaskTracker
>
> STARTUP_MSG: host = ubuntu/127.0.1.1
>
> STARTUP_MSG: args = []
>
> STARTUP_MSG: version = 0.20.2
>
> STARTUP_MSG: build =
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
>
> ************************************************************/
>
> 2012-02-23 00:44:16,080 INFO org.mortbay.log: Logging to
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
> org.mortbay.log.Slf4jLog
>
> 2012-02-23 00:44:16,199 INFO org.apache.hadoop.http.HttpServer: Port
> returned by webServer.getConnectors()[0].getLocalPort() before open() is -1.
> Opening the listener on 50060
>
> 2012-02-23 00:44:16,205 INFO org.apache.hadoop.http.HttpServer:
> listener.getLocalPort() returned 50060
> webServer.getConnectors()[0].getLocalPort() returned 50060
>
> 2012-02-23 00:44:16,205 INFO org.apache.hadoop.http.HttpServer: Jetty
> bound to port 50060
>
> 2012-02-23 00:44:16,205 INFO org.mortbay.log: jetty-6.1.14
>
> 2012-02-23 00:45:08,741 INFO org.mortbay.log: Started
> SelectChannelConnector@0.0.0.0:50060
>
> 2012-02-23 00:45:08,808 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
> Initializing JVM Metrics with processName=TaskTracker, sessionId=
>
> 2012-02-23 00:45:08,848 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
> Initializing RPC Metrics with hostName=TaskTracker, port=49689
>
> 2012-02-23 00:45:08,909 INFO org.apache.hadoop.ipc.Server: IPC Server
> Responder: starting
>
> 2012-02-23 00:45:08,912 INFO org.apache.hadoop.mapred.TaskTracker:
> TaskTracker up at: localhost/127.0.0.1:49689
>
> 2012-02-23 00:45:08,912 INFO org.apache.hadoop.mapred.TaskTracker:
> Starting tracker tracker_ubuntu:localhost/127.0.0.1:49689
>
> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server
> listener on 49689: starting
>
> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 49689: starting
>
> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 1 on 49689: starting
>
> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 2 on 49689: starting
>
> 2012-02-23 00:45:08,919 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 3 on 49689: starting
>
> 2012-02-23 00:47:53,638 INFO org.apache.hadoop.mapred.TaskTracker:
> Using MemoryCalculatorPlugin :
> org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@cafb56
>
> 2012-02-23 00:47:53,641 INFO org.apache.hadoop.mapred.TaskTracker:
> Starting
> thread: Map-events fetcher for all reduce tasks on
> tracker_ubuntu:localhost/127.0.0.1:49689
>
> 2012-02-23 00:47:53,646 WARN org.apache.hadoop.mapred.TaskTracker:
> TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is
> disabled.
>
> 2012-02-23 00:47:53,647 INFO org.apache.hadoop.mapred.IndexCache:
> IndexCache created with max memory = 10485760
>
> 2012-02-23 00:47:55,110 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 0 time(s).
>
> 2012-02-23 00:47:56,112 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 1 time(s).
>
> 2012-02-23 00:47:57,114 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 2 time(s).
>
> 2012-02-23 00:47:58,116 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 3 time(s).
>
> 2012-02-23 00:47:59,118 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 4 time(s).
>
> 2012-02-23 00:48:00,120 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 5 time(s).
>
> 2012-02-23 00:48:01,122 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 6 time(s).
>
> 2012-02-23 00:48:02,124 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 7 time(s).
>
> 2012-02-23 00:48:03,126 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 8 time(s).
>
> 2012-02-23 00:48:04,130 INFO org.apache.hadoop.ipc.Client: Retrying
> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 9 time(s).
>
> 2012-02-23 00:48:04,132 ERROR org.apache.hadoop.mapred.TaskTracker:
> Caught
> exception: java.net.ConnectException: Call to
> ubuntu.local/192.168.164.138:9100 failed on connection exception:
> java.net.ConnectException: Connection refused
>
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:767)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:743)
>
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>
> at $Proxy5.getProtocolVersion(Unknown Source)
>
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>
> at
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFil
> eSyste
> m.java:82)
>
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>
> at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>
> at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
>
> at
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:103
> 3)
>
> at
> org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1720)
>
> at
> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2833)
>
> Caused by: java.net.ConnectException: Connection refused
>
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.
> java:2
> 06)
>
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
>
> at
> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304
> )
>
> at
> org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
>
> at org.apache.hadoop.ipc.Client.getConnection(Client.java:860)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:720)
>
> ... 15 more
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about
