Thanks for your report. I'll look at this this week.
On Wed, Jul 6, 2011 at 1:21 AM, Claudio Martella <[email protected]> wrote: > Ok, > > apparently there are two options to set the ZK client port, i set it > correctly through zookeeper.client.port. > > Now I'm getting this error at groom start: > > localhost: Exception in thread "regionservernull" > java.lang.NoSuchMethodError: > org.apache.hadoop.filecache.DistributedCache.purgeCache(Lorg/apache/hadoop/conf/Configuration;)V > localhost: at > org.apache.hama.bsp.GroomServer.initialize(GroomServer.java:202) > localhost: at org.apache.hama.bsp.GroomServer.run(GroomServer.java:541) > localhost: at java.lang.Thread.run(Thread.java:662) > > > I'm afraid there are mismatches between hadoop 0.20.2 as hama is built > on (and shipped with) and the hadoop from CDH I'm using right now. The > method has been probably removed. > > Any suggestion? > > I can easily comment the call out, apparently it's bad practice for the > user to call the purgeCache directly. > > > On 7/5/11 5:14 PM, Claudio Martella wrote: >> Hello, >> >> I'm trying to install hama-0.2.0 in pseudo-distributed mode. As I have >> already HBase, which I'm going to read from, installed on my system, I'm >> basically trying to get hama to work with CDH3. So basically: >> >> - hbase 0.90.1-cdh3u0 >> - hadoop 0.20.2-cdh3u0 >> - zookeeper 3.3.3-cdh3u0 >> >> >> I configured hadoop in pseudo-distributed mode and launched the namenode >> and the datanode. I've configured zookpeeper with default configs. I'm >> also running hbase and from the client shell i can connect to ZK. >> >> This is hbase's shell zkdump: >> >> base(main):004:0> zk_dump >> HBase is rooted at /hbase >> Master address: localhost:60000 >> Region server holding ROOT: localhost:60020 >> Region servers: >> localhost:60020 >> Quorum Server Statistics: >> localhost:2181 >> Zookeeper version: 3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT >> Clients: >> /127.0.0.1:35176[1](queued=0,recved=190,sent=199) >> /0:0:0:0:0:0:0:1:57645[1](queued=0,recved=163,sent=164) >> /127.0.0.1:35163[1](queued=0,recved=201,sent=213) >> /127.0.0.1:53414[0](queued=0,recved=1,sent=0) >> /0:0:0:0:0:0:0:1:38233[1](queued=0,recved=14,sent=14) >> /0:0:0:0:0:0:0:1:57647[1](queued=0,recved=161,sent=162) >> >> Latency min/avg/max: 0/0/47 >> Received: 731 >> Sent: 753 >> Outstanding: 0 >> Zxid: 0x28 >> Mode: standalone >> Node count: 12 >> >> >> So apparently ZK is up and running. I've started hama with these >> hama-site.xml: >> >> <configuration> >> <property> >> <name>hama.zookeeper.property.clientPort</name> >> <value>2181</value> >> </property> >> <property> >> <name>fs.default.name</name> >> <value>hdfs://localhost:8020</value> >> </property> >> </configuration> >> >> But i get these errors at startup of groom: >> >> STARTUP_MSG: build = -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14; >> compiled by 'hudson' on Fri Mar 25 19:56:23 PDT 2011 >> ************************************************************/ >> 2011-07-05 17:09:49,566 INFO org.apache.hama.bsp.GroomServer: groom start >> 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: >> [Ljava.lang.String;@20b9b538 >> 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: >> /tmp/hadoop-cmartella/bsp/local >> 2011-07-05 17:09:49,742 INFO org.apache.hadoop.ipc.Server: Starting >> Socket Reader #1 for port 61000 >> 2011-07-05 17:09:49,744 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: >> Initializing RPC Metrics with hostName=BSPPeer, port=61000 >> 2011-07-05 17:09:49,775 INFO >> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC >> Metrics with hostName=BSPPeer, port=61000 >> 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server >> Responder: starting >> 2011-07-05 17:09:49,778 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 0 on 61000: starting >> 2011-07-05 17:09:49,778 INFO org.apache.hama.bsp.BSPPeer: BSPPeer >> address:rose.inf.unibz.it port:61000 >> 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server >> listener on 61000: starting >> 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:zookeeper.version=3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT >> 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:host.name=rose.inf.unibz.it >> 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.version=1.6.0_22 >> 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.vendor=Sun Microsystems Inc. >> 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.22/jre >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.class.path=/data/cmartella/graffiti/hama-0.2.0/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-examples.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-test.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/GroboTestingJUnit-1.2.1-core.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-cli-1.2.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-api-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/hadoop-core-0.20.2-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/junit-3.8.1.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/log4j-1.2.15.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/zookeeper-3.3.3-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/jsp-2.1/*.jar >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.io.tmpdir=/tmp >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:java.compiler=<NA> >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:os.name=Linux >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:os.arch=amd64 >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:os.version=2.6.32-27-server >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:user.name=cmartella >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:user.home=/home/students/cmartella >> 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client >> environment:user.dir=/data/cmartella/graffiti/hama-0.2.0 >> 2011-07-05 17:09:49,809 INFO org.apache.zookeeper.ZooKeeper: Initiating >> client connection, connectString=localhost:21810 sessionTimeout=3000 >> watcher=org.apache.hama.bsp.BSPPeer@217f242c >> 2011-07-05 17:09:49,820 INFO org.apache.zookeeper.ClientCnxn: Opening >> socket connection to server localhost/127.0.0.1:21810 >> 2011-07-05 17:09:49,824 WARN org.apache.zookeeper.ClientCnxn: Session >> 0x0 for server null, unexpected error, closing socket connection and >> attempting reconnect >> java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) >> 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: >> 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening >> socket connection to server localhost/0:0:0:0:0:0:0:1:21810 >> 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session >> 0x0 for server null, unexpected error, closing socket connection and >> attempting reconnect >> java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) >> 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: >> 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening >> socket connection to server localhost/0:0:0:0:0:0:0:1:21810 >> 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session >> 0x0 for server null, unexpected error, closing socket connection and >> attempting reconnect >> java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) >> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) >> 2011-07-05 17:09:50,338 ERROR org.apache.hama.bsp.BSPPeer: >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /bsp >> 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.Server: Starting >> Socket Reader #1 for port 50000 >> 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: >> Initializing RPC Metrics with hostName=GroomServer, port=50000 >> 2011-07-05 17:09:50,341 INFO >> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC >> Metrics with hostName=GroomServer, port=50000 >> 2011-07-05 17:09:50,342 INFO org.apache.hadoop.ipc.Server: IPC Server >> Responder: starting >> 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server >> listener on 50000: starting >> 2011-07-05 17:09:50,343 INFO org.apache.hama.bsp.GroomServer: Worker rpc >> server --> rose.inf.unibz.it:50000 >> 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 0 on 50000: starting >> 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.Server: Starting >> Socket Reader #1 for port 47963 >> 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: >> Initializing RPC Metrics with hostName=GroomServer, port=47963 >> 2011-07-05 17:09:50,345 INFO >> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC >> Metrics with hostName=GroomServer, port=47963 >> 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server >> Responder: starting >> 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server >> listener on 47963: starting >> 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 0 on 47963: starting >> 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 1 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 2 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 3 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 4 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 5 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 6 on 47963: starting >> 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 7 on 47963: starting >> 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 8 on 47963: starting >> 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server >> handler 9 on 47963: starting >> 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: >> GroomServer up at: localhost/127.0.0.1:47963 >> 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: Starting >> groom: groomd_rose.inf.unibz.it_61000 >> 2011-07-05 17:09:50,351 INFO org.apache.hama.GroomServerRunner: >> SHUTDOWN_MSG: >> >> As there were compatibility problems, I substituted the hadoop-core and >> zookeper jars in lib/ with those shipped by CDH. >> Apparenty groom is trying to connect to ZK on (hama's default) 21810 >> although I've set the port to ZK's default 2181. >> >> Any idea why this is happening? >> >> Best, >> Claudio >> > > > -- > Claudio Martella > Free Software & Open Technologies > Analyst > > TIS innovation park > Via Siemens 19 | Siemensstr. 19 > 39100 Bolzano | 39100 Bozen > Tel. +39 0471 068 123 > Fax +39 0471 068 129 > [email protected] http://www.tis.bz.it > > Short information regarding use of personal data. According to Section 13 of > Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we > process your personal data in order to fulfil contractual and fiscal > obligations and also to send you information regarding our services and > events. Your personal data are processed with and without electronic means > and by respecting data subjects' rights, fundamental freedoms and dignity, > particularly with regard to confidentiality, personal identity and the right > to personal data protection. At any time and without formalities you can > write an e-mail to [email protected] in order to object the processing of > your personal data for the purpose of sending advertising materials and also > to exercise the right to access personal data and other rights referred to in > Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation > Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete > information on the web site www.tis.bz.it. > > > > > -- Best Regards, Edward J. Yoon @eddieyoon
