Dave, I have switched it to 3.2.2. Still the same despite ZK says:"Client connection successful"
On Wed, Dec 8, 2010 at 11:53 PM, Buttler, David <[email protected]> wrote: > Why are you using zookeeper 3.3.0 with hbase 0.20.6? hbase uses zk 3.2.2 > internally -- mixing and matching zk versions is not a great plan > > Dave > > -----Original Message----- > From: Gökhan Çapan [mailto:[email protected]] > Sent: Wednesday, December 08, 2010 1:19 PM > To: [email protected] > Subject: Re: Importing to HBase from Java problem > > Sure. > > This is a prototype project for me to test some algorithms before deploying > them to cloud for production. > > I wanted to persist results of a Mapreduced MinHash algorithm to a NoSQL > storage. > After that, I wanted to execute some scans, gets,... as a simple test for > me. > > It is a maven project which has dependencies to hadoop 0.20.2, hbase 0.20.6 > (i have added the downloaded jar to our internal archiva), zookeeper 3.3.0. > Plus, an hbase-site.xml: > <configuration> > <property> > <name>hbase.zookeeper.quorum</name> > <value>localhost</value> > <description> > </description> > </property> > </configuration> > > is in its classpath. > > > > One module of the project is a series of MR jobs that finally persists > results to an HBase table. When I tried to run it, I got some errors. > Following your instructions, they've been solved. > > > There is another module with exactly same dependencies. This is a "has > nothing but a main" class that performs some scans on the HBase table > produced in the previous step. > > after a mvn package with dependencies and adding the hbase-site.xml above > as > a resource, with SimpleClient as main class, I couldn't java -jar > SimpleClient.jar, it gave me the exception I have posted. > > Thanks in advance. > > > On Wed, Dec 8, 2010 at 8:11 PM, Stack <[email protected]> wrote: > > > Tell us more about your client project. What is it? Is it a > > container of some type? A tomcat or something? You could try logging > > the CLASSPATH your client sees. The ZooKeeper that we have in TRUNK > > and 0.90 branch does it. It doesn't look like the one in 0.20.6 does > > going by the log below. > > St.Ack > > > > On Wed, Dec 8, 2010 at 6:44 AM, Gökhan Çapan <[email protected]> wrote: > > > Hi, > > > > > > Now I am getting the same Exception while trying to run a simple HBase > > > client. > > > > > > HBase 0.20.6 is both in classpath of Hadoop and the Client project. > > > HBase and hadoop configuration is in the classpath of the Client > Project > > > HBase configuration is in the Hadoop classpath. > > > HBase shell runs well. > > > Hadoop runs well. > > > My previous MapReduce app with same dependencies works well. > > > > > > Below is the log and Exception: > > > > > > 43:44 INFO zookeeper.ZooKeeper: Client > > > > > > environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > > > > environment:java.io.tmpdir=/var/folders/9x/9x-OkSEyFDWn+13ntAMH6++++TI/-Tmp-/ > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > environment:java.compiler=<NA> > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client environment:os.name > > =Mac > > > OS X > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > environment:os.arch=x86_64 > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > environment:os.version=10.6.5 > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client environment: > user.name > > > =gcapan > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > environment:user.home=/Users/gcapan > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Client > > > environment:user.dir=/Users/gcapan/workspace/*** > > > 10/12/08 13:43:44 INFO zookeeper.ZooKeeper: Initiating client > connection, > > > connectString=localhost:2181 sessionTimeout=60000 > > > > > > watcher=org.apache.hadoop.hbase.client.hconnectionmanager$tableserv...@2b2d96f2 > > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Opening socket connection > to > > > server localhost/127.0.0.1:2181 > > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Socket connection > > established > > > to localhost/127.0.0.1:2181, initiating session > > > 10/12/08 13:43:44 INFO zookeeper.ClientCnxn: Session establishment > > complete > > > on server localhost/127.0.0.1:2181, sessionid = 0x12cc531bd4c0006, > > > negotiated timeout = 60000 > > > Exception in thread "main" > java.lang.reflect.UndeclaredThrowableException > > > at $Proxy0.getRegionInfo(Unknown Source) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:874) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:524) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:528) > > > at > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:123) > > > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101) > > > at com.***.***.***.***HBaseClientDemo.main(HBaseClientDemo.java:26) > > > Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: > > > java.io.IOException: Could not find requested method, the usual cause > is > > a > > > version mismatch between client and server. > > > at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648) > > > at > > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > > > > > at > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723) > > > at > > > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328) > > > ... 13 more > > > > > > > > > > > > On Tue, Dec 7, 2010 at 11:30 AM, Gökhan Çapan <[email protected]> > wrote: > > > > > >> I had forgotten add HBase-conf to MapReduce ClassPath. > > >> > > >> That solved the problem, > > >> > > >> Thanks. > > >> > > >> > > >> > > >> On Mon, Dec 6, 2010 at 8:40 PM, Gökhan Çapan <[email protected]> > wrote: > > >> > > >>> > > >>> > > >>> On Mon, Dec 6, 2010 at 7:54 PM, Stack <[email protected]> wrote: > > >>> > > >>>> That looks like a mismatch between client and server hbase versions. > > >>>> Ensure you have same running all over your cluster. Make sure you > > >>>> don't have a mix of 0.20.x and 0.89... or 0.90 release candidates. > > >>>> > > >>>> You seem to be feeling your way. Have you seen > > >>>> > > >>>> > > > https://hudson.apache.org/hudson/view/G-L/view/HBase/job/hbase-0.90/ws/trunk/target/site/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#package_description > > >>>> ? > > >>>> > > >>>> You added zookeeper. Be sure to add the hbase conf directory to > your > > >>>> MapReduce CLASSPATH. > > >>>> > > >>>> St.Ack > > >>>> > > >>> > > >>> I hadn't added the hbase conf directory to my MapReduce CLASSPATH. > > >>> Unfortunately, now I am not able to try. I will try tomorrow, and let > > you > > >>> know. > > >>> > > >>> Thanks. > > >>> > > >>>> > > >>>> > > >>>> On Mon, Dec 6, 2010 at 8:12 AM, Gökhan Çapan <[email protected]> > > wrote: > > >>>> > Hi, > > >>>> > > > >>>> > I am facing with some issues when I try to write the results of my > > >>>> Reducer > > >>>> > to an HBase table, in pseudo-distributed mode. > > >>>> > > > >>>> > HBase version: 0.20.6 > > >>>> > Hadoop version: 0.20.2 > > >>>> > > > >>>> > I let HBase manage Zookeeper. > > >>>> > > > >>>> > After starting HBase, I can successfully start HBase shell, and > > create > > >>>> a > > >>>> > table. > > >>>> > > > >>>> > In my MapReduce app, I have a dependency to HBase 0.20.2. > > >>>> > > >>> > > >>> Sorry, that was also HBase 0.20.6 > > >>> > > >>> > > >>>> > > > >>>> > However, it gave me the following error: > > >>>> > Error: java.lang.ClassNotFoundException: > > org.apache.zookeeper.Watcher > > >>>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > >>>> > at java.security.AccessController.doPrivileged(Native Method) > > >>>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > >>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > > >>>> > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > >>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > > >>>> > at java.lang.ClassLoader.defineClass1(Native Method) > > >>>> > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) > > >>>> > at java.lang.ClassLoader.defineClass(ClassLoader.java:616) > > >>>> > at > > >>>> > > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) > > >>>> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) > > >>>> > at java.net.URLClassLoader.access$000(URLClassLoader.java:58) > > >>>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:197) > > >>>> > at java.security.AccessController.doPrivileged(Native Method) > > >>>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > >>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > > >>>> > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > > >>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:93) > > >>>> > at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:119) > > >>>> > at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.mapreduce.TableOutputFormat.getRecordWriter(TableOutputFormat.java:116) > > >>>> > at > > >>>> > > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) > > >>>> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) > > >>>> > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > >>>> > . > > >>>> > . > > >>>> > . > > >>>> > > > >>>> > > > >>>> > > > >>>> > So I have added ZooKeeper as a dependency. Now it throws the > > following > > >>>> > Exception for ZooKeeper 3.2.2, 3.3.0, and 3.3.1: > > >>>> > > > >>>> > 10/12/06 17:51:53 INFO mapred.JobClient: map 100% reduce 27% > > >>>> > 10/12/06 17:51:58 INFO mapred.JobClient: Task Id : > > >>>> > attempt_201012061721_0002_r_000000_2, Status : FAILED > > >>>> > java.lang.reflect.UndeclaredThrowableException > > >>>> > at $Proxy2.getRegionInfo(Unknown Source) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:874) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:515) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:524) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:565) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:528) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:491) > > >>>> > at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:123) > > >>>> > at > org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:101) > > >>>> > at > > >>>> > > > >>>> > > > org.apache.hadoop.hbase.mapreduce.TableOutputFormat.getRecordWriter(TableOutputFormat.java:116) > > >>>> > at > > >>>> > > > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) > > >>>> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) > > >>>> > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > >>>> > Caused by: org.apache.hadoop.ipc.RemoteException: > > java.io.IOException: > > >>>> > java.io.IOException: Could not find requested method, the usual > > cause > > >>>> is a > > >>>> > version mismatch between client and server. > > >>>> > at > > >>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648) > > >>>> > at > > >>>> > > > >>>> > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > >>>> > > > >>>> > at > > >>>> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:723) > > >>>> > at > > >>>> > > > org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328) > > >>>> > ... 16 more > > >>>> > > > >>>> > > > >>>> > I guess this is a compatibility issue between HBase and ZooKeeper, > > but > > >>>> I > > >>>> > couldn't find the right version of ZooKeeper to add as a > dependency. > > >>>> > > > >>>> > > > >>>> > Thanks in advance, > > >>>> > > > >
