Hey Pete, Ignore my other post, looks like you found the issue.
HBase ships with a hadoop jar because it needs to ship with all of its dependencies. If we didn't, you would have to download the Hadoop release separately just to run the unit tests or a single-node test setup. In the past, we have also had some client-side HBase-only patches applied to the hadoop jar with which we shipped. Of course, when using a full Hadoop distribution, you need to ensure compatibility between the server and your client library as would be the case with anything else. Glad to see things are working better! JG > -----Original Message----- > From: Pete Haidinyak [mailto:[email protected]] > Sent: Tuesday, December 21, 2010 11:07 PM > To: [email protected] > Subject: hadoop jar in hbase lib? > > Is there a reason there is a hadoop jar in the hbase lib directory? > Couldn't hbase us the jar in the Hadoop directory? > > THanks > > -Pete > > On Tue, 21 Dec 2010 21:37:28 -0800, Pete Haidinyak <[email protected]> > wrote: > > > Good call, looks like that might have been my problem. Seems simple now. > > ;-) > > > > Thanks > > > > -Pete > > > > > > On Tue, 21 Dec 2010 20:33:31 -0800, Claudio Martella > > <[email protected]> wrote: > > > >> Could you check if you're using the same hadoop jar version on hdfs > >> and hbase? I had a similar problem once. > >> > >> > >> On 12/22/10 12:32 AM, Pete Haidinyak wrote: > >>> After 11 hours I give up. I am trying to get a 5 node system up on > >>> my ESXi server. > >>> OS CentOS 5.5 x64 > >>> Hadoop Version 0.20.2+737 > >>> HBase Version 0.87.20100924+28 > >>> > >>> Name Node - Zoo Keeper - HBase Master on one machine > >>> 4 Region Server > >>> > >>> Hadoop starts up on every node without any problems. When I try to > >>> start HBase on the Master server I get zookeeper coming up but no > >>> HBase Master. When I do a hbase-stop.sh the zookeeper goes away and > >>> then the HMaster appears in a 'JPS' command. Here is the log file > >>> > >>> > >>> Tue Dec 21 18:17:35 EST 2010 Starting master on hadoop-name-node > >>> ulimit -n 1024 > >>> 2010-12-21 18:17:36,380 INFO > >>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: > Reconnecting to > >>> zookeeper > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 > >>> 17:14 GMT > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:host.name=hadoop-name-node > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.version=1.6.0_23 > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.vendor=Sun Microsystems Inc. > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.home=/usr/java/jdk1.6.0_23/jre > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.class.path=/usr/local/hbase/conf:/usr/java/default/ > >>> lib/tools.jar::/usr/local/hbase:/usr/local/hbase/hbase-0.89.2010 > >>> 0924+28.jar:/usr/local/hbase/hbase-0.89.20100924+28-tests.jar:/usr/l > >>> 0924+ocal/hbase/lib/activation-1.1.jar:/usr/local/hbase/lib/ant-1.6. > >>> 0924+5.jar:/usr/local/hbase/lib/ant-1.7.1.jar:/usr/local/hbase/lib/a > >>> 0924+nt-lau > >>> > >>> ncher-1.7.1.jar:/usr/local/hbase/lib/asm-3.1.jar:/usr/local/hbase/li > >>> b/avro-1.3.2.jar:/usr/local/hbase/lib/commons-cli-1.2.jar:/usr/local > >>> /hbase/lib/commons-codec-1.2.jar:/usr/local/hbase/lib/commons-el > >>> > >>> -1.0.jar:/usr/local/hbase/lib/commons-httpclient-3.1.jar:/usr/local/ > >>> hbase/lib/commons-lang-2.5.jar:/usr/local/hbase/lib/commons-logging- > >>> 1.1.1.jar:/usr/local/hbase/lib/commons-net-1.4.1.jar:/usr/local/ > >>> > >>> hbase/lib/core-3.1.1.jar:/usr/local/hbase/lib/guava-r05.jar:/usr/loc > >>> al/hbase/lib/hadoop-core-0.20.2-737.jar:/usr/local/hbase/lib/hsqldb- > >>> 1.8.0.10.jar:/usr/local/hbase/lib/jackson-core-asl-1.5.2.jar:/us > >>> > >>> r/local/hbase/lib/jackson-mapper-asl-1.5.2.jar:/usr/local/hbase/lib/ > >>> jasper-compiler-5.5.23.jar:/usr/local/hbase/lib/jasper-runtime-5.5.2 > >>> 3.jar:/usr/local/hbase/lib/jaxb-api-2.1.jar:/usr/local/hbase/lib > >>> > >>> /jaxb-impl-2.1.12.jar:/usr/local/hbase/lib/jersey-core-1.1.5.1.jar:/ > >>> usr/local/hbase/lib/jersey-json-1.1.5.1.jar:/usr/local/hbase/lib/jer > >>> sey-server-1.1.5.1.jar:/usr/local/hbase/lib/jets3t-0.7.1.jar:/us > >>> > >>> r/local/hbase/lib/jettison-1.1.jar:/usr/local/hbase/lib/jetty- > 6.1.24.jar:/usr/local/hbase/lib/jetty-util-6.1.24.jar:/usr/local/hbase/lib/jline- > 0.9.94.jar:/usr/local/hbase/lib/jruby-complete-1.4.0.jar: > >>> > >>> /usr/local/hbase/lib/jsp-2.1-6.1.14.jar:/usr/local/hbase/lib/jsp-api > >>> -2.0.jar:/usr/local/hbase/lib/jsp-api-2.1-6.1.14.jar:/usr/local/hbas > >>> e/lib/jsr311-api-1.1.1.jar:/usr/local/hbase/lib/kfs-0.3.jar:/usr > >>> > >>> /local/hbase/lib/log4j-1.2.15.jar:/usr/local/hbase/lib/oro- > 2.0.8.jar:/usr/local/hbase/lib/paranamer- > 2.2.jar:/usr/local/hbase/lib/paranamer-ant- > 2.2.jar:/usr/local/hbase/lib/paranamer-generator-2.2.jar: > >>> > >>> /usr/local/hbase/lib/protobuf-java-2.3.0.jar:/usr/local/hbase/lib/qd > >>> ox-1.10.1.jar:/usr/local/hbase/lib/servlet-api-2.4.jar:/usr/local/hb > >>> ase/lib/servlet-api-2.5-20081211.jar:/usr/local/hbase/lib/servle > >>> > >>> t-api-2.5-6.1.14.jar:/usr/local/hbase/lib/slf4j-api-1.5.8.jar:/usr/l > >>> ocal/hbase/lib/slf4j-log4j12-1.5.8.jar:/usr/local/hbase/lib/stax-api > >>> -1.0.1.jar:/usr/local/hbase/lib/stax-api-1.0-2.jar:/usr/local/hb > >>> > >>> ase/lib/thrift-0.2.0.jar:/usr/local/hbase/lib/xmlenc-0.52.jar:/usr/l > >>> ocal/hbase/lib/zookeeper-3.3.1.jar:/etc/zookeeper:/etc/hadoop-0.20/c > >>> onf > >>> > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.library.path=/usr/java/jdk1.6.0_23/jre/lib/amd64/se > >>> rver:/usr/java/jdk1.6.0_23/jre/lib/amd64:/usr/java/jdk1.6.0_23/j > >>> re/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib: > >>> /usr/lib > >>> > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.io.tmpdir=/tmp > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:java.compiler=<NA> > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:os.name=Linux > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:os.arch=amd64 > >>> 2010-12-21 18:17:36,462 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:os.version=2.6.18-194.3.1.el5 > >>> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:user.name=hadoop > >>> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:user.home=/home/hadoop > >>> 2010-12-21 18:17:36,463 INFO org.apache.zookeeper.ZooKeeper: Client > >>> environment:user.dir=/usr/local/hbase/logs > >>> 2010-12-21 18:17:36,464 INFO org.apache.zookeeper.ZooKeeper: > >>> Initiating client connection, connectString=hadoop-name-node:2222 > >>> sessionTimeout=60000 > >>> watcher=org.apache.hadoop.hbase.zookeeper.ZooKeeperW > >>> rap...@1bbb60c3 > >>> 2010-12-21 18:17:36,502 INFO org.apache.zookeeper.ClientCnxn: > >>> Opening socket connection to server > >>> hadoop-name-node/192.168.1.210:2222 > >>> 2010-12-21 18:17:36,508 DEBUG > >>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: > >>> <hadoop-name- > node:/hbase,org.apache.hadoop.hbase.master.HMaster>Conn > >>> ected > >>> to zookeeper again > >>> 2010-12-21 18:17:36,516 INFO org.apache.zookeeper.ClientCnxn: Socket > >>> connection established to hadoop-name-node/192.168.1.210:2222, > >>> initiating session > >>> 2010-12-21 18:17:36,542 INFO org.apache.zookeeper.ClientCnxn: > >>> Session establishment complete on server > >>> hadoop-name-node/192.168.1.210:2222, > >>> sessionid = 0x12d0b3937d20000, negotiated timeout = 60000 > >>> 2010-12-21 18:17:36,569 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,570 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,570 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,571 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,571 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,572 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,574 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,576 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,578 INFO > >>> org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC > >>> Metrics with hostName=HMaster, port=60000 > >>> 2010-12-21 18:17:36,583 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,585 INFO org.apache.hadoop.ipc.HBaseServer: > >>> Starting SocketReader > >>> 2010-12-21 18:17:36,880 ERROR > >>> org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start > >>> master > >>> java.lang.RuntimeException: Failed construction of Master: class > >>> org.apache.hadoop.hbase.master.HMasternull > >>> at > >>> > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1 > 243) > >>> at > >>> > org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMas > terCommandLine.java:143) > >>> at > >>> > org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCom > mandLine.java:102) > >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > >>> at > >>> > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerComman > dLine.java:76) > >>> at > >>> org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1286) > >>> Caused by: java.lang.reflect.UndeclaredThrowableException: Unknown > >>> exception in doAs > >>> at > >>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform > ation.java:1075) > >>> at > >>> > org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1 > 236) > >>> ... 5 more > >>> Caused by: java.security.PrivilegedActionException: > >>> java.lang.reflect.InvocationTargetException > >>> at java.security.AccessController.doPrivileged(Native Method) > >>> at javax.security.auth.Subject.doAs(Subject.java:396) > >>> at > >>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform > ation.java:1063) > >>> ... 6 more > >>> Caused by: java.lang.reflect.InvocationTargetException > >>> at > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > >>> at > >>> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructor > AccessorImpl.java:39) > >>> at > >>> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCon > structorAccessorImpl.java:27) > >>> at > >>> java.lang.reflect.Constructor.newInstance(Constructor.java:513) > >>> at > >>> org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1238) > >>> at > >>> org.apache.hadoop.hbase.master.HMaster$3.run(HMaster.java:1236) > >>> ... 9 more > >>> Caused by: java.io.EOFException > >>> at > >>> java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323) > >>> at java.io.DataInputStream.readUTF(DataInputStream.java:572) > >>> at > >>> org.apache.hadoop.hbase.util.FSUtils.getVersion(FSUtils.java:151) > >>> at > >>> org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:170) > >>> at > >>> > org.apache.hadoop.hbase.master.HMaster.checkRootDir(HMaster.java:304) > >>> at > >>> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:221) > >>> ... 15 more > >>> > >>> > >> > >
