Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Hive/HBaseIntegration" page has been changed by JohnSichi. http://wiki.apache.org/hadoop/Hive/HBaseIntegration?action=diff&rev1=33&rev2=34 -------------------------------------------------- == Usage == The storage handler is built as an independent module, - {{{hive_hbase_handler.jar}}}, which must be available on the Hive + {{{hive-hbase-handler-x.y.z.jar}}}, which must be available on the Hive client auxpath, along with HBase and Zookeeper jars. It also requires the correct configuration property to be set in order to connect to the right HBase master. See [[http://hadoop.apache.org/hbase/docs/current/api/overview-summary.html#overview_description|the HBase documentation]] for how to set up an HBase cluster. - Here's an example using CLI from a source build environment, targeting a single-node HBase server: + Here's an example using CLI from a source build environment, targeting a single-node HBase server. (Note that the jar locations and names have changed in Hive 0.7, so for earlier releases, some changes are needed.) {{{ - $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive_hbase-handler.jar,$HIVE_SRC/build/dist/lib/hbase-0.20.3.jar,$HIVE_SRC/hbase-handler/build/dist/lib/zookeeper-3.2.2.jar -hiveconf hbase.master=hbase.yoyodyne.com:60000 + $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.7.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.89.0-SNAPSHOT.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.1.jar -hiveconf hbase.master=hbase.yoyodyne.com:60000 }}} Here's an example which instead targets a distributed HBase cluster where a quorum of 3 zookeepers is used to elect the HBase master: {{{ - $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive_hbase-handler.jar,$HIVE_SRC/build/dist/lib/hbase-0.20.3.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.2.2.jar -hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com + $HIVE_SRC/build/dist/bin/hive --auxpath $HIVE_SRC/build/dist/lib/hive-hbase-handler-0.7.0.jar,$HIVE_SRC/build/dist/lib/hbase-0.89.0-SNAPSHOT.jar,$HIVE_SRC/build/dist/lib/zookeeper-3.3.1.jar -hiveconf hbase.zookeeper.quorum=zk1.yoyodyne.com,zk2.yoyodyne.com,zk3.yoyodyne.com }}} The handler requires Hadoop 0.20 or higher, and has only been tested - with dependency versions hadoop-0.20.0, hbase-0.20.3 and zookeeper-3.2.2. If you are not using hbase-0.20.3, you will need to rebuild the handler with the HBase jar matching your version, and change the --auxpath above accordingly. Failure to use matching versions will lead to misleading connection failures such as MasterNotRunningException since the HBase RPC protocol changes often. + with dependency versions hadoop-0.20.x, hbase-0.89.0 and zookeeper-3.3.1. If you are not using hbase-0.89.0, you will need to rebuild the handler with the HBase jar matching your version, and change the --auxpath above accordingly. Failure to use matching versions will lead to misleading connection failures such as MasterNotRunningException since the HBase RPC protocol changes often. In order to create a new HBase table which is to be managed by Hive, use the STORED BY clause on CREATE TABLE: @@ -323, +323 @@ == Build == Code for the storage handler is located under + {{{hive/trunk/hbase-handler}}}. - {{{hive/trunk/hbase-handler}}}. The Hive build automatically enables - the storage handler build for {{{hadoop.version=0.20.x}}}, and - disables it for any other Hadoop version. This behavior can be - overridden by setting ant property {{{hbase.enabled}}} to either - {{{true}}} or {{{false}}}. - HBase and Zookeeper dependencies are currently checked in under + HBase and Zookeeper dependencies are fetched via ivy. - {{{hbase-handler/lib}}}. We will convert this to use Ivy instead once - the corresponding POM's are available. == Tests ==
