Hi,
> Is your huve able to connect to hadoop?
It appears to:
hadoop$ hdfs dfs -ls /user/hive
Found 1 items
drwxrwxr-x - hadoop supergroup 0 2014-02-12 17:58
/user/hive/warehouse
hadoop$ hive
hive> CREATE TABLE myTable(id INT, name STRING);
OK
Time taken: 5.893 seconds
hive> quit;
hadoop$ hdfs dfs -ls /user/hive
Found 1 items
drwxrwxr-x - hadoop supergroup 0 2014-02-12 18:10
/user/hive/warehouse
> Try putting hadoop and hive configuration files in hive conf dir
I tried this but to no avail:
hadoop$ stop-hbase.sh
...
hadoop$ stop-yarn.sh
...
hadoop$ stop-dfs.sh
...
hadoop$ cp -i /usr/local/hive/conf/* /usr/local/hadoop/etc/hadoop
hadoop$ rm -rf /usr/local/hive/conf
hadoop$ ln -s /usr/local/hadoop/etc/hadoop /usr/local/hive/conf
hadoop$ cp -i /usr/local/hbase/conf/* /usr/local/hadoop/etc/hadoop
hadoop$ rm -rf /usr/local/hbase/conf
hadoop$ ln -s /usr/local/hadoop/etc/hadoop /usr/local/hbase/conf
hadoop$ start-dfs.sh
...
hadoop$ start-yarn.sh
...
hadoop$ start-hbase.sh
...
hadoop$ jps
3543 NameNode
5521 HMaster
4150 SecondaryNameNode
5845 HRegionServer
4359 ResourceManager
6066 Jps
3818 DataNode
5428 HQuorumPeer
4617 NodeManager
hadoop$ hive
hive> CREATE TABLE foo(rowkey INT, name
STRING)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1')
> TBLPROPERTIES ('hbase.table.name' = 'foo');
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
org/apache/hadoop/hbase/HBaseConfiguration
Any other ideas?
Thanks,
-Josh Draner
On Tue, Feb 11, 2014 at 8:20 PM, dwivedishashwat
<dwivedishash...@gmail.com>wrote:
> Is your huve able to connect to hadoop???? Try putting hadoop and hive
> configuration files in hive conf dir
>
>
> Sent from Samsung Mobile
>
>
>
> -------- Original message --------
> From: Josh Draner <joshdra...@gmail.com>
> Date: 12/02/2014 08:35 (GMT+05:30)
> To: user@hive.apache.org
> Subject: FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> org/apache/hadoop/hbase/HBaseConfiguration
>
>
> Hi,
>
> I have been unsuccessful trying to setup Hive 0.12 with HBase 0.96.1.1. I
> am able to get both Hive and HBase to work independently of each other, but
> when I attempt to map a Hive table to an HBase table I am greeted with:
>
> hive> CREATE TABLE foo(rowkey INT, name STRING)
> > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> > WITH SERDEPROPERTIES ('hbase.columns.mapping' = ':key,f:c1')
> > TBLPROPERTIES ('hbase.table.name' = 'foo');
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> org/apache/hadoop/hbase/HBaseConfiguration
>
> Could anyone suggest what I need to do to fix this? I initially ran into
> the following problem:
>
> Exception in thread "main" java.lang.IllegalArgumentException: Not a
> host:port pair: PBUF
>
> and found several links (e.x.
> http://hank-ca.blogspot.com/2014/02/setup-install-hadoop-220-hbase-096-on.html,
> http://hortonworks.com/community/forums/topic/hive-and-hbase-integration/)
> suggesting that $HIVE_HOME/lib/hbase-0.94.6.1.jar should be deleted and
> replaced with hbase's jars. So I've added the following symlinks:
>
> hadoop$ for filn in `find /usr/local/hive/lib -type l`; do ls -al $filn;
> done
> lrwxrwxrwx 1 hadoop hadoop 41 Feb 11 18:03
> /usr/local/hive/lib/htrace-core-2.01.jar ->
> /usr/local/hbase/lib/htrace-core-2.01.jar
> lrwxrwxrwx 1 hadoop hadoop 54 Feb 11 18:03
> /usr/local/hive/lib/hbase-server-0.96.1.1-hadoop2.jar ->
> /usr/local/hbase/lib/hbase-server-0.96.1.1-hadoop2.jar
> lrwxrwxrwx 1 hadoop hadoop 54 Feb 11 18:03
> /usr/local/hive/lib/hbase-client-0.96.1.1-hadoop2.jar ->
> /usr/local/hbase/lib/hbase-client-0.96.1.1-hadoop2.jar
> lrwxrwxrwx 1 hadoop hadoop 56 Feb 11 18:03
> /usr/local/hive/lib/hbase-protocol-0.96.1.1-hadoop2.jar ->
> /usr/local/hbase/lib/hbase-protocol-0.96.1.1-hadoop2.jar
> lrwxrwxrwx 1 hadoop hadoop 40 Feb 11 18:04
> /usr/local/hive/lib/zookeeper-3.4.5.jar ->
> /usr/local/hbase/lib/zookeeper-3.4.5.jar
>
> The HBase table in question was created via:
>
> hbase(main):009:0> create 'foo', 'cf'
> 0 row(s) in 0.4840 seconds
>
> => Hbase::Table - foo
> hbase(main):010:0> put 'foo',2,'cf:name','Test'
> 0 row(s) in 0.0580 seconds
>
> hbase(main):011:0> scan 'foo'
> ROW COLUMN+CELL
>
> 2 column=cf:name, timestamp=1392171457664, value=Test
>
> 1 row(s) in 0.0180 seconds
>
> Version & config information:
>
> hadoop$ hadoop version
> Hadoop 2.2.0
> Subversion Unknown -r Unknown
> Compiled by jdraner on 2014-02-10T20:20Z
> Compiled with protoc 2.5.0
> From source with checksum 79e53ce7994d1628b240f09af91e1af4
> This command was run using
> /usr/src/hadoop-2.2.0-src/hadoop-dist/target/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar
>
> hadoop$ hbase version
> 2014-02-11 18:34:48,932 INFO [main] util.VersionInfo: HBase
> 0.96.1.1-hadoop2
> 2014-02-11 18:34:48,933 INFO [main] util.VersionInfo: Subversion
> file:///home/jon/proj/hbase-svn/hbase-0.96.1.1 -r Unknown
> 2014-02-11 18:34:48,933 INFO [main] util.VersionInfo: Compiled by jon on
> Tue Dec 17 12:22:12 PST 2013
>
> hadoop$ jps
> 17080 NodeManager
> 19045 HMaster
> 19370 HRegionServer
> 17704 NameNode
> 17981 DataNode
> 336 Jps
> 16822 ResourceManager
> 18313 SecondaryNameNode
> 18954 HQuorumPeer
> 32032 Main
>
> hadoop$ cat /usr/local/hbase/conf/hbase-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!--
> /**
> *
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements. See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> -->
> <configuration>
>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://localhost:9000/hbase</value>
> </property>
> <property>
> <name>hbase.master</name>
> <value>localhost:60000</value>
> <description>The host and port that the HBase master runs
> at.</description>
> </property>
> <property>
> <name>hbase.regionserver.port</name>
> <value>60020</value>
> <description>The host and port that the HBase master runs
> at.</description>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>localhost</value>
> </property>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>hbase.zookeeper.property.clientPort</name>
> <value>2181</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> The port at which the clients will connect.
> </description>
> </property>
> <property>
> <name>zookeeper.session.timeout</name>
> <value>1800000</value>
> <description>Session Time out.</description>
> </property>
> <property>
> <name>hbase.client.scanner.caching</name>
> <value>500</value>
> </property>
> <property>
> <name>hbase.regionserver.lease.period</name>
> <value>240000</value>
> </property>
> <property>
> <name>dfs.support.append</name>
> <value>true</value>
> </property>
>
> </configuration>
>
> hadoop$ tail -10 /usr/local/hive/conf/hive-env.sh
>
> # Hive Configuration Directory can be controlled by:
> # export HIVE_CONF_DIR=
>
> # Folder containing extra ibraries required for hive compilation/execution
> can be controlled by:
> # export HIVE_AUX_JARS_PATH=
>
> export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
> export HIVE_HOME=/usr/local/hive
> export HIVE_CONF_DIR=$HIVE_HOME/conf
>
> hadoop$ tail -10 /usr/local/hbase/conf/hbase-env.sh
> # For example:
> # HBASE_ROOT_LOGGER=INFO,DRFA
> # The reason for changing default to RFA is to avoid the boundary case of
> filling out disk space as
> # DRFA doesn't put any cap on the log size. Please refer to HBase-5655 for
> more context.
>
> export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
> export HBASE_REGIONSERVERS=/usr/local/hbase/conf/regionservers
> export HBASE_LOG_DIR=/tmp/hbase/logs
> export HBASE_PID_DIR=/tmp/hbase/pid
> export HBASE_MANAGES_ZK=true
>
> hadoop$ cat /etc/issue.net
> Ubuntu 12.04.4 LTS
>
> hadoop$ uname -a
> Linux host1 3.8.0-35-generic #52~precise1-Ubuntu SMP Thu Jan 30 17:24:40
> UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>
> Thanks,
>
> -Josh Draner
>
>
