Hmmm, what kind of HBase cluster are you running (standalone,
pseudo-distributed, or distributed)? If distributed, you may need to pass
zookeeper connection info instead of hbase.master; I haven't tested out
distributed yet but will be soon.
JVS
On Mar 25, 2010, at 4:43 PM, Ted Yu wrote:
I removed hbase-site.xml
[r...@tyu-linux dist]# bin/hive -hiveconf hbase.master=snv-it-lin-006:60000
Hive history file=/tmp/root/hive_job_log_root_201003251614_1493474415.txt
hive> CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string,
lpm_cat int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> "hbase.key.type" = "string",
> "hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
> "hbase.table.name<http://hbase.table.name/>" = "ruletable"
> );
FAILED: Error in metadata:
MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:374)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:72)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:63)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:320)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:1445)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:124)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
)
But http://snv-it-lin-006:60010/master.jsp shows everything is normal.
And hbase.master.port is 60000 in hbase-default.xml
Some suggestions ?
Thanks
On Thu, Mar 25, 2010 at 4:21 PM, John Sichi
<[email protected]<mailto:[email protected]>> wrote:
Don't copy hbase-site.xml; you don't want to set up a new HBase cluster. Just
set your hive conf to point to your existing HBase master:
hbase.master=hbase.yoyodyne.com:60000<http://hbase.yoyodyne.com:60000/>
http://wiki.apache.org/hadoop/Hive/HBaseIntegration#Usage
JVS
On Mar 25, 2010, at 4:16 PM, Ted Yu wrote:
I got pass that error when I use CLI from trunk.
I copied hbase-site.xml from hbase master machine to trunk/dist/conf.
When I issue the following command in CLI:
CREATE EXTERNAL TABLE ruletable(key string, exactmatch_cat string, lpm_cat int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.key.type" = "string",
"hbase.columns.mapping" = "exactmatch_1.0:category,lpm_1.0:category",
"hbase.table.name<http://hbase.table.name/>" = "ruletable"
);
I get:
2010-03-25 15:49:52,557 WARN client.HConnectionManager$TableServers
(HConnectionManager.java:tableExists(411)) - Testing for table existence threw
exception
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
region server 10.10.31.143:60020<http://10.10.31.143:60020/> for region
.META.,,1, row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
java.io.IOException: Call to /10.10.31.143:60020<http://10.10.31.143:60020/>
failed on local exception: java.io.EOFException
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1048)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:454)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.tableExists(HConnectionManager.java:404)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:113)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:100)
at
org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:149)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:280)
However, I don't see any exception in hbase-hbaseadmin-regionserver.log on
10.10.31.143 within the past 4 hours.
No exception in hbase-hbaseadmin-master.log on hbase master machine either.
Please comment.
On Thu, Mar 25, 2010 at 11:22 AM, Carl Steinbach
<[email protected]<mailto:[email protected]>> wrote:
Hi Ted,
It looks like your copy of Hive does not have the changes that implemented
support for integration with HBase. HBase support was committed to trunk on
March 12th, and currently it is only available on trunk. In order to use it you
need to checkout (or update) the source from the svn repository. Instructions
describing how to do this are on the Hive wiki.
Carl
On Thu, Mar 25, 2010 at 9:29 AM, Ted Yu
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I encountered Parse Error creating table in CLI:
[r...@tyu-linux hive-0.5.0-bin]# bin/hive
Hive history file=/tmp/root/hive_job_log_root_201003240832_1752130264.txt
hive> SHOW TABLES;
OK
Time taken: 8.09 seconds
hive> CREATE EXTERNAL TABLE users(key string, state string, country string,
country_id int)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> WITH SERDEPROPERTIES (
>
> "hbase.key.type" = "string",
>
> "hbase.columns.mapping" = "info:state,info:country,info:country_id",
>
> "hbase.table.name<http://hbase.table.name/>" = "users"
>
> );
FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table file
format specification
Here is the tail of hive.log:
2010-03-24 08:32:07,937 ERROR DataNucleus.Plugin
(Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.core" requires
"org.eclipse.text" but it cannot be resolved.
2010-03-24 08:34:18,337 ERROR ql.Driver (SessionState.java:printError(248))
- FAILED: Parse Error: line 2:0 cannot recognize input 'STORED' in table
file format specification
org.apache.hadoop.hive.ql.parse.ParseException: line 2:0 cannot recognize
input 'STORED' in table file format specification
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:357)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:267)
at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:320)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:287)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
If you know how to fix my query, please share.
