Hive Hadoop/HBase dependencies

downstream projects should be using. Most of those artifacts shade Hadoop's dependencies, to avoid causing conflicts in users' projects. HBase does the same with hbase-shaded-client. Hive doesn't seem to use these shaded artifacts, but instead refers to artifacts like hbase-client, or hadoop-hdfs, which

Re: Does Hive support Hbase-synced partitioned tables?

Thanks for answer Furcy. On Sun, Apr 22, 2018 at 8:59 PM, Furcy Pin wrote: > Hi Oleksiy, > > I must say that I don't know if partitioned HBase-backed tables are > supported in Hive, but I don't understand why you would need it. What are > you trying to do exactly? I suspect

Re: Does Hive support Hbase-synced partitioned tables?

Hi Oleksiy, I must say that I don't know if partitioned HBase-backed tables are supported in Hive, but I don't understand why you would need it. What are you trying to do exactly? I suspect that you could do it by using composite keys (Department, doc_id). Also, I would advise against using

Re: Does Hive support Hbase-synced partitioned tables?

Hi Oleksiy, I must say that I don't know if partitioned HBase-backed tables are supported in Hive, but I don't understand why you would need it. What are you trying to do exactly? I suspect that you could do it by using composite keys (Department, doc_id). Also, I would advise against using

Re: Does Hive support Hbase-synced partitioned tables?

Any updates? On Fri, Apr 20, 2018 at 10:54 AM, Oleksiy S wrote: > Hi all. > > I can create following table > > create table hbase_partitioned(doc_id STRING, EmployeeID Int, FirstName > String, Designation String, Salary Int) PARTITIONED BY (Department String) >

Does Hive support Hbase-synced partitioned tables?

Hi all. I can create following table create table hbase_partitioned(doc_id STRING, EmployeeID Int, FirstName String, Designation String, Salary Int) PARTITIONED BY (Department String) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" =

Re: Hive and HBase

Probably it is outdated. Hive can access hbase tables via external tables. The execution engine in Hive can be mr, tez, spark. Hiveql is nowadays very similar to sql . In fact, Hortonworks plans to make it sql2011:analytics compatible. Hbase can be accessed independently of Hive via sql using

Re: Hive and HBase

infrastructure built on top of Hadoop for > providing data summary, ad hoc queries, and the analysis of large data sets > using an SQL-like language called HiveQL. > > Hive transparently translates queries into MapReduce jobs that are executed > in HBase. Hive is considered

Re: Hive and HBase

k about Hadoop that says > > Apache Hive is a data warehouse infrastructure built on top of Hadoop for > providing data summary, ad hoc queries, and the analysis of large data sets > using an SQL-like language called HiveQL. > > Hive transparently translates queries into MapRedu

Re: Hive and HBase

hoc queries, and the analysis of large data sets using an SQL-like language called HiveQL. Hive transparently translates queries into MapReduce jobs that are executed in HBase. Hive is considered the de facto standard for interactive SQL queries over petabytes of data On Tuesday, 10 Nove

Is it worth of using ORC format in my case. Can I replace hive with HBase.

HBase, which is faster. Can I replace hive with HBase for processing of data daily faster. Currently it is taking 15hrs daily with hive. We have two use cases 1. For daily incremental data of around 5-10 million records. 2. For processing of 2billion records. Please inform me if any other information

Fail to create hive and hbase table.

Dear All, I install hadoop 2.6, hbase 1.0 and hive 1.1 in ubuntu and they are running normally. But, I integrate hive with hbase and execute the script below, hive CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler

Hive and Hbase types

Hi Guys, I am trying to to map Hbase table to a Hive table ( an external table). I am using a hive create statement to create the hbase tables. That works, I was wondering how does HiveSQL types relate to Hbase types? I mean how does one go about translate Hbase long to Hive SQL type. I am

Can Hive with Hbase perform with cube statement

Hi guys, I am trying to integrate Hive with HBase and perform a with cube statement. I just want to know how much of the native Hive sql like syntax is actually supported when Hbase tables are used ? I am still trying to programmatically integrate the Hbase tables, if anyone has a success

Hive over Hbase and slow query of sample data

Hello, I have a HIVE over HBASE setup using two decent sized machines. I put 1 row of sample data and a count(1) query takes roughly 10 minutes. A select * of that 1 row runs in 0.2 seconds I am using latest production cloudera dist (CDH4) and followed along https://cwiki.apache.org/confluence

Re: Hive + Secure Hbase

cannot query any of our Hive external Hbase tables. Is there anyway to make this work, or plans to fix what appears to be an impersonation issue with Hbase? I wasn't able to find and specific Jira's. I am using CDH4.3.0 with the standard components that shipped with CDH4.3.0. Thanks Justin

Re: Hive + Secure Hbase

an external Hive table around several of our Hbase tables. This works fine until we enabled secure (kerberos) client access to Hbase. Now we cannot query any of our Hive external Hbase tables. Is there anyway to make this work, or plans to fix what appears to be an impersonation issue with Hbase? I

Re: Hive + Secure Hbase

an external Hive table around several of our Hbase tables. This works fine until we enabled secure (kerberos) client access to Hbase. Now we cannot query any of our Hive external Hbase tables. Is there anyway to make this work, or plans to fix what appears to be an impersonation issue with Hbase? I

Re: Hive + Secure Hbase

Hbase tables. This works fine until we enabled secure (kerberos) client access to Hbase. Now we cannot query any of our Hive external Hbase tables. Is there anyway to make this work, or plans to fix what appears to be an impersonation issue with Hbase? I wasn't able to find and specific Jira's. I

Hive + Secure Hbase

I have just started to experiment with Hbase in our cluster. I have an secure Hbase cluster setup and would like to create an external Hive table around several of our Hbase tables. This works fine until we enabled secure (kerberos) client access to Hbase. Now we cannot query any of our Hive

Problem with Hive JDBC + HBase

I am getting an error when using the Hive JDBC driver: java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B I've seen several messages suggesting that this is a problem due to different thrift jars in hbase and hive. To avoid this problem, I copied the Hive thrift

Re: integration issure about hive and hbase

I am attaching portions from a document I had written last year while investigating Hbase and Hive. You may have already crossed that bridge….nevertheless… Please forgive me :-) if some steps seamy hacky and not very well explained….I was on a solo mission to build a Hive Data platform from

Re: integration issure about hive and hbase

Did you hbase cluster start up? The error message is more like that something wrong with the classpath. So maybe you'd better also check that. On Mon, Jul 8, 2013 at 1:54 PM, ch huang justlo...@gmail.com wrote: i get error when try create table on hbase use hive, anyone can help? hive

Re: integration issure about hive and hbase

hbase cluster is running and i also define the jar path in hive-site.xml property namehive.aux.jars.path/name valuefile:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u4.jar,file:///usr/lib/hive/lib/hbase-0.90.6-cdh3u4.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar/value /property On Mon

Re: integration issure about hive and hbase

at 1:54 PM, ch huang justlo...@gmail.com wrote: i get error when try create table on hbase use hive, anyone can help? hive CREATE TABLE hive_hbasetable_demo(key int,value string) STORED BY 'ora.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (hbase.columns.mapping

Re: integration issure about hive and hbase

...@gmail.com wrote: i get error when try create table on hbase use hive, anyone can help? hive CREATE TABLE hive_hbasetable_demo(key int,value string) STORED BY 'ora.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf1:val

Re: integration issure about hive and hbase

Hi Can you try including the zookeeper quorum and port in your hive configuration as shown below hive --auxpath .../hbase-handler.jar, .../hbase.jar, ...zookeeper.jar, .../guava.jar -hiveconf hbase.zookeeper.quorum=zk server names separated by comma -hiveconf

integration issure about hive and hbase

i get error when try create table on hbase use hive, anyone can help? hive CREATE TABLE hive_hbasetable_demo(key int,value string) STORED BY 'ora.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES (hbase.columns.mapping = :key,cf1:val) TBLPROPERTIES

Re: hive to hbase mapping

@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Date: Friday, June 14, 2013 9:54 AM To: user@hive.apache.orgmailto:user@hive.apache.org user@hive.apache.orgmailto:user@hive.apache.org Subject: Re: hive to hbase mapping Hi Sanjay, thanks for the response. I need Hbase because

Re: hive to hbase mapping

hour for example) Hbase data in Hive table? Mario 2013/6/14 Sanjay Subramanian sanjay.subraman...@wizecommerce.com 6 months back I was tasked with building a Data platform for logs and I benchmarked Hbase + Hive (queries were 8X slower) Hive only So I decided for Hive option and am

Re: hive to hbase mapping

6 months back I was tasked with building a Data platform for logs and I benchmarked Hbase + Hive (queries were 8X slower) Hive only So I decided for Hive option and am deploying that solution to production. Couple of things u can think while u design if u really want to go HBase+Hive (also

Re: hive to hbase mapping

Hi Sanjay, thanks for the response. I need Hbase because is perfect for aggregating data through the counters, and write performance is great. Now the problem is...Which is the best way for loading periodically (every hour for example) Hbase data in Hive table? Mario 2013/6/14 Sanjay

Hive over HBase Integration Issue

Hi all, In Hive over HBase case, how to configure and get Hive as HBase client to reference hbase-site.xml (for example the one in /etc/hbase/conf)? In this case it's necessary to do that since there're some specific and important client configurations for all HBase clients including Hive

Re: Hive over HBase Integration Issue

You could replicate the configs or place the client-sided hbase-site.xml in the same directory as hive-site.xml for it to get picked up via the classpath by HBase centric classes. On Sun, Feb 17, 2013 at 2:41 PM, Zheng, Kai kai.zh...@intel.com wrote: Hi all, In Hive over HBase case, how

RE: Hive over HBase Integration Issue

to my environment (like Hive or Hbase version). Regards, Kai -Original Message- From: Harsh J [mailto:ha...@cloudera.com] Sent: Sunday, February 17, 2013 10:10 PM To: hive request Cc: u...@hbase.apache.org; Andrew Purtell apurt...@apache.org (apurt...@apache.org) Subject: Re: Hive over

Re: Hive and Hbase integration issue

' WITH SERDEPROPERTIES (hbase.columns.mapping = data:raw) TBLPROPERTIES(hbase.table.name = raw_data); Failed with exception org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.org/apache//hadoop/hive/hbase/HBaseStorageHandler FAILED: Execution Error, return code 1

Re: Hive and Hbase integration issue

If you go to /tmp/user/hive.log you would get a detailed explanation on what might be wrong. Can you provide that? From here atleast, it seems like a classpath issue of hive not being able to find the hbase handler jar which can be solved by 'add jar hive-hbase-handler-xxx.jar'. On Jan 31

Hive and Hbase performance

I want to ask if a Hive Count or Scan Query will provide better performance when adding more nodes ?? if an Hbase Count or Scan Query will provide better performance when adding more nodes ??

RE: Hive or Hbase connectivity to jaspersoft reporting

: iwannaplay games [mailto:funnlearnfork...@gmail.com] Sent: Wednesday, September 12, 2012 12:01 PM To: user; user Subject: Hive or Hbase connectivity to jaspersoft reporting Hi all, I want to generate reports from hbase and hive tables.I installed jaspersoft for the same but i am unable

Re: HIVE AND HBASE

Apache Bigtop project combines Hadoop, Hbase, Hive etc into a repository. All tools are tested to be compatible with each other. There's a streamlined install process for all tools and it just makes things easy. Artem Ervits Data Analyst New York Presbyterian Hospital - Original

RE: HIVE AND HBASE

@hive.apache.org Subject: HIVE AND HBASE hi all, I am trying to install HIVE and HBASE Is there any dependency needs to be installed ? what versions of HIVE and HBASE are compatible? Is there any good document on configuring HIVE with HBASE , can any please share? Regards Abhishek

Re: HIVE AND HBASE

Look at Apache Bigtop project. Artem Ervits Data Analyst New York Presbyterian Hospital - Original Message - From: abhiTowson cal [mailto:abhishek.dod...@gmail.com] Sent: Friday, July 27, 2012 12:30 PM To: user@hive.apache.org user@hive.apache.org Subject: HIVE AND HBASE hi all, I

Re: HIVE AND HBASE

of config headaches. Chuck Connell Nuance RD Data Team Burlington, MA -Original Message- From: abhiTowson cal [mailto:abhishek.dod...@gmail.com] Sent: Friday, July 27, 2012 12:31 PM To: user@hive.apache.org Subject: HIVE AND HBASE hi all, I am trying to install HIVE

Re: HIVE AND HBASE

- Original Message - From: abhiTowson cal [mailto:abhishek.dod...@gmail.com] Sent: Friday, July 27, 2012 12:30 PM To: user@hive.apache.org user@hive.apache.org Subject: HIVE AND HBASE hi all, I am trying to install HIVE and HBASE Is there any dependency needs to be installed ? what

RE: HIVE AND HBASE

compute nodes, it is FREE. Beyond 50 nodes, you pay for the commercial version. I have used Cloudera Manager several times and it actually works. I built a development machine (1 node) with everything I need for HDFS, MapReduce, Hive, HBase, Oozie, Zookeeper... Chuck

Re: HIVE AND HBASE

(1 node) with everything I need for HDFS, MapReduce, Hive, HBase, Oozie, Zookeeper... Chuck From: Abhishek [abhishek.dod...@gmail.com] Sent: Friday, July 27, 2012 6:53 PM To: user@hive.apache.org Subject: Re: HIVE AND HBASE Hi chuck, Thanks

Error when integrate hive with hbase

Hello all, I’m a fresher with hadoop/hbase/hive, there is a requirement with our biz, I need to build the env with hive and hbase for audit log data store and analytics. I just tried to integrate hive with hbase follow the doc: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration

Re: Error when integrate hive with hbase

: Thursday, April 26, 2012 3:42 PM To: user@hive.apache.org Subject: Error when integrate hive with hbase Hello all, I’m a fresher with hadoop/hbase/hive, there is a requirement with our biz, I need to build the env with hive and hbase for audit log data store and analytics. I just tried

Re: Hbase vs Hive ve Hbase/Hive vs Pig

://permalink.gmane.org/gmane.comp.jakarta.lucene.hadoop.user/13187 Regards,     Mohammad Tariq 2012/1/22 Dalia Sobhy dalia.mohso...@hotmail.com: H all, I know I have asked this question before.. But I want to state more specific... Whats the difference in performance between Hive, Hbase, Hive

Understanding Hive and Hbase

Hi everybody, I have a question about hbase hive integration that, I have not can found in any where: if I run a hive query, this query will read the HFiles from hdfs and run into my hadoop cluster, Will the data in my region servers (not flushed yet) be process into the query? Thank you

Re: Understanding Hive and Hbase

The queries go through the region servers, not directly to HDFS. JVS On Dec 2, 2011, at 10:53 AM, Gabriel Eisbruch wrote: Hi everybody, I have a question about hbase hive integration that, I have not can found in any where: if I run a hive query, this query will read the HFiles

Re: Understanding Hive and Hbase

a question about hbase hive integration that, I have not can found in any where: if I run a hive query, this query will read the HFiles from hdfs and run into my hadoop cluster, Will the data in my region servers (not flushed yet) be process into the query? Thank you Very Much Gabriel

Re: Understanding Hive and Hbase

, not directly to HDFS. JVS On Dec 2, 2011, at 10:53 AM, Gabriel Eisbruch wrote: Hi everybody, I have a question about hbase hive integration that, I have not can found in any where: if I run a hive query, this query will read the HFiles from hdfs and run into my hadoop cluster

Re: Hive and HBase integration in Cloudera CDH3U1

the problem you've meet? On Thu, Nov 3, 2011 at 1:56 AM, Weishung Chung weish...@gmail.com wrote: Hi, I am trying to integrate Hive and HBase using Cloudera cdh3u1, but still can't get it to work? Anyone has any success? Thank you, Wei Shung

Hive and HBase integration in Cloudera CDH3U1

Hi, I am trying to integrate Hive and HBase using Cloudera cdh3u1, but still can't get it to work? Anyone has any success? Thank you, Wei Shung

RE: Hive and HBase integration in Cloudera CDH3U1

I have it working. What sort of errors are you getting? From: Weishung Chung [mailto:weish...@gmail.com] Sent: Wednesday, November 02, 2011 1:56 PM To: user@hive.apache.org Subject: Hive and HBase integration in Cloudera CDH3U1 Hi, I am trying to integrate Hive and HBase using Cloudera cdh3u1

hadoop, hive and hbase problem

Hello, I have a hadoop cluster running with the hadoop_append-jar (hadoop-core-0.20-append-r1056497-core.jar) for hbase reason. I tried hive 0.6.0 and 0.7.0 and for both each when I start it I get Exception in thread main java.lang.RuntimeException: Could not load shims in class null

Re: hadoop, hive and hbase problem

Try one of these suggestions: (1) run HBase and Hive in separate clusters (downside is that map/reduce tasks will have to issue remote request to region servers whereas normally they could run on the same nodes) (2) debug the shim exception and see if you can contribute a patch that makes

Hive on HBase

Hi, Wonder if anyone can help please? I've read that you can use Hive to create HBase tables and map the columns across. When this is used, if the HBase data is changed directly, will Hive be able to see this 'new data' when queries are run? (Or is Hive still read-only/insert once when

Re: Hive on HBase

Hi stuart, If the hbase data is changed directory,hive will able to read thoes data.In that case,hive only contains metadata information. On Tue, Apr 19, 2011 at 3:29 AM, Stuart Scott stuart.sc...@e-mis.comwrote: Hi, Wonder if anyone can help please? I’ve read that you can use Hive

Re: Hive on HBase

Hi stuart, In above case,we create hive external table.so,actual data is reside in hbase and hive only contains metadata information.In that case,whenever we run any query over hive , it read data from hbase. On Tue, Apr 19, 2011 at 6:17 AM, Ankit Jain ankitjainc...@gmail.com wrote: Hi stuart

Re: Performance between Hive queries vs. Hive over HBase queries

. Hive over HBase queries For native tables, Hive reads rows directly from HDFS. For HBase tables, it has to go through the HBase region servers, which reconstruct rows from column families (combining cache + HDFS). HBase makes it possible to keep your table up to date in real time

Re: Performance between Hive queries vs. Hive over HBase queries

Hi, Biju's example shows a factor of 5 decrease in performance when Hive points to HBase tables. Does anyone know how much this factor varies? Is if often closer to 1 or is is more often close to 10? Just trying to get a better feel for this... Thanks, Otis Sematext :: http

Re: Performance between Hive queries vs. Hive over HBase queries

Factor of 5 closely matches the results I got when I was testing. JVS On Mar 9, 2011, at 1:23 PM, Otis Gospodnetic wrote: Hi, Biju's example shows a factor of 5 decrease in performance when Hive points to HBase tables. Does anyone know how much this factor varies? Is if often closer

Re: Performance between Hive queries vs. Hive over HBase queries

On Wed, Mar 9, 2011 at 4:31 PM, John Sichi jsi...@fb.com wrote: Factor of 5 closely matches the results I got when I was testing. JVS On Mar 9, 2011, at 1:23 PM, Otis Gospodnetic wrote: Hi, Biju's example shows a factor of 5 decrease in performance when Hive points to HBase tables

Re: Performance between Hive queries vs. Hive over HBase queries

, 2011 at 11:35 AM, John Sichi jsi...@fb.com wrote: Yes. JVS On Mar 7, 2011, at 9:59 PM, Biju Kaimal wrote: Hi, I loaded a data set which has 1 million rows into both Hive and HBase tables. For the HBase table, I created a corresponding Hive table so that the data

Performance between Hive queries vs. Hive over HBase queries

Hi, I loaded a data set which has 1 million rows into both Hive and HBase tables. For the HBase table, I created a corresponding Hive table so that the data in HBase can be queried from Hive QL. Both tables have a key column and a value column For the same query (select value, count(*) from

Re: Performance between Hive queries vs. Hive over HBase queries

Yes. JVS On Mar 7, 2011, at 9:59 PM, Biju Kaimal wrote: Hi, I loaded a data set which has 1 million rows into both Hive and HBase tables. For the HBase table, I created a corresponding Hive table so that the data in HBase can be queried from Hive QL. Both tables have a key column

Re: Performance between Hive queries vs. Hive over HBase queries

...@fb.com wrote: Yes. JVS On Mar 7, 2011, at 9:59 PM, Biju Kaimal wrote: Hi, I loaded a data set which has 1 million rows into both Hive and HBase tables. For the HBase table, I created a corresponding Hive table so that the data in HBase can be queried from Hive QL. Both tables