Suppose I have a view that joins 3 tables together. If I execute a query
against this view that is answerable by only joining 2 of these 3 tables
together, can hive preform this optimization automatically?
Example:
For the below select and view, I'd like hive to avoid the join on iede_xff6.
You forgot horizontal scaling.
A fully denormalized columnar store in Hive will out preform a star schema in
Oracle in every way imaginable at scale; however, if your data isn't big enough
then this is a moot point.
If your data fits in a traditional BI warehouse, and especially if it does so
r.
Can you please describe a bit more detail your vision of "A fully denormalized
columnar store"? Are you referring to get rid of star schema altogether in Hive
and replace it with ORC tables?
Regards
On Friday, 18 December 2015, 21:13, Grant Overby (groverby)
<grove...@cisco.com&l
Looks like the user running hiveserver2 didn’t have permission (local file
system) to write to the directory specified by
HiveConf.ConfVars.LOCALSCRATCHDIR . The default scratch directory is:
“${system:java.io.tmpdir}” + File.separator + “${system:user.name}” , so it’s
unusual to see this
I call it my billion-dollar mistake. It was the invention of the null
reference in 1965.”
— Tony Hoare
[http://www.cisco.com/web/europe/images/email/signature/est2014/logo_06.png?ct=1398192119726]
Grant Overby
Software Engineer
Cisco.comhttp://www.cisco.com/
://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
From: Grant Overby (groverby) grove...@cisco.commailto:grove...@cisco.com
Date: Friday, May 22, 2015 at 1:13 PM
To: user@hive.apache.orgmailto:user@hive.apache.org
user@hive.apache.orgmailto:user
I don’t understand the question.
Why do you want them larger?
Are you looking to merge parquet files?
Are you looking to append to parquet files?
Are you concerned about the small size?
[http://www.cisco.com/web/europe/images/email/signature/est2014/logo_06.png?ct=1398192119726]
Grant Overby
I’m getting the following exception when Hive executes a query on an external
table. It seems the postscript isn’t written even though .close() is called and
returns normally. Any thoughts?
java.io.IOException: Malformed ORC file
you send me (omal...@apache.orgmailto:omal...@apache.org) the
incorrect ORC file? Which file system were you using? hdfs? Which version of
Hadoop and Hive?
Thanks,
Owen
On Fri, May 22, 2015 at 9:37 AM, Grant Overby (groverby)
grove...@cisco.commailto:grove...@cisco.com wrote:
I’m getting
I’m trying to get hive streaming ingest working with hive 1.1.0 .
The streaming ingest docs
(https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest) state:
“Partition creation being an atomic action, multiple clients can race to
create the partition, but only one will succeed,
...@cisco.com
Subject: Re: FAILED: LockException [Error 10280]: Error communicating with the
metastore
Execute below property on hive prompt
set hive.support.concurrency=false;
and then try running your query, it should work. Let me know if it won't.
On Thu, May 7, 2015 at 8:29 PM, Grant Overby (groverby
My environment has HDP 2.2 installed without hive. Hive 1.1 is installed
independently of HDP. This is a new setup.
I can get a hive cli prompt, but when I run ‘show databases;’ I get ‘FAILED:
LockException [Error 10280]: Error communicating with the metastore’. The
metastore is running. If I
Expanding on Alan’s post:
Files are intended to span many blocks and a single file may be read by many
mappers. In order for a file to be read by many mappers, it goes through a
process called input splits which splits the input around hdfs block boundaries.
If a unit of data within a file
orc files.
Grant Overby (groverby) wrote:
Thanks for the link to the hive streaming bolt. We rolled our own bolt
many moons ago to utilize hive streaming. We’ve tried it against 0.13 and
0.14 . Acid tables have been a real pain for us. We don’t believe they are
production ready. At least in our
of the recipient to ensure that this email is virus free,
therefore neither Peridale Ltd, its subsidiaries nor their employees
accept
any responsibility.
-Original Message-
From: Grant Overby (groverby) [mailto:grove...@cisco.com]
Sent: 14 April 2015 22:02
To: Gopal Vijayaraghavan; user
directly. Data must be written to disk in a huge format and then
must be copied.
I’ll dig deep here as soon as I get a chance.
On 4/14/15, 6:09 PM, Grant Overby (groverby) grove...@cisco.com wrote:
Submitting patches or test cases is tricky business for a Cisco employee.
I’ll put in the legal admin
Thanks for the link to the hive streaming bolt. We rolled our own bolt
many moons ago to utilize hive streaming. We’ve tried it against 0.13 and
0.14 . Acid tables have been a real pain for us. We don’t believe they are
production ready. At least in our use cases, Tez crashes for assorted
reasons
, Grant Overby (groverby)
grove...@cisco.commailto:grove...@cisco.com wrote:
addRow() is called in execute(). Does something look wrong with the call?
[http://www.cisco.com/web/europe/images/email/signature/est2014/logo_06.png?ct=1398192119726]
Grant Overby
Software Engineer
Cisco.comhttp
I have a Storm Trident Bolt for writing ORC File. The files are created;
however, they are always zero length. This code eventually causes an OOME. I
suspect I am missing some sort of flushing action, but don’t see anything like
that in the api.
My bolt follows. Any thoughts as to what I’m
experience doing this.
On Tue, Apr 7, 2015 at 8:43 AM, Grant Overby (groverby)
grove...@cisco.commailto:grove...@cisco.com wrote:
I have a Storm Trident Bolt for writing ORC File. The files are created;
however, they are always zero length. This code eventually causes an OOME. I
suspect I am missing
Give me a bit to extract a minimal example case and I’ll send it over.
Grant Overby
Software Engineer
Cisco.com http://www.cisco.com/
grove...@cisco.com
Mobile: 865 724 4910
Think before you print.This email may contain confidential and privileged
material for the sole use of the intended
of this
message.
Please click here
http://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
On 4/7/15, 1:23 PM, Grant Overby (groverby) grove...@cisco.com wrote:
Give me a bit to extract a minimal example case and I’ll send it over.
Grant Overby
22 matches
Mail list logo
