Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread amareshwari sriramdasu
Measure 1: +1 Measure 2: +1 Thanks Amareshwari On Tue, Jan 29, 2013 at 1:17 AM, Carl Steinbach wrote: > I am calling a vote on the following two measures. > > Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers > > If this measure passes the Apache Hive Project Bylaws wil

Re: Automating the partition creation process

2013-01-28 Thread Mark Grover
Sadananda, See if this helps: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Recoverpartitions On Mon, Jan 28, 2013 at 8:05 PM, Sadananda Hegde wrote: > Hello, > > My hive table is partitioned by year, month and day. I have defined it as > external table. Th

Re: Automating the partition creation process

2013-01-28 Thread abhishek
Sadananda, Look at Oozie workflow. Regards Abhishek On Jan 28, 2013, at 11:05 PM, Sadananda Hegde wrote: > Hello, > > My hive table is partitioned by year, month and day. I have defined it as > external table. The M/R job correctly loads the files into the daily > subfolders. The hdfs file

Automating the partition creation process

2013-01-28 Thread Sadananda Hegde
Hello, My hive table is partitioned by year, month and day. I have defined it as external table. The M/R job correctly loads the files into the daily subfolders. The hdfs files will be loaded to /year=/month=mm/day=dd/ folders by the scheduled M/R jobs. The M/R job has some business logic in d

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread Jarek Jarcec Cecho
Measure 1: +1 (non binding) Measure 2: +1 (non binding) Jarcec On Mon, Jan 28, 2013 at 07:00:14PM -0600, Brock Noland wrote: > +1 and +1 (non-binding) > > On Mon, Jan 28, 2013 at 5:09 PM, Owen O'Malley wrote: > > +1 and +1 > > > > > > On Mon, Jan 28, 2013 at 1:56 PM, Ashish Thusoo wrote: > > >

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread Brock Noland
+1 and +1 (non-binding) On Mon, Jan 28, 2013 at 5:09 PM, Owen O'Malley wrote: > +1 and +1 > > > On Mon, Jan 28, 2013 at 1:56 PM, Ashish Thusoo wrote: > >> Measure 1: +1 >> Measure 2: +1 >> >> Ashish >> >> >> On Mon, Jan 28, 2013 at 1:11 PM, Ashutosh Chauhan > >wrote: >> >> > Measure 1: +1 >> > M

limit on hive partition string length

2013-01-28 Thread Ariel Marcus
Hey All, Just wanted to bring to everyone's attention that there is a hard limit on the length of the string that can be used for a hive partition. Unless I am mistaken, the field that stores the partition string in the metastore (PARTITIONS.PART_NAME) is a varchar(767). That seems like a pretty

Re: Thrift Hive client for CDH 4.1 HiveServer2?

2013-01-28 Thread Carl Steinbach
Hi, The Thrift IDL file for HiveServer2 is located here: service/if/cli_service.thrift Here's what you need to do in order to build the HiveServer2 Perl Thrift client using the Hive tarball from CDH4.1.x: # Set THRIFT_HOME to point to the installation directory of Thrift 0.9.0 % export THRIFT_H

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread Ashutosh Chauhan
Measure 1: +1 Measure 2: +1 Ashutosh On Mon, Jan 28, 2013 at 11:48 AM, Carl Steinbach wrote: > Measure 1: +1 (binding) > Measure 2: +1 (binding) > > On Mon, Jan 28, 2013 at 11:47 AM, Carl Steinbach wrote: > > > I am calling a vote on the following two measures. > > > > Measure 1: Amend Hive B

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread Carl Steinbach
Measure 1: +1 (binding) Measure 2: +1 (binding) On Mon, Jan 28, 2013 at 11:47 AM, Carl Steinbach wrote: > I am calling a vote on the following two measures. > > Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers > > If this measure passes the Apache Hive Project Bylaws wi

[VOTE] Amend Hive Bylaws + Add HCatalog Submodule

2013-01-28 Thread Carl Steinbach
I am calling a vote on the following two measures. Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers If this measure passes the Apache Hive Project Bylaws will be amended with the following changes: https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hiv

Re: A few JIRAs closed in v0.10.0 that don't actually appear to be working

2013-01-28 Thread Dean Wampler
Oh, another one is https://issues.apache.org/jira/browse/HIVE-446 - Implement TRUNCATE. The CLI doesn't recognize it. dean On Mon, Jan 28, 2013 at 11:44 AM, Dean Wampler < dean.wamp...@thinkbiganalytics.com> wrote: > I've noticed a few JIRA items for new features that are supposed to work > in

A few JIRAs closed in v0.10.0 that don't actually appear to be working

2013-01-28 Thread Dean Wampler
I've noticed a few JIRA items for new features that are supposed to work in v0.10.0, but don't appear to actually work: https://issues.apache.org/jira/browse/HIVE-3066 - Add the option -database DATABASE in hive cli to specify a default database to use for the cli session. The option is not recog

Providing a user name to Hive shell

2013-01-28 Thread Arun Vasu
Hi, I was using Hive with it's security enabled. It is configured with a user name "testuser" with create privilege on "default" database. It is not allowing me to use a different unix user name other than "testuser" to work in Hive. My question is, can I pass a user name of my choice to Hive for

Re: Cartesian product detection in the query plan?

2013-01-28 Thread David Morel
On 28 Jan 2013, at 17:58, Edward Capriolo wrote: Currently order by is very inefficient in hive. Hopefully you can use sort by in most cases. Makes complete sense, I should have thought of it sooner. Thanks a lot! It should be "visible" in the plan. The plan should be very different if you a

Re: Cartesian product detection in the query plan?

2013-01-28 Thread Dean Wampler
By the way, v0.10.0 adds a new CROSS JOIN feature, when you really intended to do one ;) SELECT a.x, b.y FROM tablea a CROSS JOIN tableb b; On Mon, Jan 28, 2013 at 10:58 AM, Edward Capriolo wrote: > Currently order by is very inefficient in hive. Hopefully you can use sort > by in most cases.

Re: Cartesian product detection in the query plan?

2013-01-28 Thread Edward Capriolo
Currently order by is very inefficient in hive. Hopefully you can use sort by in most cases. It should be "visible" in the plan. The plan should be very different if you are using the ON clause vs not. Can it be easily detected is another question. On Mon, Jan 28, 2013 at 11:45 AM, David Morel

Re: Cartesian product detection in the query plan?

2013-01-28 Thread David Morel
On 28 Jan 2013, at 14:29, Edward Capriolo wrote: Iirc hive.mapred.mode strict should prevent this. If not we should add it. hi Edward, Yes, that's indeed what the book claims (quoting): hive> SELECT * FROM fracture_act JOIN fracture_ads > WHERE fracture_act.planner_id = fracture_ads.plann

Re: Cartesian product detection in the query plan?

2013-01-28 Thread Edward Capriolo
Iirc hive.mapred.mode strict should prevent this. If not we should add it. On Monday, January 28, 2013, David Morel wrote: > Hi everyone, > > I had to kill some queries that were taking forever, and it turns out > they were doing cartesian products (missing ON clause on a JOIN). > > I wonder how

Re: missing documentation?

2013-01-28 Thread Lefty Leverenz
A jira was created for the missing documents but you don't really need them because most of the Hive documentation is in the wiki, and docs that aren't in the wiki haven't changed since release 0.7.1. - jira for missing docs: https://issues.apache.org/jira/browse/HIVE-3896 - Hive wiki doc

Cartesian product detection in the query plan?

2013-01-28 Thread David Morel
Hi everyone, I had to kill some queries that were taking forever, and it turns out they were doing cartesian products (missing ON clause on a JOIN). I wonder how I could see that in the EXPLAIN output (which I still find a bit cryptic). Specifically, the stage that it was stuck in was this: S