Making Thrift work with Hive in client-server mode

2011-02-03 Thread Jay Ramadorai
Can someone explain how the Thriftserver finds the Hive metastore? I am running with all non-default values and need to know how to connect to Thrift so it finds Hive with the right metastore. I am running Derby in server mode on a non-default port. And my metastore name is non-default. And I

Re: Making Thrift work with Hive in client-server mode

2011-02-03 Thread Jay Ramadorai
Sorry. I had an error in my message below. I start up Derby on the same port that is specified in hive-site. So my derby start looks like: nohup $DERBY_HOME/bin/startNetworkServer -h 0.0.0.0 -p (not ) BTW, all the ports shown here are examples only. On Feb 3, 2011, at 9:22 AM, Jay

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Edward Capriolo
On Thu, Feb 3, 2011 at 12:16 AM, Alan Gates ga...@yahoo-inc.com wrote: Edward, I understand your concern with having a copy of the metastore code in Howl.  However, let's separate code from governance.  The reason Howl has a copy of Hive's metastore is not because we're proposing it for the

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Ashutosh Chauhan
+1 On Wed, Feb 2, 2011 at 13:18, Alan Gates ga...@yahoo-inc.com wrote: Howl is a table management system built to provide metadata and storage management across data processing tools in Hadoop (Pig, Hive, MapReduce, ...).  You can learn more details at http://wiki.apache.org/pig/Howl.  For

RE: Please read if you plan to use Hive 0.7.0 on Hadoop 0.20.0

2011-02-03 Thread Severance, Steve
We are not using 0.20 at eBay so we are fine with this. Steve From: Ajo Fod [mailto:ajo@gmail.com] Sent: Monday, January 31, 2011 9:49 PM To: user@hive.apache.org Subject: Re: Please read if you plan to use Hive 0.7.0 on Hadoop 0.20.0 I am new to hive and hadoop and I got the packaged

Hive queries consuming 100% cpu

2011-02-03 Thread Vijay
Hi, The simplest of hive queries seem to be consuming 100% cpu. This is with a small 4-node cluster. The machines are pretty beefy (16 cores per machine, tons of RAM, 16 M+R maximum tasks configured, 1GB RAM for mapred.child.java.opts, etc). A simple query like select count(1) from events where

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Jeff Hammerbacher
Hey, If we do go ahead with pulling the metastore out of Hive, it might make most sense for Howl to become its own TLP rather than a subproject. Yes, I did not read the proposal closely enough. I think an end state as a TLP makes more sense for Howl than as a Pig subproject. I'd really love

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread yongqiang he
I am interested in some numbers around the lines of code changes (or files of changes) which are in Howl but not in Hive? Can anyone give some information here? Thanks Yongqiang On Thu, Feb 3, 2011 at 1:15 PM, Jeff Hammerbacher ham...@cloudera.com wrote: Hey, If we do go ahead with pulling

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Ashutosh Chauhan
There are none as of today. In the past, whenever we had to have changes, we do it in a separate branch in Howl and once those get committed to hive repo, we pull it over in our trunk and drop the branch. Ashutosh On Thu, Feb 3, 2011 at 13:41, yongqiang he heyongqiang...@gmail.com wrote: I am

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread John Sichi
But Howl does layer on some additional code, right? https://github.com/yahoo/howl/tree/howl/howl JVS On Feb 3, 2011, at 1:49 PM, Ashutosh Chauhan wrote: There are none as of today. In the past, whenever we had to have changes, we do it in a separate branch in Howl and once those get

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Ashutosh Chauhan
What I am referring to is metastore/ dir of hive, part of hive code which howl cares about most. Other howl code is for additional functionalities that Howl provides (none of which lives in metastore/ dir) they are in howl/ dir. There are few build file changes, but they are trivial. Ashutosh On

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread John Sichi
I forgot about the serde dependencies...can you add those to the Initial Source note in [[HowlProposal]] just for completeness? JVS On Feb 3, 2011, at 3:11 PM, Alan Gates wrote: Yes, it adds Input and Output formats for MapReduce and load and store functions for Pig. In the future it we

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Alex Boisvert
Hi John, Just to clarify where I was going with my line of questioning. There's no Apache policy that prevents dependencies on incubator project, whether it's releases, snapshots or even home-made hacked-together packaging of an incubator project.It's been done before and as long as the

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread Alan Gates
Are you referring to the serde jar or any particular serde's we are making use of? Alan. On Feb 3, 2011, at 4:30 PM, John Sichi wrote: I forgot about the serde dependencies...can you add those to the Initial Source note in [[HowlProposal]] just for completeness? JVS On Feb 3, 2011, at

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread John Sichi
On Feb 3, 2011, at 5:09 PM, Alan Gates wrote: Are you referring to the serde jar or any particular serde's we are making use of? Both (see below). JVS [jsichi@dev1066 ~/open/howl/howl/howl/src/java/org/apache/hadoop/hive/howl] ls cli/ common/ data/ mapreduce/ pig/ rcfile/

Re: [VOTE] Sponsoring Howl as an Apache Incubator project

2011-02-03 Thread John Sichi
Got it, thanks for the correction. JVS On Feb 3, 2011, at 4:56 PM, Alex Boisvert wrote: Hi John, Just to clarify where I was going with my line of questioning. There's no Apache policy that prevents dependencies on incubator project, whether it's releases, snapshots or even home-made

if query in hive

2011-02-03 Thread Amlan Mandal
Actually I need to port some SQL queries to hive QL. Lets say I have hive table t which has columns mobile_no, cookie, ip, access_id. Lets say I want to count unique users. My definition of of unique user = all unique mobile numbers + all unique cookie (if for them mobile number not present) +

Re: if query in hive

2011-02-03 Thread Viral Bajaria
http://wiki.apache.org/hadoop/Hive/LanguageManual/UDF http://wiki.apache.org/hadoop/Hive/LanguageManual/UDFcheck conditional functions in the link above, it has the IF and CASE statement definitions. I am guessing some of them might not work with older version of Hive but not too sure. On Thu,