Re: Would like to request write access to the Hive Wiki

[Carl Steinbach]

Done!

On Wed, Aug 8, 2018 at 6:44 PM, Lu  wrote:

> Hi Hive team,
>
> I would like to request the write access to the Hive Wiki because I am
> planning to write a wiki for Teradata Binary File SerDe (JIRA ticket is
> here: https://issues.apache.org/jira/browse/HIVE-20225)
>
> My confluence username is: luli (https://cwiki.apache.org/conf
> luence/display/~luli)
>
> Thanks
>
> --
>
> Lu Li
>

Re: [VOTE] Bylaws change to allow some commits without review

[Carl Steinbach]

-0
On Apr 22, 2016 4:04 PM, "Chao Sun"  wrote:

> +1
>
> On Fri, Apr 22, 2016 at 3:45 PM, Edward Capriolo 
> wrote:
>
>> +1
>>
>>
>> On Friday, April 22, 2016, Lars Francke  wrote:
>>
>>> Yet another update. I went through the PMC list.
>>>
>>> These seven have not been active (still the same list as Vikram posted
>>> during the last vote):
>>> Ashish Thusoo
>>> Kevin Wilfong
>>> He Yongqiang
>>> Namit Jain
>>> Joydeep Sensarma
>>> Ning Zhang
>>> Raghotham Murthy
>>>
>>> There are 29 PMCs in total - 7 = 22 active * 2/3 = 15 votes required
>>>
>>> So far the following PMCs have voted:
>>>
>>> Alan Gates
>>> Jason Dere
>>> Sushanth Sowmyan
>>> Lefty Leverenz
>>> Navis Ryu
>>> Owen O'Malley
>>> Prasanth J
>>> Sergey Shelukhin
>>> Thejas Nair
>>>
>>> = 9 +1s
>>>
>>> So I'm hoping for six more. I've contacted a bunch of PMCs (sorry for
>>> the spam!) and hope to get a few more.
>>>
>>> In addition there have been six non-binding +1s. Thank you everyone for
>>> voting.
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Apr 22, 2016 at 10:42 PM, Lars Francke 
>>> wrote:
>>>
 Hi everyone, thanks for the votes. I've been held up by personal stuff
 this week but as there have been no -1s or other objections I'd like to
 keep this vote open a bit longer until I've had time to go through the PMCs
 and contact those that have not yet voted.

 On Thu, Apr 21, 2016 at 9:12 PM, Denise Rogers 
 wrote:

> +1
>
> Regards,
> Denise
> Cell - (860)989-3431
>
> Sent from mi iPhone
>
> On Apr 21, 2016, at 2:56 PM, Sergey Shelukhin 
> wrote:
>
> +1
>
> From: Tim Robertson 
> Reply-To: "user@hive.apache.org" 
> Date: Wednesday, April 20, 2016 at 06:17
> To: "user@hive.apache.org" 
> Subject: Re: [VOTE] Bylaws change to allow some commits without review
>
> +1
>
> On Wed, Apr 20, 2016 at 1:24 AM, Jimmy Xiang 
> wrote:
>
>> +1
>>
>> On Tue, Apr 19, 2016 at 2:58 PM, Alpesh Patel 
>> wrote:
>> > +1
>> >
>> > On Tue, Apr 19, 2016 at 1:29 PM, Lars Francke <
>> lars.fran...@gmail.com>
>> > wrote:
>> >>
>> >> Thanks everyone! Vote runs for at least one more day. I'd
>> appreciate it if
>> >> you could ping/bump your colleagues to chime in here.
>> >>
>> >> I'm not entirely sure how many PMC members are active and how many
>> votes
>> >> we need but I think a few more are probably needed.
>> >>
>> >> On Mon, Apr 18, 2016 at 8:02 PM, Thejas Nair <
>> the...@hortonworks.com>
>> >> wrote:
>> >>>
>> >>> +1
>> >>>
>> >>> 
>> >>> From: Wei Zheng 
>> >>> Sent: Monday, April 18, 2016 10:51 AM
>> >>> To: user@hive.apache.org
>> >>> Subject: Re: [VOTE] Bylaws change to allow some commits without
>> review
>> >>>
>> >>> +1
>> >>>
>> >>> Thanks,
>> >>> Wei
>> >>>
>> >>> From: Siddharth Seth 
>> >>> Reply-To: "user@hive.apache.org" 
>> >>> Date: Monday, April 18, 2016 at 10:29
>> >>> To: "user@hive.apache.org" 
>> >>> Subject: Re: [VOTE] Bylaws change to allow some commits without
>> review
>> >>>
>> >>> +1
>> >>>
>> >>> On Wed, Apr 13, 2016 at 3:58 PM, Lars Francke <
>> lars.fran...@gmail.com>
>> >>> wrote:
>> 
>>  Hi everyone,
>> 
>>  we had a discussion on the dev@ list about allowing some forms
>> of
>>  contributions to be committed without a review.
>> 
>>  The exact sentence I propose to add is: "Minor issues (e.g.
>> typos, code
>>  style issues, JavaDoc changes. At committer's discretion) can be
>> committed
>>  after soliciting feedback/review on the mailing list and not
>> receiving
>>  feedback within 2 days."
>> 
>>  The proposed bylaws can also be seen here
>>  <
>> https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Project+Bylaws+-+April+2016
>> >
>> 
>>  This vote requires a 2/3 majority of all Active PMC members so
>> I'd love
>>  to get as many votes as possible. The vote will run for at least
>> six days.
>> 
>>  Thanks,
>>  Lars
>> >>>
>> >>>
>> >>
>> >
>>
>
>

>>>
>>
>> --
>> Sorry this was sent from mobile. Will do less grammar and spell check
>> than usual.
>>
>
>

[ANNOUNCE] New Hive PMC Chair - Ashutosh Chauhan

[Carl Steinbach]

I am very happy to announce that Ashutosh Chauhan is taking over as the new
VP of the Apache Hive project. Ashutosh has been a longtime contributor to
Hive and has played a pivotal role in many of the major advances that have
been made over the past couple of years. Please join me in congratulating
Ashutosh on his new role!

[ANNOUNCE] New Hive Committer - Lars Francke

[Carl Steinbach]

The Apache Hive PMC has voted to make Lars Francke a committer on the
Apache Hive Project.

Please join me in congratulating Lars!

Thanks.

- Carl

[ANNOUNCE] New Hive Committer - Mithun Radhakrishnan

[Carl Steinbach]

The Apache Hive PMC has voted to make Mithun Radhakrishnan a committer on
the Apache Hive Project.

Please join me in congratulating Mithun.

Thanks.

- Carl

[ANNOUNCE] New Hive Committers - Jimmy Xiang, Matt McCline, and Sergio Pena

[Carl Steinbach]

The Apache Hive PMC has voted to make Jimmy Xiang, Matt McCline, and Sergio
Pena committers on the Apache Hive Project.

Please join me in congratulating Jimmy, Matt, and Sergio.

Thanks.

- Carl

[ANNOUNCE] New Hive PMC Member - Sergey Shelukhin

[Carl Steinbach]

I am pleased to announce that Sergey Shelukhin has been elected to the Hive
Project Management Committee. Please join me in congratulating Sergey!

Thanks.

- Carl

[ANNOUNCE] New Hive Committers -- Chao Sun, Chengxiang Li, and Rui Li

[Carl Steinbach]

The Apache Hive PMC has voted to make Chao Sun, Chengxiang Li, and Rui Li
committers on the Apache Hive Project.

Please join me in congratulating Chao, Chengxiang, and Rui!

Thanks.

- Carl

[ANNOUNCE] New Hive PMC Members - Szehon Ho, Vikram Dixit, Jason Dere, Owen O'Malley and Prasanth Jayachandran

[Carl Steinbach]

I am pleased to announce that Szehon Ho, Vikram Dixit, Jason Dere, Owen
O'Malley and Prasanth Jayachandran have been elected to the Hive Project
Management Committee. Please join me in congratulating the these new PMC
members!

Thanks.

- Carl

[ANNOUNCE] New Hive PMC Member - Prasad Mujumdar

[Carl Steinbach]

I am pleased to announce that Prasad Mujumdar has been elected to the Hive
Project Management Committee. Please join me in congratulating Prasad!

Thanks.

- Carl

[ANNOUNCE] New Hive PMC Member - Alan Gates

[Carl Steinbach]

I am pleased to announce that Alan Gates has been elected to the Hive
Project Management Committee. Please join me in congratulating Alan!

Thanks.

- Carl

[ANNOUNCE] New Hive Committer - John Pullokkaran

[Carl Steinbach]

The Apache Hive PMC has voted to make John Pullokkaran a committer on the
Apache Hive Project.

Please join me in congratulating John!

Thanks.

- Carl

[ANNOUNCE] New Hive Committer - Eugene Koifman

[Carl Steinbach]

The Apache Hive PMC has voted to make Eugene Koifman a committer on the
Apache Hive Project.

Please join me in congratulating Eugene!

Thanks.

- Carl

[ANNOUNCE] New Hive Committers - Gopal Vijayaraghavan and Szehon Ho

[Carl Steinbach]

The Apache Hive PMC has voted to make Gopal Vijayaraghavan and Szehon Ho
committers on the Apache Hive Project.

Please join me in congratulating Gopal and Szehon!

Thanks.

- Carl

Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta

[Carl Steinbach]

+ Prasanth's correct email address


On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang  wrote:

> Congratulations to Prasanth and Vaibhav!
>
> --Xuefu
>
>
> On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach  wrote:
>
> > The Apache Hive PMC has voted to make Prasanth J and Vaibhav
> > Gumashta committers on the Apache Hive Project.
> >
> > Please join me in congratulating Prasanth and Vaibhav!
> >
> > Thanks.
> >
> > - Carl
> >
>

[ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta

[Carl Steinbach]

The Apache Hive PMC has voted to make Prasanth J and Vaibhav
Gumashta committers on the Apache Hive Project.

Please join me in congratulating Prasanth and Vaibhav!

Thanks.

- Carl

[ANNOUNCE] New Hive Committers - Alan Gates, Daniel Dai, and Sushanth Sowmyan

[Carl Steinbach]

The Apache Hive PMC has voted to make Alan Gates, Daniel Dai, and Sushanth
Sowmyan committers on the Apache Hive Project.

Please join me in congratulating Alan, Daniel, and Sushanth!

- Carl

[ANNOUNCE] New Hive PMC Member - Xuefu Zhang

[Carl Steinbach]

I am pleased to announce that Xuefu Zhang has been elected to the Hive
Project Management Committee. Please join me in congratulating Xuefu!

Thanks.

Carl

[ANNOUNCE] New Hive Committer - Remus Rusanu

[Carl Steinbach]

The Apache Hive PMC has voted to make Remus Rusanu a committer on the
Apache Hive Project.

Please join me in congratulating Remus!

Thanks.

Carl

[ANNOUNCE] New Hive Committers - Sergey Shelukhin and Jason Dere

[Carl Steinbach]

The Apache Hive PMC has voted to make Sergey Shelukhin and Jason Dere
committers on the Apache Hive Project.

Please join me in congratulating Sergey and Jason!

Thanks.

Carl

Re: [DISCUSS] Proposed Changes to the Apache Hive Project Bylaws

[Carl Steinbach]

gt; In other words, I think this can be a guidance instead of a law in
> the
> >> >> by-laws. What do others in hive community think about this ?
> >> >>
> >> >> This has been working well in case of other apache hadoop related
> >> projects.
> >> >>
> >> >>
> >> >> On Fri, Dec 27, 2013 at 2:28 PM, Sergey Shelukhin
> >> >>  wrote:
> >> >> > I actually have a patch out on a jira that says it will be
> committed
> >> in
> >> >> 24
> >> >> > hours from long ago ;)
> >> >> >
> >> >> > Is 24h rule is needed at all? In other projects, I've seen patches
> >> simply
> >> >> > reverted by author (or someone else). It's a rare occurrence, and
> it
> >> >> should
> >> >> > be possible to revert a patch if someone -1s it after commit, esp.
> >> within
> >> >> > the same 24 hours when not many other changes are in.
> >> >> >
> >> >> >
> >> >> > On Fri, Dec 27, 2013 at 1:03 PM, Thejas Nair <
> the...@hortonworks.com>
> >> >> wrote:
> >> >> >>
> >> >> >> I agree with Ashutosh that the 24 hour waiting period after +1 is
> >> >> >> cumbersome, I have also forgotten to commit patches after +1,
> >> >> >> resulting in patches going stale.
> >> >> >>
> >> >> >> But I think 24 hours wait between creation of jira and patch
> commit
> >> is
> >> >> >> not very useful, as the thing to be examined is the patch and not
> the
> >> >> >> jira summary/description.
> >> >> >> I think having a waiting period of 24 hours between a jira being
> made
> >> >> >> 'patch available' and committing is better and sufficient.
> >> >> >>
> >> >> >>
> >> >> >> On Fri, Dec 27, 2013 at 11:44 AM, Ashutosh Chauhan <
> >> >> hashut...@apache.org>
> >> >> >> wrote:
> >> >> >> > Proposed changes look good to me, both suggested by Carl and
> >> Thejas.
> >> >> >> > Another one I would like to add for consideration is: 24 hour
> rule
> >> >> >> > between
> >> >> >> > +1 and commit. Since this exists only in Hive (no other apache
> >> project
> >> >> >> > which I am aware of) this surprises new contributors. More
> >> >> importantly,
> >> >> >> > I
> >> >> >> > have seen multiple cases where patch didn't get committed
> because
> >> >> >> > committer
> >> >> >> > after +1 forgot to commit after 24 hours have passed. I propose
> to
> >> >> >> > modify
> >> >> >> > that one such that there must be 24 hour duration between
> creation
> >> of
> >> >> >> > jira
> >> >> >> > and patch commit, that will ensure that there is sufficient time
> >> for
> >> >> >> > folks
> >> >> >> > to see changes which are happening on trunk.
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Ashutosh
> >> >> >> >
> >> >> >> >
> >> >> >> > On Fri, Dec 27, 2013 at 9:33 AM, Thejas Nair <
> >> the...@hortonworks.com>
> >> >> >> > wrote:
> >> >> >> >
> >> >> >> >> The changes look good to me.
> >> >> >> >> Only concern I have is with the 7 days for release candidate
> >> voting.
> >> >> >> >> Based on my experience with releases, it often takes few
> cycles to
> >> >> get
> >> >> >> >> the candidate out, and people tend to vote closer to the end of
> >> the
> >> >> >> >> voting period. This can mean that it takes several weeks to
> get a
> >> >> >> >> release out. But this will not be so much of a problem as long
> as
> >> >> >> >> people don't wait for end of the voting period to vote, or if
> they
> >> >> >> >> look at the candidate branch even before the release candidate
> is
&

[ANNOUNCE] New Hive Committer - Vikram Dixit

[Carl Steinbach]

The Apache Hive PMC has voted to make Vikram Dixit a committer on the
Apache Hive Project.

Please join me in congratulating Vikram!

Thanks.

Carl

[ANNOUNCE] New Hive PMC Member - Gunther Hagleitner

[Carl Steinbach]

I am pleased to announce that Gunther Hagleitner has been elected to the
Hive Project Management Committee. Please join me in congratulating Gunther!

Thanks.

Carl

[DISCUSS] Proposed Changes to the Apache Hive Project Bylaws

[Carl Steinbach]

I think we should make several changes to the Apache Hive Project Bylaws.
The proposed changes are available for review here:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=38568856

Most of the changes were directly inspired by provisions found in the
Apache Hadoop Project Bylaws.

Summary of proposed changes:

* Add provisions for branch committers and speculative branches.

* Define the responsibilities of a release manager.

* PMC Chairs serve for one year and are elected by the PMC using Single
Transferable Vote (STV) voting.

* With the exception of code change votes, the minimum length of all voting
periods is extended to seven days.

Thanks.

Carl

[ANNOUNCE] New Hive Committers - Jitendra Nath Pandey and Eric Hanson

[Carl Steinbach]

The Apache Hive PMC has voted to make Jitendra Nath Pandey and Eric Hanson
committers on the Apache Hive project.

Please join me in congratulating Jitendra and Eric!

Thanks.

Carl

[ANNOUNCE] New Hive Committer and PMC Member - Lefty Leverenz

[Carl Steinbach]

The Apache Hive PMC has voted to make Lefty Leverenz a committer and PMC
member on the Apache Hive Project.

Please join me in congratulating Lefty!

Thanks.

Carl

[ANNOUNCE] New Hive PMC Member - Harish Butani

[Carl Steinbach]

I am pleased to announce that Harish Butani has been elected to the Hive
Project Management Committee. Please join me in congratulating Harish!

Thanks.

Carl

[ANNOUNCE] New Hive Committer - Prasad Mujumdar

[Carl Steinbach]

The Apache Hive PMC has voted to make Prasad Mujumdar a committer on the
Apache Hive Project.

Please join me in congratulating Prasad!

Thanks.

Carl

[ANNOUNCE] New Hive Committer - Xuefu Zhang

[Carl Steinbach]

The Apache Hive PMC has voted to make Xuefu Zhang a committer on the Apache
Hive project.

Please join me in congratulating Xuefu!

Thanks.

Carl

Re: Hive compilation error

[Carl Steinbach]

Hi Mohammad,

I can't find the mybucket_*.q files on trunk. Are you using a different
branch?

1. Change the code in TestCliDriver.vm to consider this use cases during
> source generation.
>

I think replacing the call to indexOf() with one to lastIndexOf() will fix
the problem. If you want to submit a patch I'll review it.

Thanks.

Carl

[ANNOUNCE] New Hive PMC Members - Thejas Nair and Brock Noland

[Carl Steinbach]

I am pleased to announce that Thejas Nair and Brock Noland have been
elected to the Hive Project Management Committee. Please join me in
congratulating Thejas and Brock!

Thanks.

Carl

Re: HS2 ODBC incompatibility

[Carl Steinbach]

I think you need to set TFetchResultsReq.maxRows to a positive value.


On Tue, Oct 22, 2013 at 10:47 AM, Haroon Muhammad
wrote:

>
> Looking at some of the sample code in java implementation, can't figure
> out where I am going wrong. Is it possible to point out what is that I am
> doing wrong here ? It executes the select query fine at the server end (
> looking at the logs), however, fetchRes->results.rows.size() turns out to
> be zero.
>
> // quer_str = "SELECT * FROM foo";
>
> TOpenSessionReq *sessionReq = *new* TOpenSessionReq();
>
> sessionReq->__set_client_protocol((TProtocolVersion::type)0);
>
>
> TOpenSessionResp *sessionRes = *new* TOpenSessionResp();
>
> //TSessionHandle *sessionHandle = *new* TSessionHandle();
>
>
> connection->client->OpenSession(*sessionRes, *sessionReq);
>
>
> TExecuteStatementReq *execStmtReq = *new* TExecuteStatementReq();
>
> TExecuteStatementResp *execStmtRes = *new* TExecuteStatementResp();
>
>
> execStmtReq->__set_sessionHandle(sessionRes->sessionHandle);
>
> execStmtReq->__set_statement(query_str);
>
>
>  connection->client->ExecuteStatement(*execStmtRes, *execStmtReq);
>
>
>  TFetchResultsReq *fetchReq = *new* TFetchResultsReq();
>
> fetchReq->__set_operationHandle(execStmtRes->operationHandle);
>
> fetchReq->__set_orientation((TFetchOrientation::type)4);
>
>
>  TFetchResultsResp *fetchRes = *new* TFetchResultsResp();
>
> connection->client->FetchResults(*fetchRes, *fetchReq);
>
> std::vector vRow = fetchRes->results.rows;
>
> cout<<"Row size:" << vRow.size() <<*endl*;
>
>
>
> Thanks,
> --
> Date: Tue, 22 Oct 2013 09:53:18 -0700
>
> Subject: Re: HS2 ODBC incompatibility
> From: pras...@cloudera.com
> To: user@hive.apache.org
>
>
>
>The Apache Hive JDBC driver (
> https://github.com/apache/hive/tree/trunk/jdbc/src/java/org/apache/hive/jdbc) 
> is
> Java implementation of thrift client for HS2.
>
> thanks
> Prasad
>
>
>
> On Tue, Oct 22, 2013 at 7:04 AM, Haroon Muhammad  > wrote:
>
> Thanks Carl and Tejas!!
>
> Actually my application ( written in C) is based on the ODBC source i.e.
> makes use of ODBC client source files. It works fine for Hiveserver1 but
> fails with Hiveserver2 for obvious reasons. I am trying to incorporate
> TCLIService now and facing issue with "FetchResults" i.e. 
> TFetchResultsResp->results.rows.size()
> is always zero (I am doing a SELECT which should return 10k rows in my
> table).
>
> Would it be possible to share/point to a working thrift client (any
> language) sample code for HiveServer2? It would help me verify the steps I
> am doing.
>
> Thanks
> --
> Date: Mon, 21 Oct 2013 12:11:27 -0700
> Subject: Re: HS2 ODBC incompatibility
> From: cwsteinb...@gmail.com
> To: user@hive.apache.org
>
>
> Cloudera has an ODBC driver that is compatible with both HiveServer1 and
> HiveServer2. More info here:
>
>
> http://www.cloudera.com/content/cloudera-content/cloudera-docs/Connectors/PDF/Cloudera-ODBC-Driver-for-Apache-Hive-Install-Guide.pdf
>
> Thanks.
>
> Carl
>
>
> On Mon, Oct 21, 2013 at 12:01 PM, Thejas Nair wrote:
>
> Yes, the current odbc driver source in hive is not compatible with
> hive server2. I am not aware of any body working on it.
> But you can download odbc driver add on for hive server2, for free
> from the hortonworks website -
> http://hortonworks.com/download/download-archives/
>
> On Mon, Oct 21, 2013 at 5:06 AM, Haroon Muhammad
>  wrote:
> > Hi,
> >
> > Source under ODBC seems to be incompatible with HS2's changed RPC thrift
> > interface. Are there any plans on getting an updated version out any time
> > sooner ?
> >
> > Thanks,
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
>
>
>

Re: HS2 ODBC incompatibility

[Carl Steinbach]

Cloudera has an ODBC driver that is compatible with both HiveServer1 and
HiveServer2. More info here:

http://www.cloudera.com/content/cloudera-content/cloudera-docs/Connectors/PDF/Cloudera-ODBC-Driver-for-Apache-Hive-Install-Guide.pdf

Thanks.

Carl


On Mon, Oct 21, 2013 at 12:01 PM, Thejas Nair wrote:

> Yes, the current odbc driver source in hive is not compatible with
> hive server2. I am not aware of any body working on it.
> But you can download odbc driver add on for hive server2, for free
> from the hortonworks website -
> http://hortonworks.com/download/download-archives/
>
> On Mon, Oct 21, 2013 at 5:06 AM, Haroon Muhammad
>  wrote:
> > Hi,
> >
> > Source under ODBC seems to be incompatible with HS2's changed RPC thrift
> > interface. Are there any plans on getting an updated version out any time
> > sooner ?
> >
> > Thanks,
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Apache Hive 0.12.0 Release Candidate 1

[Carl Steinbach]

+1 (binding)


> Regarding the 3 day deadline for voting, that is what is in the hive
> bylaws. I also see that has been followed in last few releases I
> checked.


3 days is the minimum length of the voting period, not the maximum.

Thanks.

Carl

Re: Interesting claims that seem untrue

[Carl Steinbach]

It doesn't sound like others have seen this yet so I thought I'd point out
that
Carter Shanklin explains his methodology for computing these statistics in
this comment:

http://hortonworks.com/blog/stinger-phase-2-the-journey-to-100x-faster-hive/#comment-160943


On Thu, Sep 12, 2013 at 6:09 PM, Navis류승우  wrote:

> It looks like counting codes from company of committer, not of the author.
>
> Considering the huge amount of works by Ashutosh, it's not strange.
>
>
> 2013/9/13 Sanjay Subramanian 
>
>>  I have not read the full blogs but in the year 2013 , IMHO , LOC is a
>> very old metric that defines good software any more...
>>
>>   From: Edward Capriolo 
>> Reply-To: "user@hive.apache.org" 
>> Date: Thursday, September 12, 2013 7:19 AM
>> To: "hive-u...@hadoop.apache.org" , "<
>> hive-...@hadoop.apache.org>" 
>> Subject: Interesting claims that seem untrue
>>
>>   I was reading the horton-works blog and found an interesting article.
>>
>> http://hortonworks.com/blog/stinger-phase-2-the-journey-to-100x-faster-hive/#comment-160753
>>
>>  There is a very interesting graphic which attempts to demonstrate lines
>> of code in the 12 release.
>> http://hortonworks.com/wp-content/uploads/2013/09/hive4.png
>>
>>  Although I do not know how they are calculated, they are probably
>> counting code generated by tests output, but besides that they are wrong.
>>
>>  One claim is that Cloudera contributed 4,244 lines of code.
>>
>>  So to debunk that claim:
>>
>>  In https://issues.apache.org/jira/browse/HIVE-4675 Brock Noland from
>> cloudera, created the ptest2 testing framework. He did all the work for
>> ptest2 in hive 12, and it is clearly more then 4,244
>>
>>  This consists of 84 java files
>> [edward@desksandra ptest2]$ find . -name "*.java" | wc -l
>> 84
>>  and by itself is 8001 lines of code.
>> [edward@desksandra ptest2]$ find . -name "*.java" | xargs cat | wc -l
>> 8001
>>
>>  [edward@desksandra hive-trunk]$ wc -l HIVE-4675.patch
>> 7902 HIVE-4675.patch
>>
>>  This is not the only feature from cloudera in hive 12.
>>
>>  There is also a section of the article that talks of a "ROAD MAP" for
>> hive features. I did not know we (hive) had a road map. I have advocated
>> switching to feature based release and having a road map before, but it was
>> suggested that might limit people from itch-scratching.
>>
>>
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> ==
>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>
>

[ANNOUNCE] New Hive Committer - Yin Huai

[Carl Steinbach]

The Apache Hive PMC has voted to make Yin Huai a committer on the Apache
Hive project.

Please join me in congratulating Yin!

Thanks.

Carl

Re: Broken Hive docs -- links not working

[Carl Steinbach]

The best way to get this resolved is to file an ASFINFRA JIRA ticket. It
doesn't look like any of us have the privileges necessary to stop the
autoexport job, so the ball is clearly in INFRA's court.

Thanks.

Carl


On Mon, Sep 2, 2013 at 3:37 PM, Lefty Leverenz wrote:

> Edward Capriolo talked about broken links in the Hive wiki in another
> thread on the users list ("What is this windowing query supposed to do?")
> so I'm moving the discussion here.
>
> Recap:
>
> Edward:
>
> > Most of the links here:
> > https://cwiki.apache.org/Hive/languagemanual.html
> >
> > are broken...
> > What gives?
> > The language manual used to be fairly useful, now it is in major
> dis-array.
> >
>
> Lefty:
>
> > Once again the /confluence/display version of the wiki is fine while the
> > non-display version is glitchy.  First the {code} tags gave invisible
> > sample code, and now some of the doc links don't work -- but they're fine
> > here:  https://cwiki.apache.org/confluence/display/Hive/LanguageManual.
> >
> > Can anyone explain this, or better yet solve it?  Until it's solved,
> > should we include a link to the display version at the top of each
> wikidoc?
> >
>
> Edward:
>
> > The problem is if you follow our site from the main page you find the
> > broken docs, not the confluence ones.
> >
>
> New reply from Lefty:  Do you mean the Wiki tab?  When I follow it, I go to
> the "display" version at
> https://cwiki.apache.org/confluence/display/Hive/Home, not the broken
> version at https://cwiki.apache.org/Hive/home.html.
>
> Another problem is that there's no link to the wiki in the menu under
> Documentation.  That can be fixed fairly easily, so I'll open a JIRA.
>
> Spot check for broken links in the non-display
> version
> :
>
>- Getting Started:  404 Not Found
>(https://cwiki.apache.org/Hive/gettingstarted.html)
>- Tutorial:  404 Not Found
>(https://cwiki.apache.org/Hive/tutorial.html)
>- Language Manual:  okay
>(https://cwiki.apache.org/Hive/languagemanual.html)
>- Operators and Functions: 404 Not Found
>(https://cwiki.apache.org/Hive/operatorsandfunctions.html)
>- Web Interface:  404 Not Found
>(https://cwiki.apache.org/Hive/hivewebinterface.html)
>- Hive Client: okay
>(https://cwiki.apache.org/Hive/hiveclient.html)
>- HiveServer2 Clients:  okay
>(https://cwiki.apache.org/Hive/hiveserver2-clients.html)
>- Change Log:  404 Not Found
>(https://cwiki.apache.org/Hive/hivechangelog.html)
>- Avro SerDe:  okay
>(https://cwiki.apache.org/Hive/avroserde.html)
>
> So much for my hope of finding a pattern.  Now to check how the links are
> marked up:
>
> h1. General Information about Hive
>
> * [Getting Started|GettingStarted]
> [...]
>
> h1. User Documentation
>
> * [Hive Tutorial|Tutorial]
> * [HiveQL Language Manual (Queries, DML, DDL, and CLI)|LanguageManual]
> * [Hive Operators and Functions|OperatorsAndFunctions]
> * [Hive Web Interface|HiveWebInterface]
> * [Hive Client (JDBC, ODBC, Thrift, etc)|HiveClient]
> * [HiveServer2 Client|
> https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients]
> * [Hive Change Log|HiveChangeLog]
> * [Avro SerDe|AvroSerDe]
>
>
> Again, nothing obvious.  I think the use of a URL for HS2 Clients was just
> to get around the space in the title, but there's probably another way to
> do it -- I'll check my notes and fix it if I can.
>
> I'll check a few more later, but this doesn't seem to be fruitful
> debugging.  It isn't a matter of new vs. old docs (both Hive Client and HS2
> Clients are okay).  All of these links work fine in the
> "display"
> version of the wiki.  Most of them are top-level files in the
> hierarchy<
> https://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=Hive&openId=27362069#selectedPageInHierarchy
> >,
> except for Operators and Functions (a broken link), HiveServer2 Clients (a
> link that works), and Hive Change Log (broken) and those are all children
> of the Home page.
>
> It's a mystery.
>
> -- Lefty
>

[ANNOUNCE] New Hive Committer - Thejas Nair

[Carl Steinbach]

The Apache Hive PMC has voted to make Thejas Nair a committer on the Apache
Hive project.

Please join me in congratulating Thejas!

[ANNOUNCE] New Hive Committer - Gunther Hagleitner

[Carl Steinbach]

The Apache Hive PMC has voted to make Gunther Hagleitner a
committer on the Apache Hive project.

Congratulations Gunther!

Carl

[ANNOUNCE] New Hive Committer - Brock Noland

[Carl Steinbach]

The Apache Hive PMC has passed a vote to make Brock Noland a new committer
on the project.

Brock, please submit your ICLA to the Apache Software Foundation as
described here:

http://www.apache.org/licenses/#clas

Thanks.

Carl

[ANNOUNCE] July Hive User Group Meetup @ LinkedIn

[Carl Steinbach]

The next Bay Area Hive User Group Meetup is happening on
Wednesday, July 24th at LinkedIn's offices in Mountain View. The
format will be a series of short (15 min) talks preceded by
refreshments and networking. Please join me in thanking LinkedIn
for providing the meeting space and refreshments.

Important: You must RSVP on the Meetup page in order to attend.

http://meetu.ps/1LQ4PP

Schedule

* 6pm: Doors Open
* 6pm-6:30pm: Networking and Refreshments
* 6:30-8pm: Presentations


Talks:

* TBA (contact me if you want to give a talk!)

Re: hiveserver2 Thrift Interface With Perl

[Carl Steinbach]

This SASL module for Perl may be of interest:

http://search.cpan.org/~gbarr/Authen-SASL-2.16/



On Tue, May 14, 2013 at 9:44 AM, Prasad Mujumdar wrote:

>
>HiveServer2 uses Thrift SASL transport by default. AFAIK, Thrift
> doesn't support sasl transport for perl. In order to connect from a perl
> thrift client, you need to turn off the SASL in HiveServer2. This can be
> done by setting a property hive.server2.authentication to NOSASL in the
> hive-site.xml. Note that with this change, all other clients like JDBC
> applications and beeline, will also need to make non-sasl connection to
> HiveServer2. The JDBC connection URL should
> be jdbc:hive2://:/default;auth=noSasl
>
> thanks
> Prasad
>
>
> On Tue, May 14, 2013 at 3:35 AM, Dave Cardwell wrote:
>
>> I wrote a few reporting scripts in Perl that connected to Hive via the
>> Thrift interface.
>>
>> Since we upgraded CDH to 4.2.0 and hiveserver2 was installed (with Hive
>> 0.10.0) the scripts no longer ran due to the API changes.
>>
>> I regenerated the Perl modules from the .thrift files and have tried to
>> translate the Java examples I’ve found online, but cannot for the life of
>> me get it working with the new interface.
>>
>> The Java examples seem to use a TOpenSessionReq class but I cannot find
>> this anywhere in the generated Perl modules. If I try to skip that part and
>> go straight to $client->OpenSession() without an argument, the TCLIService
>> module itself complains that it cannot create a TOpenSessionResp object
>> because the class is not loaded.
>>
>> I have attached example code. Can anyone advise me on how to get past
>> this block?
>>
>> --
>> Best wishes,
>> Dave Cardwell.
>>
>> http://davecardwell.co.uk/
>>
>
>

[ANNOUNCE] The next Bay Area Hive User Group Meetup, June 4th in SF

[Carl Steinbach]

The next Bay Area Hive User Group Meetup is happening on Tuesday, June 4th
at RichRelevance's offices in San Francisco. As usual the format will be a
series of short (15 min) talks followed by un-conference style sessions and
networking. Please join me in thanking the folks at RichRelevance for
providing the meeting space and refreshments.

In order to attend you must RSVP here:

http://www.meetup.com/Hive-User-Group-Meeting/events/118637862/


*Schedule*
6pm: Doors open
6pm-6:30pm: Networking and refreshments
6:30pm-9pm: Talks

*Talks:*

TBA

*Remote Viewing:*

We plan to do a live screencast of the meetup. More details will be posted
in the following weeks.

Re: [VOTE] Apache Hive 0.11.0 Release Candidate 1

[Carl Steinbach]

I took a quick look. Here's a list of things I noticed:

* NOTICE
** Copyright is out of date.

* README.txt
** Add one more '=' to the title underline.
** Requirements section only lists Hadoop 0.20.x
** Will it work with Java 1.7?

* RELEASE_NOTES.txt
** The 0.11 section lists tickets that aren't marked as resolved/fixed on
JIRA, and which are not included in the git commit log (e.g. HIVE-3979).
This may also extend to the 0.10 release notes. Related question: How were
these release notes generated?

* docs/index.html
** Update email list links (e.g. hive-u...@hadoop.apache.org)

* src/build.properties
** year=2012

* src/eclipse-files/.classpath
** The guava jar is not in the right location. This breaks the eclipse
import.

Thanks.

Carl



On Tue, Apr 30, 2013 at 5:18 PM, Ashutosh Chauhan wrote:

> Hey all,
>
> Based on feedback from folks, I have respun release candidate, RC1.
> Please take a look. It basically fixes the size bloat of tarball.
>
> Source tag for RC1 is at:
>
> https://svn.apache.org/repos/asf/hive/tags/release-0.11.0-rc1
>
>
> Source tar ball and convenience binary artifacts can be found
> at:http://people.apache.org/~hashutosh/hive-0.11.0-rc1/
>
> Maven artifacts for hive are available
> at:https://repository.apache.org/content/repositories/orgapachehive-158/
>
> Maven artifacts for hcatalog are available at:
>
> https://repository.apache.org/content/repositories/orgapachehcatalog-159/
>
>
> This release has many goodies including HiveServer2, integrated
> hcatalog, windowing and analytical functions, decimal data type,
> better query planning, performance enhancements and various bug fixes.
> In total, we resolved more than 350 issues. Full list of fixed issues
> can be found at:  http://s.apache.org/8Fr
>
>
> Voting will conclude in 72 hours.
>
> Hive PMC Members: Please test and vote.
>
> Thanks,
>
> Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
>

Re: HIVE-3979 in Hive 0.11

[Carl Steinbach]

Hi John,

This is a mistake in the release notes. It will be fixed in the next 0.11
release candidate.

Thanks.

Carl


On Sat, May 4, 2013 at 6:18 AM, John Omernik  wrote:

> I see in the release notes for HIVE -3979
>
> [HIVE-3979 ] - Provide
> syntax for unescaped regex on rlike, and other regexp_* functions
>
> Yet when I click on that JIRA there are not notes etc.  Could it be that
> this was included by mistake?  I am curious, if this isn't a mistake, how
> this was implemented.
>
>
> Thanks!
>
> John
>

Re: [VOTE] Apache Hive 0.11.0 Release Candidate 0

[Carl Steinbach]

I think the source tarball must be corrupted. It's 664MB in size, which is
roughly 630MB larger than the 0.10.0 release tarball. I haven't been able
to take a look at it yet because the apache archive site keeps throttling
my connection midway through the download.



On Mon, Apr 29, 2013 at 10:42 PM, Ashutosh Chauhan wrote:

> Hey all,
>
> I am excited to announce availability of Apache Hive 0.11.0 Release
> Candidate 0 at:
> http://people.apache.org/~hashutosh/hive-0.11.0-rc0/
>
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-154/
>
> This release has many goodies including HiveServer2, windowing and
> analytical functions, decimal data type, better query planning,
> performance enhancements and various bug fixes. In total, we resolved
> more than 350 issues. Full list of fixed issues can be found at:
> http://s.apache.org/8Fr
>
>
> Voting will conclude in 72 hours.
>
> Hive PMC Members: Please test and vote.
>
> Thanks,
>
> Ashutosh (On behalf of Hive contributors who made 0.11 a possibility)
>

Re: Compiling Hive ODBC

[Carl Steinbach]

Hi Sebastien,

Most (if not all) of the ant targets will fail if run from the submodule
directories. You must run the targets from the root source directory
instead.

Thanks.

Carl


On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH  wrote:

> I have now installed Thrift, but I still get the error with the "osfamily"
> task:
>
> Please help!
>
> hive@orca:~/hive-0.10.0/src/**odbc$ ant compile-cpp
> -Dthrift.home=/home/hive/**thrift-0.9.0
>
>
> Buildfile: /home/hive/hive-0.10.0/src/**odbc/build.xml
>
> BUILD FAILED
> /home/hive/hive-0.10.0/src/**odbc/build.xml:30: The following error
> occurred while executing this line:
> /home/hive/hive-0.10.0/src/**build-common.xml:117: Problem: failed to
> create task or type osfamily
> Cause: The name is undefined.
> Action: Check the spelling.
> Action: Check that any custom tasks/types have been declared.
> Action: Check that any / declarations have taken
> place.
>
>
> Total time: 0 seconds
>
> Seb
>
>
> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>
>> Go it, need to install Apache Thrift first ... sorry for this mail.
>> Seb
>>
>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>
>>> Hi all,
>>>
>>> Just started with an Hadoop / Hive POC to write a
>>>
>>> So far, I have installed:
>>>
>>> So far I have downloaded the Hadoop and Hive packages from:
>>>
>>> http://hadoop.apache.org/**releases.html
>>> http://hive.apache.org/**releases.html
>>>
>>> I took:
>>>
>>> http://hadoop.apache.org/**releases.html#18+April%2C+**
>>> 2013%3A+Release+0.23.7+**available
>>>
>>>
>>>
>>> and
>>>
>>> http://hive.apache.org/**releases.html#11+January%2C+**
>>> 2013%3A+release+0.10.0+**available
>>>
>>>
>>>
>>>
>>> After setting the env, it seems that hive is working:
>>>
>>> hive@orca:~$ $HIVE_HOME/bin/hive
>>> ...
>>> hive> create table t1 ( k int, s string );
>>> OK
>>>
>>> hive> select count(*) from t1;
>>> Total MapReduce jobs = 1
>>> ...
>>> OK
>>> 0
>>> Time taken: 7.634 seconds
>>>
>>>
>>> No I want to connect through ODBC...
>>>
>>> Following the instructions of this page:
>>>
>>> https://cwiki.apache.org/**confluence/display/Hive/**HiveODBC
>>>
>>> Section "Hive Client Build/Setup", I tried:
>>>
>>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-**0.10.0/odbc
>>> Buildfile: build.xml does not exist!
>>> Build failed
>>>
>>> Then, from the sources directory src/odbc:
>>>
>>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>>> hive@orca:~/hive-0.10.0/src/**odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-**0.10.0/odbc
>>> Buildfile: /home/hive/hive-0.10.0/src/**odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/**odbc/build.xml:30: The following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/**build-common.xml:117: Problem: failed to
>>> create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any / declarations have taken
>>> place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>>
>>> Can someone help or point me to an up to date documentation?
>>>
>>> Thanks!
>>> Seb
>>>
>>>
>>
>

Re: wiki-editing permission

[Carl Steinbach]

Done.


On Fri, Apr 26, 2013 at 6:23 AM, Scott Preece  wrote:

>  I would like permission to edit the Hive wiki, so I can fix problems I
> see as I find them.
>
>  My Apache username is preece.
>
>  Thanks,
> scott
>

Re: What's the URL of Hive IRC Channel

[Carl Steinbach]

I think you probably want this:

https://webchat.freenode.net/

Thanks.

Carl


On Mon, Apr 22, 2013 at 10:23 PM, YouPeng Yang wrote:

> Hi
>Does anyone know the exact URL of the Hive IRC Channel.
>
>I try http://irc.freenode.net/#hive,and it does not works.
>
>
> Regards
>

[ANNOUNCE] April Hive User Group Meetup

[Carl Steinbach]

The April Bay Area Hive User Group Meetup is happening on
Thursday, April 25th at Hortonworks in Palo Alto. The format will
be a series of short (15 min) talks preceded by refreshments and
networking. Please join me in thanking the folks at Hortonworks
for providing the meeting space and refreshments.

Important: You must RSVP on the Meetup page in order to attend.

http://meetu.ps/B9X6T


Schedule

* 5pm: Doors Open
* 5pm-6pm: Networking and Refreshments
* 6pm-8pm: Presentations


Talks:

* Hive and HCatalog -- Travis Crawford
* The State of Security in Hive -- Carl Steinbach
* New Features in Hive -- Ashutosh Chauhan
* Making Your Hive Queries Run Faster -- Gunther Hagleitner
* TBD

[ANNOUNCE] Kevin Wilfong elected to Hive PMC

[Carl Steinbach]

On behalf of the Apache Hive PMC I am pleased to welcome
Kevin Wilfong as a member of the Apache Hive PMC.

Please join me in congratulating Kevin on his new role!

Thanks.

Carl

Re: [REMINDER] February Hive Meetup is Tomorrow

[Carl Steinbach]

Several people have emailed me requesting a copy of Kixeye's NDA.
The NDA is available for review here:

http://www.scribd.com/doc/125005674/Kixeye-NDA

Thanks.

Carl

On Mon, Feb 11, 2013 at 10:20 AM, Carl Steinbach wrote:

> RSVP here: http://www.meetup.com/Hive-User-Group-Meeting/events/99774232/
>
>
> On Mon, Feb 11, 2013 at 9:57 AM, Carl Steinbach  wrote:
>
>> The next Bay Area Hive User Group Meetup is tomorrow, Tuesday, February
>> 12th at the offices of KIXEYE in San Francisco. The format will be a series
>> of short (15 min) talks followed by un-conference style sessions and
>> networking. Please join me in thanking the folks at KIXEYE for providing
>> the meeting space and refreshments.
>>
>> IMPORTANT: You must present a valid photo ID and sign Kixeye's NDA in
>> orded to gain admission to the event.
>>
>>
>> Schedule
>>
>> 6pm: Doors open
>>
>> 6-7pm: Networking and refreshments
>>
>> 7-9pm: Talks
>>
>>
>>
>> Talks
>>
>> Hive at KIXEYE and in the Gaming Industry -- Ben Speakmon and Aaron Sun
>>
>> Brickhouse: Klout's Open Source UDF Library for Hive -- Jerome Banks
>>
>> New Features in Hive 0.10.0 -- Ashutosh Chauhan
>>
>> Hive Client/Server Deployment Options -- Prasad Mujumdar
>>
>> Case Study: Utilizing Windowing and Partitioned Table Functions with Hive
>> -- Murtaza Doctor
>>
>
>

Re: [REMINDER] February Hive Meetup is Tomorrow

[Carl Steinbach]

RSVP here: http://www.meetup.com/Hive-User-Group-Meeting/events/99774232/

On Mon, Feb 11, 2013 at 9:57 AM, Carl Steinbach  wrote:

> The next Bay Area Hive User Group Meetup is tomorrow, Tuesday, February
> 12th at the offices of KIXEYE in San Francisco. The format will be a series
> of short (15 min) talks followed by un-conference style sessions and
> networking. Please join me in thanking the folks at KIXEYE for providing
> the meeting space and refreshments.
>
> IMPORTANT: You must present a valid photo ID and sign Kixeye's NDA in
> orded to gain admission to the event.
>
>
> Schedule
>
> 6pm: Doors open
>
> 6-7pm: Networking and refreshments
>
> 7-9pm: Talks
>
>
>
> Talks
>
> Hive at KIXEYE and in the Gaming Industry -- Ben Speakmon and Aaron Sun
>
> Brickhouse: Klout's Open Source UDF Library for Hive -- Jerome Banks
>
> New Features in Hive 0.10.0 -- Ashutosh Chauhan
>
> Hive Client/Server Deployment Options -- Prasad Mujumdar
>
> Case Study: Utilizing Windowing and Partitioned Table Functions with Hive
> -- Murtaza Doctor
>

[REMINDER] February Hive Meetup is Tomorrow

[Carl Steinbach]

The next Bay Area Hive User Group Meetup is tomorrow, Tuesday, February
12th at the offices of KIXEYE in San Francisco. The format will be a series
of short (15 min) talks followed by un-conference style sessions and
networking. Please join me in thanking the folks at KIXEYE for providing
the meeting space and refreshments.

IMPORTANT: You must present a valid photo ID and sign Kixeye's NDA in orded
to gain admission to the event.


Schedule

6pm: Doors open

6-7pm: Networking and refreshments

7-9pm: Talks



Talks

Hive at KIXEYE and in the Gaming Industry -- Ben Speakmon and Aaron Sun

Brickhouse: Klout's Open Source UDF Library for Hive -- Jerome Banks

New Features in Hive 0.10.0 -- Ashutosh Chauhan

Hive Client/Server Deployment Options -- Prasad Mujumdar

Case Study: Utilizing Windowing and Partitioned Table Functions with Hive
-- Murtaza Doctor

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

[Carl Steinbach]

The following active Hive PMC members have cast votes:

Carl Steinbach: +1, +1
Ashutosh Chauhan: +1, +1
Edward Capriolo: +1, +1
Ashish Thusoo: +1, +1
Yongqiang He: +1, +1
Namit Jain: +1, +1

Three active PMC members have abstained from voting.

Over the last week the following four Hive PMC members requested
that their status be changed from active to emeritus member:
jvs, prasadc, zhao, pauly.

Voting on these measures is now closed. Both measures have been approved
with the required 2/3 majority of active Hive PMC members.

Thanks.

Carl

On Thu, Jan 31, 2013 at 2:04 PM, Vinod Kumar Vavilapalli <
vino...@hortonworks.com> wrote:

>
> +1 and +1 non-binding.
>
> Great to see this happen!
>
> Thanks,
> +Vinod
>
>
> On Thu, Jan 31, 2013 at 12:14 AM, Namit Jain  wrote:
>
>> +1 and +1
>>
>> On 1/30/13 6:53 AM, "Gunther Hagleitner" 
>> wrote:
>>
>> >+1 and +1
>> >
>> >Thanks,
>> >Gunther.
>> >
>> >
>> >On Tue, Jan 29, 2013 at 5:18 PM, Edward Capriolo
>> >wrote:
>> >
>> >> Measure 1: +1
>> >> Measure 2: +1
>> >>
>> >> On Mon, Jan 28, 2013 at 2:47 PM, Carl Steinbach 
>> wrote:
>> >>
>> >> > I am calling a vote on the following two measures.
>> >> >
>> >> > Measure 1: Amend Hive Bylaws to Define Submodules and Submodule
>> >> Committers
>> >> >
>> >> > If this measure passes the Apache Hive Project Bylaws will be
>> >> > amended with the following changes:
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive
>> >>+Bylaws+for+Submodule+Committers
>> >> >
>> >> > The motivation for these changes is discussed in the following
>> >> > email thread which appeared on the hive-dev and hcatalog-dev
>> >> > mailing lists:
>> >> >
>> >> > http://markmail.org/thread/u5nap7ghvyo7euqa
>> >> >
>> >> >
>> >> > Measure 2: Create HCatalog Submodule and Adopt HCatalog Codebase
>> >> >
>> >> > This measure provides for 1) the establishment of an HCatalog
>> >> > submodule in the Apache Hive Project, 2) the adoption of the
>> >> > Apache HCatalog codebase into the Hive HCatalog submodule, and
>> >> > 3) adding all currently active HCatalog committers as submodule
>> >> > committers on the Hive HCatalog submodule.
>> >> >
>> >> > Passage of this measure depends on the passage of Measure 1.
>> >> >
>> >> >
>> >> > Voting:
>> >> >
>> >> > Both measures require +1 votes from 2/3 of active Hive PMC
>> >> > members in order to pass. All participants in the Hive project
>> >> > are encouraged to vote on these measures, but only votes from
>> >> > active Hive PMC members are binding. The voting period
>> >> > commences immediately and shall last a minimum of six days.
>> >> >
>> >> > Voting is carried out by replying to this email thread. You must
>> >> > indicate which measure you are voting on in order for your vote
>> >> > to be counted.
>> >> >
>> >> > More details about the voting process can be found in the Apache
>> >> > Hive Project Bylaws:
>> >> >
>> >> > https://cwiki.apache.org/confluence/display/Hive/Bylaws
>> >> >
>> >> >
>> >>
>>
>>
>
>
> --
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>

Re: Thrift Hive client for CDH 4.1 HiveServer2?

[Carl Steinbach]

Hi,

The Thrift IDL file for HiveServer2 is located here:

service/if/cli_service.thrift

Here's what you need to do in order to build the HiveServer2 Perl Thrift
client using the Hive tarball from CDH4.1.x:

# Set THRIFT_HOME to point to the installation directory of Thrift 0.9.0
% export THRIFT_HOME=<...>

# Verify that THRIFT_HOME is set properly
% $THRIFT_HOME/bin/thrift -version
Thrift Version 0.9.0

# Download and unpack the hive-0.9.0-cdh4.1.2 tarball
% wget http://archive.cloudera.com/cdh4/cdh/4/hive-0.9.0-cdh4.1.2.tar.gz
% tar -xvf hive-0.9.0-cdh4.1.2.tar.gz
% cd hive-0.9.0-cdh4.1.2/src

# Generate the Perl Thrift libraries
% ant thriftif -Dthrift.home=$THRIFT_HOME -Dthrift.args="-I $THRIFT_HOME
--gen perl"

The previous command will result in the generation of the HiveServer2 Perl
library here:

service/src/gen/thrift/gen-perl/TCLIService.pm

In order to use this library you can start with the code on stackoverflow
and modify
it to reference TCLIService instead of ThriftHiveClient. Also note that the
TCLIService
Thrift service exposes a different set of functions than the
ThriftHiveClient service.
I recommend looking at the comments in TCLIService.thrift in order to
understand
the proper workflow.

Thanks.

Carl



On Sat, Jan 5, 2013 at 3:00 AM, David Morel  wrote:

> So that would probably be generated
> using src/service/if/cli_service.thrift instead of the older
> hive_service.thrift which i suppose is for hiveserver1.
> Compiled it, still getting errors that seem transport-related
>
> 13/01/04 23:02:22 ERROR server.TThreadPoolServer: Error occurred during
> processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
>  at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
> ...
>
> This is a bit tedious...
>
> D.Morel
>
>
> On Sat, Jan 5, 2013 at 10:27 AM, Jov  wrote:
>
>> here：
>> https://issues.apache.org/jira/browse/HIVE-2935
>> https://cwiki.apache.org/Hive/hiveserver2-thrift-api.html
>> HiveServer2 now is CDH extension.
>>
>> I think you can use find cmd to search the CDH src dir to find the
>> .thrift files.
>>
>>
>> 2013/1/5 David Morel 
>>
>>> On 4 Jan 2013, at 16:04, Jov wrote:
>>>
>>> they are in the src/service/if and src/metastore/if
>>>
>>> Cool. But these would be files for HiveServer, not HiveServer2 which has
>>> a different API, right? After finally generating the libs, it turns out
>>> they work fine on the old-style hive server, but produce this in
>>> hiveserver2's log: 13/01/04 20:09:11 ERROR server.TThreadPoolServer: Error
>>> occurred during processing of message. java.lang.RuntimeException:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
>>> at
>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:170)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:662) Caused by:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at
>>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:124)
>>> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:40)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:215)
>>> ... 4 more Where should I start looking (meaning I haven't a clue)? Thanks!
>>> David
>>>
>>> 在 2013-1-4 上午7:16，"David Morel" 写道：
>>>
>>> Hi all (and happy New Year!) Is it possible to build a perl Thrift
>>> client for HiveServer2 (from Cloudera's 4.1.x) ? I'm following the
>>> instructions found here:
>>> http://stackoverflow.com/questions/5289164/perl-thrift-client-to-hiveDownloaded
>>>  Hive from Cloudera's site, then i'm a bit lost: where do I find
>>> these thrift files that I need to build the perl libs? I have the thrift
>>> compiler working ok, but thats as far as I got.
>>>
>>>
>>
>>
>> --
>> jov
>> blog: http:amutu.com/blog
>>
>
>

Re: [VOTE] Amend Hive Bylaws + Add HCatalog Submodule

[Carl Steinbach]

Measure 1: +1 (binding)
Measure 2: +1 (binding)

On Mon, Jan 28, 2013 at 11:47 AM, Carl Steinbach  wrote:

> I am calling a vote on the following two measures.
>
> Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers
>
> If this measure passes the Apache Hive Project Bylaws will be
> amended with the following changes:
>
>
> https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Bylaws+for+Submodule+Committers
>
> The motivation for these changes is discussed in the following
> email thread which appeared on the hive-dev and hcatalog-dev
> mailing lists:
>
> http://markmail.org/thread/u5nap7ghvyo7euqa
>
>
> Measure 2: Create HCatalog Submodule and Adopt HCatalog Codebase
>
> This measure provides for 1) the establishment of an HCatalog
> submodule in the Apache Hive Project, 2) the adoption of the
> Apache HCatalog codebase into the Hive HCatalog submodule, and
> 3) adding all currently active HCatalog committers as submodule
> committers on the Hive HCatalog submodule.
>
> Passage of this measure depends on the passage of Measure 1.
>
>
> Voting:
>
> Both measures require +1 votes from 2/3 of active Hive PMC
> members in order to pass. All participants in the Hive project
> are encouraged to vote on these measures, but only votes from
> active Hive PMC members are binding. The voting period
> commences immediately and shall last a minimum of six days.
>
> Voting is carried out by replying to this email thread. You must
> indicate which measure you are voting on in order for your vote
> to be counted.
>
> More details about the voting process can be found in the Apache
> Hive Project Bylaws:
>
> https://cwiki.apache.org/confluence/display/Hive/Bylaws
>
>

[VOTE] Amend Hive Bylaws + Add HCatalog Submodule

[Carl Steinbach]

I am calling a vote on the following two measures.

Measure 1: Amend Hive Bylaws to Define Submodules and Submodule Committers

If this measure passes the Apache Hive Project Bylaws will be
amended with the following changes:

https://cwiki.apache.org/confluence/display/Hive/Proposed+Changes+to+Hive+Bylaws+for+Submodule+Committers

The motivation for these changes is discussed in the following
email thread which appeared on the hive-dev and hcatalog-dev
mailing lists:

http://markmail.org/thread/u5nap7ghvyo7euqa


Measure 2: Create HCatalog Submodule and Adopt HCatalog Codebase

This measure provides for 1) the establishment of an HCatalog
submodule in the Apache Hive Project, 2) the adoption of the
Apache HCatalog codebase into the Hive HCatalog submodule, and
3) adding all currently active HCatalog committers as submodule
committers on the Hive HCatalog submodule.

Passage of this measure depends on the passage of Measure 1.


Voting:

Both measures require +1 votes from 2/3 of active Hive PMC
members in order to pass. All participants in the Hive project
are encouraged to vote on these measures, but only votes from
active Hive PMC members are binding. The voting period
commences immediately and shall last a minimum of six days.

Voting is carried out by replying to this email thread. You must
indicate which measure you are voting on in order for your vote
to be counted.

More details about the voting process can be found in the Apache
Hive Project Bylaws:

https://cwiki.apache.org/confluence/display/Hive/Bylaws

Re:

[Carl Steinbach]

Hi Dean,

I think using a Linux VM is the current path of least resistance for
Windows users who want to experiment with Hive and Hadoop. Cloudera has a
CentOS VM with CDH4.1 pre-installed and configured that can be downloaded
for free from here:

https://ccp.cloudera.com/display/SUPPORT/Cloudera%27s+Hadoop+Demo+VM+for+CDH4

More info here:
http://www.zdnet.com/hadoop-on-your-pc-clouderas-cdh4-virtual-machine-703096/

Thanks.

Carl

On Wed, Nov 14, 2012 at 6:41 AM, Dean Wampler <
dean.wamp...@thinkbiganalytics.com> wrote:

> Reading the Hortonworks page, it looks like only Windows server
> configurations are supported. While that's what you want for production
> deployments, of course, I have many, many clients would like like to run
> Hadoop and Hive on personal computers for development and testing, like you
> can do on Mac and Linux personal machines with the Apache distro. I hope
> Hortonworks and Microsoft will address this need.
>
> Also, I laughed out loud when I read this: "Microsoft is notorious for
> making things easy and simple ..."
>
> dean
>
> On Wed, Nov 14, 2012 at 2:01 AM, imen Megdiche wrote:
>
>> i find that there is WSO2 which works fine under windows,  but i still
>> want to run hive under cygwin. at the moment i can create tables with the
>> commande "hive - e 'sql query '".
>> HDP is free ??
>>
>>
>> 2012/11/14 Russell Jurney 
>>
>>> Hadoop and Hive now run natively on Windows via HDP. You do not need
>>> Cygwin. You can download this from Microsoft:
>>> http://hortonworks.com/partners/microsoft/
>>>
>>> We're working to get this open source code back into mainline Apache now.
>>>
>>>
>>> On Tue, Nov 13, 2012 at 5:57 AM, imen Megdiche 
>>> wrote:
>>>
 Hello,
 I can not find a solution to run hive under cygwin.
 Although hadoop works very well, the command hive starts  to turn as
 infinite
 Thank you in advance for your answers

>>>
>>>
>>>
>>> --
>>> Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome
>>> .com
>>>
>>
>>
>
>
> --
> *Dean Wampler, Ph.D.*
> thinkbiganalytics.com
> +1-312-339-1330
>
>
>

Re: Hive JIRA issue HIVE-3299 : Patch submitted

[Carl Steinbach]

Hi Namitha,

I left review comments on JIRA.

Thanks.

Carl

On Thu, Sep 13, 2012 at 10:04 PM, Namitha Babychan/TVM/TCS <
namitha.babyc...@tcs.com> wrote:

>
>
>
>
>
>
>
>
> Dear All,
>
>
> The patch for the issue HIVE-3299 has been submitted . Please review .
>
>
> Thanks and Regards,
>
>
> Namitha Babychan
> 533296
> NextGen Solutions - Kochi
> Ph:+914846618356,9446483467
> =-=-=
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>

Re: A question about HiveStatement.executeUpdate's exception

[Carl Steinbach]

Hi Ed,

Namit has requested that I add negative testcases for each of the
exceptions thrown by executeUpdate. I added a note to the phabricator
review requesting clarification on this point. I think we should probably
wait until Namit has had a chance to respond.

Thanks.

Carl

On Fri, Aug 31, 2012 at 8:56 AM, Edward Capriolo wrote:

> I will commit that one later tonight. Seems like it is no big deal.
>
> On Fri, Aug 31, 2012 at 11:50 AM, Mark Grover  wrote:
> > Hi Liyifan,
> > This seems like a bug. As a workaround, I would recommend using
> HiveStatement.executeQuery() instead (since they both call execute() on the
> Thrift client).
> >
> > There is a JIRA already for this (
> https://issues.apache.org/jira/browse/HIVE-1450) and seems like a patch
> was uploaded as well (https://reviews.facebook.net/D4293) but nothing has
> been committed just yet.
> >
> > I've asked for the status on the JIRA, let's see what they say:-)
> >
> > Mark
> >
> > - Original Message -
> > From: "Liyifan" 
> > To: user@hive.apache.org
> > Sent: Tuesday, August 28, 2012 9:47:37 PM
> > Subject: A question about HiveStatement.executeUpdate's exception
> >
> >
> >
> >
> > Hi, all. I'm new to Hive, I have a question about executeUpdate.
> >
> > I executed a SQL by invoking HiveStatement.executeUpdate method. I found
> that it throws an "Method not supported" exception.
> >
> > And I checked the code in
> hive-0.9\jdbc\src\java\org\apache\hadoop\hive\jdbc\HiveStatement.java, I
> found that the "Method not supported" is always threw unless
> client.execute(sql) throws an exceptioin.
> >
> >
> >
> > I'm confused about it, why executeUpdate always throws an exeption?
> >
> >
> >
> > public int executeUpdate(String sql) throws SQLException {
> > try {
> > client.execute(sql);
> > } catch (Exception ex) {
> > throw new SQLException(ex.toString());
> > }
> > throw new SQLException("Method not supported");
> > }
>

Re: HiveServer can not handle concurrent requests from more than one client?

[Carl Steinbach]

Hi Bertrand,

According to the proposal for HiveServer2, the current hive server provides
> no insurance about "session state in between calls".
> If that was all, it is something that can be lived with. It only means
> that for a JDBC client, all requests should be conceived as isolated.
>

In the HiveServer Thrift API Execute() and Fetch() are two separate calls
and require two separate RPCs. In between these calls HiveServer has to
maintain session state so that when the Fetch() call is made it knows which
result set to look at. The current HiveServer Thrift API assumes that
Thrift will consistently map the same physical connection to the same
Thrift worker thread, and consequently it stores the session state in a
thread local variable. Unfortunately, this assumption is false. It's
possible to live with this limitation if you're ok with sometimes fetching
other people's result sets instead of your own.


> The page of the Hive Server (1) says "HiveServer can not handle
> concurrent requests from more than one client."
> According to the jira, one may run into issues when multiples users are
> running it. Is that true regardless of the configuration?
> It should not be interpreted as "query will be executed one after the
> other", like Ranjiht said?
>

Yes, this is true regardless of the configuration. Ranjiht's statement is
incorrect.


> Eg what would be the impact of hive.exec.parallel or
> hive.support.concurrency?
>

These two configuration properties are actually completely orthogonal to
the HiveServer multi-client issue, though it's hard to know that since the
configuration property names were very poorly chosen. hive.exec.parallel
controls whether or not the the MR jobs in the query plan DAG are executed
in parallel on the cluster (https://issues.apache.org/jira/browse/HIVE-549).
hive.support.concurrency controls whether or not Hive supports
coarse-grained locks on tables and partitions (see
https://cwiki.apache.org/confluence/display/Hive/Locking).


> What would be the recommended way for providing a hive access to multiple
> users to a production environnement which is thightly fire walled? Ssh is
> not a viable solution in my context and the hive web interface does not
> seem mature enough.
>

I recommend taking a look at the Beeswax web interface for Hive. More
details (including screenshots) are available here:
https://ccp.cloudera.com/display/CDHDOC/Beeswax

Thanks.

Carl

Re: HiveServer can not handle concurrent requests from more than one client?

[Carl Steinbach]

HiveServer is multi-threaded, but there is a defect in the current
HiveServer Thrift API that prevents it from robustly handling concurrent
connections. This problem is described in more detail here:

https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Thrift+API

Thanks.

Carl

On Mon, Aug 27, 2012 at 9:03 AM, Raghunath, Ranjith <
ranjith.raghuna...@usaa.com> wrote:

>  Bertrand,
>
> ** **
>
> The Hive Server is a thrift service that provides an interface for Hive.
> You can connect to it using JDBC. It is not sure (out of box) as there is
> no userid and password restrictions. On the concurrency part, it is single
> threaded…….one query gets executed after the other.
>
> ** **
>
> Thanks,
>
> Ranjith
>
> ** **
>
> *From:* Bertrand Dechoux [mailto:decho...@gmail.com]
> *Sent:* Monday, August 27, 2012 11:01 AM
> *To:* user@hive.apache.org
> *Subject:* HiveServer can not handle concurrent requests from more than
> one client?
>
> ** **
>
> Hi,
>
> I would like to have more information about this specific sentence from
> the documentation.
> "HiveServer can not handle concurrent requests from more than one client."
> https://cwiki.apache.org/Hive/hiveserver.html
>
> Does it mean it is not possible with this server to provide a JDBC access
> to an 'almost closed' environment for multiple users?
>
> Regards
>
> Bertrand
>

Re: alter external table location with new namenode address

[Carl Steinbach]

Directly accessing the metastore schema is generally not a good idea.
Instead I recommend using the ALTER TABLE SET LOCATION command:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable%2FPartitionLocation

Thanks.

Carl

On Fri, Aug 24, 2012 at 10:56 AM, Alok Kumar  wrote:

> Bejoy,
>
> Thank you for your help.
> updated metastore n its working fine.
>
> Regards
> -Alok
>
> On Fri, Aug 24, 2012 at 5:40 PM, Bejoy KS  wrote:
> > Yes you need to update the metastore db directly for this to be in
> effect.
> >
> > Regards
> > Bejoy KS
> >
> > Sent from handheld, please excuse typos.
> >
> > -Original Message-
> > From: Alok Kumar 
> > Date: Fri, 24 Aug 2012 13:30:36
> > To: ; 
> > Reply-To: user@hive.apache.org
> > Subject: alter external table location with new namenode address
> >
> > Hello,
> >
> > We have hive external table mapped to hbase, now moving
> > from pseudo distributed to fully distributed hadoop cluster.
> >
> > found that hive queries are still pointing to older namenode address
> > ie: hdfs://localhost:9000/user/hive/warehouse/ as it stores
> > full uri in its derby metastore.
> >
> > Q . what would be the best way to set new namenode address?
> >
> >  Alter table command on hive shell to set hdfs location is only
> > possible for native tables!!
> > should i update derby "DB_LOCATION_URI" column directly?
> >
> > Your help/suggestion is highly appreciated
> >
> > Thanks,
> > Alok
> >
> >
>

Re: [ANNOUNCE] New Hive Committer - Navis Ryu

[Carl Steinbach]

Congratulations Navis! This is very well deserved. Looking forward to many
more patches from you.

On Fri, Aug 10, 2012 at 8:10 AM, Bejoy KS  wrote:

> Congrats Navis.. :)
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
>
> -Original Message-
> From: alo alt 
> Date: Fri, 10 Aug 2012 17:08:07
> To: 
> Reply-To: user@hive.apache.org
> Cc: ; 
> Subject: Re: [ANNOUNCE] New Hive Committer - Navis Ryu
>
> Congratulations! Well done :)
>
> cheers,
>  ALex
>
> On Aug 10, 2012, at 11:58 AM, John Sichi  wrote:
>
> > The Apache Hive PMC has passed a vote to make Navis Ryu a new
> > committer on the project.
> >
> > JIRA is currently down, so I can't send out a link with his
> > contribution list at the moment, but if you have an account at
> > reviews.facebook.net, you can see his activity here:
> >
> > https://reviews.facebook.net/p/navis/
> >
> > Navis, please submit your CLA to the Apache Software Foundation as
> > described here:
> >
> > http://www.apache.org/licenses/#clas
> >
> > Congratulations!
> > JVS
>
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>

Re: Request write access to the Hive wiki

[Carl Steinbach]

Granted! :)

On Thu, Jul 19, 2012 at 12:56 AM, Lefty Leverenz wrote:

> Please grant me write access to the Hive wiki so that I can work on
> improving the documentation.
>
> Thank you.
>
> – Lefty Leverenz
>le...@hortonworks.com
>
>
>

Re: Migrating to hive 8.1 on EMR

[Carl Steinbach]

Hi Ranjan,

Looks like the NPE is getting generated here:

if (0 == getCols().size()) {
  throw new HiveException(
  "at least one column must be specified for the table");
}

Which would seem to indicate that the table record which was fetched
doesn't have any columns.

Did you run the metastore upgrade scripts for 0.8.0 (located in
metastore/scripts/upgrade)? Hive 0.8.0 includes HIVE-2246 which changed the
name and schema of the metastore table used to store column information.

Thanks.

Carl



On Mon, Jun 18, 2012 at 12:56 PM, Ranjan Bagchi wrote:

> Hi,
>
> I've built a datastore using Hive 7.1 backed by S3 using persistent
> metadata.  Now that hive 8.1 is available, I'd like to migrate to the new
> version.
>
> However, I'm having trouble reading tables with the persistent schema.
>  Looking in the logs, I'm getting stack traces like the following:
> 2012-06-18 19:51:23,176 ERROR org.apache.hadoop.hive.ql.exec.Task
> (SessionState.
> java:printError(380)) - Failed with exception null
> java.lang.NullPointerException
>at
> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162
> )
>at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:946)
>at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:860)
>at
> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:252
> 7)
>at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:298)
>at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
>at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
> va:57)
>at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
>at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
>at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261)
>at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409)
>at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
>at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:567)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> 2012-06-18 19:51:23,180 ERROR org.apache.hadoop.hive.ql.Driver
> (SessionState.java:printError(380)) - FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask
>
> Does anyone have experience with this?
>
> Thanks,
>
> Ranjan
>
>

Hadoop Summit Hive Meetup presentations now available online

[Carl Steinbach]

Hi,

Slides from the talks which were presented at the Hadoop Summit Hive meetup
are now available:

https://cwiki.apache.org/confluence/display/Hive/Presentations

I'd like to thank all of the speakers for making this such a great event,
as well as the Hadoop Summit organizers for giving us a place to meet!

Carl

Re: Block Sampling

[Carl Steinbach]

Done!

On Fri, Jun 15, 2012 at 12:26 PM, Ladda, Anand wrote:

>  Thanks Carl. Could you give me edit rights to the wiki (
> ala...@microstrategy.com) to update the sampling page with this info
>
> ** **
>
> *From:* Carl Steinbach [mailto:c...@cloudera.com]
> *Sent:* Friday, June 15, 2012 3:20 PM
> *To:* user@hive.apache.org
> *Subject:* Re: Block Sampling
>
> ** **
>
> Hi Anand,
>
> ** **
>
> This feature was implemented in HIVE-2121 and appeared in Hive 0.8.0.
>
> ** **
>
> Ref: https://issues.apache.org/jira/browse/HIVE-2121
>
> ** **
>
> Thanks.
>
> ** **
>
> Carl
>
> On Fri, Jun 15, 2012 at 11:59 AM, Ladda, Anand 
> wrote:
>
> Has the block sampling feature been added to one of the latest (Hive 0.8
> or Hive 0.9) releases. The wiki has the blurb below on block sampling
>
> *Block Sampling*
>
> It is a feature that is still on trunk and is not yet in any release
> version.
>
> block_sample: TABLESAMPLE (n PERCENT)
>
> This will allow Hive to pick up at least n% data size (notice it doesn't
> necessarily mean number of rows) as inputs. Only CombineHiveInputFormat is
> supported and some special compression formats are not handled. If we fail
> to sample it, the input of MapReduce job will be the whole table/partition.
> We do it in HDFS block level so that the sampling granularity is block
> size. For example, if block size is 256MB, even if n% of input size is only
> 100MB, you get 256MB of data.
>
> In the following example the input size 0.1% or more will be used for the
> query.
>
> SELECT * 
>
> FROM source TABLESAMPLE(0.1 PERCENT) s; 
>
> Sometimes you want to sample the same data with different blocks, you can
> change this seed number:
>
> set hive.sample.seednumber=;
>
>  
>
> ** **
>

Re: Block Sampling

[Carl Steinbach]

Hi Anand,

This feature was implemented in HIVE-2121 and appeared in Hive 0.8.0.

Ref: https://issues.apache.org/jira/browse/HIVE-2121

Thanks.

Carl

On Fri, Jun 15, 2012 at 11:59 AM, Ladda, Anand wrote:

>  Has the block sampling feature been added to one of the latest (Hive 0.8
> or Hive 0.9) releases. The wiki has the blurb below on block sampling
>
> *Block Sampling*
>
> It is a feature that is still on trunk and is not yet in any release
> version.
>
> block_sample: TABLESAMPLE (n PERCENT)
>
> This will allow Hive to pick up at least n% data size (notice it doesn't
> necessarily mean number of rows) as inputs. Only CombineHiveInputFormat is
> supported and some special compression formats are not handled. If we fail
> to sample it, the input of MapReduce job will be the whole table/partition.
> We do it in HDFS block level so that the sampling granularity is block
> size. For example, if block size is 256MB, even if n% of input size is only
> 100MB, you get 256MB of data.
>
> In the following example the input size 0.1% or more will be used for the
> query.
>
> SELECT * ** **
>
> FROM source TABLESAMPLE(0.1 PERCENT) s; 
>
> Sometimes you want to sample the same data with different blocks, you can
> change this seed number:
>
> set hive.sample.seednumber=;
>
> ** **
>

Re: Edit Rights to Hive Wiki

[Carl Steinbach]

Hi Anand,

Please create a wiki account and send me your user id.

Thanks.

Carl

On Tue, May 29, 2012 at 7:33 AM, Ladda, Anand wrote:

>  Can someone grant me edit rights to the Hive Wiki?
>
> Thanks
>
> Anand
>

Re: Want to give a short talk at the next Hive User Group meetup?

[Carl Steinbach]

Hi Ed,

Sounds good. Please send me a copy of your slides and I'll find someone to
present them (or do it myself).

Thanks.

Carl

On Wed, May 23, 2012 at 7:17 AM, Edward Capriolo wrote:

> I can give you a PPT on the upcoming "programming hive" book that
> someone else can run thought. but I wont be able to present unless you
> want to fly me out :)
>
> Edward
>
> On Tue, May 22, 2012 at 9:28 PM, Carl Steinbach  wrote:
> > Hi,
> >
> > I just wanted to remind everyone that the next Hive User Group meetup is
> > happening on June 12th (the day before the Hadoop Summit) in San Jose.
> More
> > details about the meetup can be found on the Hive User Group page located
> > here:
> >
> > http://www.meetup.com/Hive-User-Group-Meeting/events/62458462/
> >
> > I also wanted to remind everyone that I'm looking for speakers for this
> > event. Our plan is to have people give short 15 minute talks on topics
> that
> > are relevant to the Hive community, and at this point I have still have a
> > couple slots left to fill. Please send me an email with your proposed
> topic
> > if you're interested in speaking.
> >
> > Thanks.
> >
> > Carl
>

Want to give a short talk at the next Hive User Group meetup?

[Carl Steinbach]

Hi,

I just wanted to remind everyone that the next Hive User Group meetup is
happening on June 12th (the day before the Hadoop Summit) in San Jose. More
details about the meetup can be found on the Hive User Group page located
here:

http://www.meetup.com/Hive-User-Group-Meeting/events/62458462/

I also wanted to remind everyone that I'm looking for speakers for this
event. Our plan is to have people give short 15 minute talks on topics that
are relevant to the Hive community, and at this point I have still have a
couple slots left to fill. Please send me an email with your proposed topic
if you're interested in speaking.

Thanks.

Carl

Re: Edit access to Wiki

[Carl Steinbach]

Hi Lars,

I added you to the ACL.

Thanks.

Carl

On Thu, May 17, 2012 at 5:03 PM, Lars Francke wrote:

> *bump*
>
> Can anyone here grant me access please?
>
> Thanks,
> Lars
>

Re: Multiple SerDe per table name

[Carl Steinbach]

In Hive 0.8 and later versions you have the ability to specify the SerDe on
a per-partition basis, but that's as granular as it gets.

Thanks.

Carl

On Thu, May 17, 2012 at 8:53 AM, kulkarni.swar...@gmail.com <
kulkarni.swar...@gmail.com> wrote:

> I was thinking more from a perspective of specifying a SerDe per column
> name.
>
>
> On Thu, May 17, 2012 at 10:38 AM, Mark Grover  wrote:
>
>> Hi Swarnim,
>> What's your use case?
>> If you use multiple SerDe's, when you are writing to the table, how would
>> you want Hive to decide which one to use?
>>
>> Mark
>>
>> Mark Grover, Business Intelligence Analyst
>> OANDA Corporation
>>
>> www: oanda.com www: fxtrade.com
>>
>> - Original Message -
>> From: "kulkarni swarnim" 
>> To: user@hive.apache.org
>> Sent: Thursday, May 17, 2012 11:29:26 AM
>> Subject: Multiple SerDe per table name
>>
>> Does hive currently support multiple SerDe s to be defined per table
>> name? Looking through the code and documentation, it seems like it doesn't
>> as only one could be specified through the ROW FORMAT SERDE but just wanted
>> to be sure.
>>
>>
>> --
>> Swarnim
>>
>
>
>
> --
> Swarnim
>

Re: Indexing in hive

[Carl Steinbach]

Hi Ranjith,

Hive 0.7 supports the ability to build indexes, but the query compiler in
0.7 doesn't know how to optimize queries with these indexes. Hive 0.8 was
the first release to include some support for optimizing query plans with
indexes, and that only applies to GROUP BY and WHERE clauses under certain
specific circumstances. At this point index support in Hive is very much a
work in progress.

Thanks.

Carl

On Wed, May 16, 2012 at 1:06 PM, Raghunath, Ranjith <
ranjith.raghuna...@usaa.com> wrote:

>  I am currently using hive 0.7.1 and creating indexes based on columns in
> the where clause. However, when I run the explain plan I do not see the
> index being leveraged. The syntax that I am using to build the index is as
> follows:
>
> CREATE INDEX x ON TABLE t(j)
> AS 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler'
> WITH DEFERRED REBUILD
>
> The query is as follows:
>
> Select a,count(*) from x where j=’and’ group by a
>
> Is there anything I am doing incorrectly?
>
> Thanks,
> Ranjith
>
>
>
>

Re: Is there a metastore schema script for postgresql for Hive version 0.9.0

[Carl Steinbach]

Hi Xiabo,

Upgrade/schema scripts for Postgres were committed to trunk last week in
HIVE-2529 (https://issues.apache.org/jira/browse/HIVE-2529). This patch
includes schema DDL for Hive 0.8.0 and 0.9.0, as well as an 0.7.0->0.8.0
upgrade script. Right now these scripts are only available in the source
code repository.

Thanks.

Carl

On Sat, May 12, 2012 at 12:58 PM, Ashutosh Chauhan wrote:

> Hi Xiaobo,
>
> It depends on what your current setup is:
> a) If you are trying Hive for the first time, then you don't necessarily
> need to create schema in postgres upfront. As wd suggested, datanucleus
> (ORM used by Hive) can autocreate the schema if it doesn't exist.
> b) If you are already using Hive 0.8 then there is no upgrade step, there
> are no metastore schema changes between 0.8 & 0.9, so your existing Hive
> 0.8 postgres will continue to work with Hive 0.9
> c) If you are currently using Hive 0.7 or earlier, then you do need to
> migrate. Unfortunately, with Hive 0.8 we didn't publish upgrade script for
> 0.7 to 0.8. But, good news is support for Postgres is reinstated in trunk.
> You can find Postgres upgrade scripts here:
> http://svn.apache.org/repos/asf/hive/trunk/metastore/scripts/upgrade/postgres/
>
> Hope it helps,
> Ashutosh
>
>
> On Fri, May 11, 2012 at 11:28 PM, Xiaobo Gu wrote:
>
>> **
>> I can't find it in the release package.
>>
>> --
>> Xiaobo Gu
>>
>
>

Re: requesting write access to hive confluence wiki

[Carl Steinbach]

Hi Thejas,

I granted you write access. Please email me if you encounter any problems.

Thanks.

Carl

On Mon, Apr 30, 2012 at 10:44 AM, Thejas Nair wrote:

> Hi,
> I would like to update/fix some sections in the hive confluence wiki.
> Please grant write access. My user name is thejas.
>
> Thanks,
> Thejas
>

Re: Getting CNF during map reduce execution

[Carl Steinbach]

HI Buddhika,

Which version of Hive are you running?

Thanks.

Carl

On Thu, Apr 5, 2012 at 10:12 AM, buddhika chamith
wrote:

> Hi All,
>
> I am trying to use hive inside an OSGi enviornment. So far I faced several
> OSGi issues in hive and some issues related to JDO implementation used
> (datanucleaus) under the OSGi enviornemnt and was able to overcome them.
> However when I run a query involving a map reduce operation I am getting a
> ClassNotFound exception from hadoop side. (Hadoop is running in pseudo
> distributed mode in the local machine).
>
> java.lang.RuntimeException: java.lang.RuntimeException: 
> java.lang.ClassNotFoundException: 
> org.apache.hadoop.hive.shims.HadoopShimsSecure$NullOutputCommitter
>   at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:898)
>   at org.apache.hadoop.mapred.JobConf.getOutputCommitter(JobConf.java:613)
>   at org.apache.hadoop.mapred.Task.initialize(Task.java:492)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
>
>
> I think this is probably due to some jar/ resource not being bundled
> properly in to the deployed archive to Hadoop. Would some one be able to
> explain a bit of how the job submission part from Hive to Hadoop works and
> any possible reason for this behaviour? Any pointers to look in to the code
> is also appreciated.
>
> Regards
> Buddhika
>

Re: INSERT INTO Overwriting Data

[Carl Steinbach]

Hi Steve,

It's possible that Hive is still picking up some old JARs on its classpath.
Can you please try dumping the classpath from within the HiveCLI and
respond with the output, e.g.

% hive -e 'set system:java.class.path'

Thanks.

Carl

On Sat, Feb 25, 2012 at 4:42 PM, Steve Severance
wrote:

> Hi,
>
> I have a job that use INSERT INTO from Hive 0.8. The query is something
> like:
>
> INSERT INTO TABLE MyTable PARTITION (dt='2012-01-01) SELECT ...
>
> When I run this query both with and without dynamic partitioning it
> overwrites the data that was already in the partition.
>
> I am running Hive 0.8 on CDH3u2. I have confirmed that append is enabled
> in the job conf.
>
> Is there anything else I need to do that would enable appending to my hive
> table?
>
> Thanks.
>
> Steve
>

Re: Can hive 0.8.1 work with hadoop 0.23.0?

[Carl Steinbach]

Hi Xiaofeng,

Which mode are you running Hadoop in, e.g. local, pseudo-distributed, or
distributed?

Thanks.

Carl

2012/2/1 张晓峰 

> Hi,
>
> ** **
>
> I installed hadoop 0.23.0 which can work.
>
> The version of my hive is 0.8.1. The query like ‘select * from tablename’
> can work. But an exception is thrown when executing query like ‘select col1
> form tablename’.
>
> ** **
>
> 2012-02-01 16:32:20,296 WARN  mapreduce.JobSubmitter
> (JobSubmitter.java:copyAndConfigureFiles(139)) - Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for the same.
> 
>
> 2012-02-01 16:32:20,389 INFO  mapreduce.JobSubmitter
> (JobSubmitter.java:submitJobInternal(388)) - Cleaning up the staging area
> file:/tmp/hadoop-hadoop/mapred/staging/hadoop-469936305/.staging/job_local_0001
> 
>
> 2012-02-01 16:32:20,392 ERROR exec.ExecDriver
> (SessionState.java:printError(380)) - Job Submission failed with exception
> 'java.io.FileNotFoundException(File does not exist:
> /home/hadoop/hive-0.8.1/lib/hive-builtins-0.8.1.jar)'
>
> java.io.FileNotFoundException: File does not exist:
> /home/hadoop/hive-0.8.1/lib/hive-builtins-0.8.1.jar
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:764)
> 
>
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:208)
> 
>
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:71)
> 
>
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:246)
> 
>
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:284)
> 
>
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:355)
> 
>
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1159)
>
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1156)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:396)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
> 
>
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1156)
>
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:571)
> 
>
> at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:452)
>
> at
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:710)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 
>
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
>
> ** **
>
> Thanks,
>
> xiaofeng
>
> ** **
>

Re: hive -e '' not working in 0.8.0

[Carl Steinbach]

Hi Sam,

I tried running this against the hive-0.8.0 release tarball and did not
encounter any problems.
Is it possible that you have some old JARs or shell scripts hanging around
from a previous Hive install?

On Thu, Feb 16, 2012 at 11:58 AM, Sam William  wrote:

> We recently upgraded to Hive-0.8.0  and noticed that all queries   fail
> when run as  hive -e   with the error ,
>
>
> sampd@face:~$ hive -e "show tables;"
>
> FAILED: Parse Error: line 1:0 cannot recognize input near 'show' ''
> '' in ddl statement
>
>
> hive -f option works though .  Has anyone else faced this ?
>
> Sam William
> sa...@stumbleupon.com
>
>
>
>

Re: Building HiveODBC driver

[Carl Steinbach]

Hi Sam,

Hive 0.8.0 and trunk both depend on Thrift 0.7.0 (consult
ivy/libraries.properties for other dependency version numbers).

Thanks.

Carl

On Thu, Feb 9, 2012 at 6:01 PM, Sam William  wrote:

> I was able to get across this .
>
> The solution is to use  thrift-0.6.0  with the following patch
>
> https://issues.apache.org/jira/browse/THRIFT-1060
>
>
> Sam
>
>
> On Feb 8, 2012, at 5:42 PM, Sam William wrote:
>
> Hi,
> Im trying to build the HiveODBC driver.  The hive source code base I'm
> using is 0.8.0.   Im following the instructions from
> https://cwiki.apache.org/Hive/hiveodbc.html.   Basically , I had to build
> thrift/thrift-fb303 on my own  and  Im runningthe ant build  command
>  as
>
> *ant compile-cpp -Dthrift.home= *
> *
> *
> But I get the error
>
>
>
>   [exec] In file included from
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:6:
>  [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:9084:
> warning: direct base ‘apache::thrift::TProcessor’ inaccessible in
> ‘Apache::Hadoop::Hive::ThriftHiveMetastoreProcessor’ due to ambiguity
>  [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:
> In member function ‘virtual bool
> Apache::Hadoop::Hive::ThriftHiveMetastoreProcessor::process_fn(apache::thrift::protocol::TProtocol*,
> apache::thrift::protocol::TProtocol*, std::string&, int32_t, void*)’:
>  [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:18554:
> error: ‘process_fn’ is not a member of
> ‘facebook::fb303::FacebookServiceProcessor’
>  [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:
> In member function ‘void
> Apache::Hadoop::Hive::ThriftHiveMetastoreProcessor::process_create_database(int32_t,
> apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*,
> void*)’:
>  [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:18563:
> error: reference to ‘eventHandler_’ is ambiguous
>  [exec]
> /home/sampd/dev/thrift_install/include/thrift/TProcessor.h:161: error:
> candidates are: boost::shared_ptr
> apache::thrift::TProcessor::eventHandler_
>  [exec]
> /home/sampd/dev/thrift_install/include/thrift/TProcessor.h:161: error:
> boost::shared_ptr
> apache::thrift::TProcessor::eventHandler_ [exec]
> /home/sampd/git/hive/hive_su/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:18563:
> error: reference to ‘eventHandler_’ is ambiguous
>
> 
> ...
>
>
>
>  I tried with a couple of versions of Thrift , 0.9.0-dev and 0.5.0  .
>  Neither of them proved to be good.Has it got to do with the thrift
> library version ?  Whats the fix.Any help is appreciated.
>
>
>
> *
> *
> *
> *
>
> Thanks,
>
> Sam William
> sa...@stumbleupon.com
>
>
>
>
> Sam William
> sa...@stumbleupon.com
>
>
>
>

[ANNOUNCE] Apache Hive 0.8.1 Released

[Carl Steinbach]

The Apache Hive team is proud to announce the the release of Apache
Hive version 0.8.1.

The Apache Hive (TM) data warehouse software facilitates querying and
managing large datasets residing in distributed storage. Built on top
of Apache Hadoop (TM), it provides:

* Tools to enable easy data extract/transform/load (ETL)

* A mechanism to impose structure on a variety of data formats

* Access to files stored either directly in Apache HDFS (TM) or in other
  data storage systems such as Apache HBase (TM)

* Query execution via MapReduce

For Hive release details and downloads, please
visit:http://hive.apache.org/releases.html

Hive 0.8.1 Release Notes are available here:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843&version=12319268

We would like to thank the many contributors who made this release
possible.

Regards,

The Apache Hive Team

Re: Wiki Write Access

[Carl Steinbach]

Hi Aniket,

I added you to the ACL. Thanks for your help with the wiki.

Carl

On Tue, Jan 24, 2012 at 11:26 PM, Aniket Mokashi wrote:

> Hi Carl,
>
> It would be helpful for me too.
> My wiki username: aniket486.
>
> Thanks,
> Aniket
>
>
> On Tue, Jan 24, 2012 at 11:57 AM, Carl Steinbach wrote:
>
>> Hi Matt,
>>
>> Great!
>>
>> Please sign up for a wiki account here:
>> https://cwiki.apache.org/confluence/signup.action
>>
>> Then email me your wiki username and I will add you to the Hive wiki ACL.
>>
>> Thanks.
>>
>> Carl
>>
>>
>> On Tue, Jan 24, 2012 at 7:10 AM, Tucker, Matt wrote:
>>
>>> Hi,
>>>
>>> ** **
>>>
>>> I would like to get write access to the Hive wiki, so that I can add
>>> documentation on existing UDFs.
>>>
>>> ** **
>>>
>>> Thanks
>>>
>>> ** **
>>>
>>> Matt Tucker
>>>
>>> Associate eBusiness Analyst
>>>
>>> Walt Disney Parks and Resorts Online
>>>
>>> Ph: 407-566-2545
>>>
>>> Tie: 8-296-2545
>>>
>>> ** **
>>>
>>
>>
>
>
> --
> "...:::Aniket:::... Quetzalco@tl"
>

Re: Hive FAQ

[Carl Steinbach]

Hi Nicolas,

I added you to the ACL.

Thanks.

Carl

2012/1/25 Nicolas Lalevée 

>
> Le 24 janv. 2012 à 22:48, Carl Steinbach a écrit :
>
> > Hi Nicolas,
> >
> > Thanks for pointing this out. If you would like to provide answers I can
> give you edit access to the wiki. Please create an account and send me your
> username.
>
> It feels a little strange that someone write all these questions in a FAQ
> without writing any answer. Why bother writing them at all ? At least it
> made me laugh :)
>
> But never mind, I would be glad to help, my id is 'hibou'. I can at least
> answer one of them.
>
> cheers,
> Nicolas
>
> >
> > Thanks.
> >
> > Carl
> >
> > 2012/1/24 Nicolas Lalevée 
> > It seems to be there:
> > https://cwiki.apache.org/confluence/display/Hive/User+FAQ
> >
> > But humm..how should I phrase it. shouldn't questions be
> followed by answers ? :D
> >
> > Nicolas
> >
> >
>
>

Re: Hive FAQ

[Carl Steinbach]

Hi Nicolas,

Thanks for pointing this out. If you would like to provide answers I can
give you edit access to the wiki. Please create an account and send me your
username.

Thanks.

Carl

2012/1/24 Nicolas Lalevée 

> It seems to be there:
> https://cwiki.apache.org/confluence/display/Hive/User+FAQ
>
> But humm..how should I phrase it. shouldn't questions be
> followed by answers ? :D
>
> Nicolas
>
>

Re: Wiki Write Access

[Carl Steinbach]

Hi Matt,

Great!

Please sign up for a wiki account here:
https://cwiki.apache.org/confluence/signup.action

Then email me your wiki username and I will add you to the Hive wiki ACL.

Thanks.

Carl

On Tue, Jan 24, 2012 at 7:10 AM, Tucker, Matt wrote:

> Hi,
>
> ** **
>
> I would like to get write access to the Hive wiki, so that I can add
> documentation on existing UDFs.
>
> ** **
>
> Thanks
>
> ** **
>
> Matt Tucker
>
> Associate eBusiness Analyst
>
> Walt Disney Parks and Resorts Online
>
> Ph: 407-566-2545
>
> Tie: 8-296-2545
>
> ** **
>

Re: error on left/right join, hive 0.8.0

[Carl Steinbach]

Hi Roberto,

I'm not sure which version of Hive you're using. If you're talking about
the version of Hive
that comes with Cloudera's distribution then it can't be version 0.8.0
because we
have not yet included that version in CDH.

On Thu, Jan 19, 2012 at 5:01 PM, Roberto Congiu wrote:

> Hey guys,
> we found an issue that looks like a bug (hive 0.8 , caldera's
> distribution).
>
> SELECT count(1)
>   FROM table1 a LEFT OUTER JOIN table2 b
>   ON ( a.key1 = b.key1 AND a.key2 = b.KEY2)
>
> fails with java.lang.IllegalArgumentException: Can not create a Path
> from an empty string ( full stack trace at the bottom of this email).
>
> which stems from this code
> (org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:379))
>
> public static String getHiveJobID(Configuration job) {
>String planPath = HiveConf.getVar(job, HiveConf.ConfVars.PLAN);
>if (planPath != null) {
>  return (new Path(planPath)).getName();
>}
>return null;
>  }
>
>
> Querying the individual tables works fine, as works fine the normal
> inner join (same query removing 'left outer').
> RIGHT JOIN and FULL OUTER JOIN don't work either.
>
> Has anybody else had this issue ?
>
> Thanks,
> Roberto
>
>
> Full stack trace:
> java.lang.InstantiationException:
> org.apache.hadoop.hive.ql.io.HiveOutputFormat
>at java.lang.Class.newInstance0(Class.java:340)
>at java.lang.Class.newInstance(Class.java:308)
>at
> org.apache.hadoop.hive.ql.exec.ExecDriver.addInputPath(ExecDriver.java:859)
>at
> org.apache.hadoop.hive.ql.exec.ExecDriver.addInputPaths(ExecDriver.java:903)
>at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:426)
>at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
>at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
>at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
>at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
>at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
>at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:338)
>at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:436)
>at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:446)
>at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:642)
>at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Job Submission failed with exception
>
> 'java.lang.InstantiationException(org.apache.hadoop.hive.ql.io.HiveOutputFormat)'
> java.lang.IllegalArgumentException: Can not create a Path from an empty
> string
>at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>at org.apache.hadoop.fs.Path.(Path.java:90)
>at
> org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:379)
>at
> org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:192)
>at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:476)
>at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
>at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
>at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
>at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
>at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
>at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
>at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:338)
>at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:436)
>at
> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:446)
>at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:642)
>at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at
> sun.reflect.N

[ANNOUNCE] Apache Hive 0.8.0 Released

[Carl Steinbach]

The Apache Hive team is proud to announce the the release of Apache
Hive version 0.8.0.

The Apache Hive (TM) data warehouse software facilitates querying and
managing large datasets residing in distributed storage. Built on top
of Apache Hadoop (TM), it provides:

* Tools to enable easy data extract/transform/load (ETL)

* A mechanism to impose structure on a variety of data formats

* Access to files stored either directly in Apache HDFS (TM) or in other
  data storage systems such as Apache HBase (TM)

* Query execution via MapReduce

For Hive release details and downloads, please visit:
http://hive.apache.org/releases.html

Hive 0.8.0 Release Notes are available here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310843&version=12316178

We would like to thank the many contributors who made this release
possible.

Regards,

The Apache Hive Team

Re: Hive Metadata URI error

[Carl Steinbach]

Hi Periya,

You should only set the hive.metastore.uris property if you are running a
standalone MetaStore server, in which case you need to set
hive.metastore.local=false and set hive.metastore.uris to a Thrift URI.
Please see this document for more details:

https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin

Thanks.

Carl

On Sun, Dec 11, 2011 at 10:36 PM, Kirk True  wrote:

>  To me it looks like the error message is getting a blank for the URI
> property value. Can you triple-check the property _name_ is correct
> (including capitalization)?
>
>
> On 12/11/11 9:35 PM, Periya.Data wrote:
>
> Sam: I added "file://". Now it looks like this:
> file:///home/users/jtv/CDH3/hive/conf/metastore_db
>
> The problem has not gone away. I still have the same problem. I tried
> rebooting my ec-2 instance. Still no difference.
>
> What does it mean by "does not have a scheme". What is it expecting?
>
> Thanks,
> PD.
>
> On Sun, Dec 11, 2011 at 9:15 PM, Sam Wilson  wrote:
>
>>  Try file:// in front of the property value...
>>
>> Sent from my iPhone
>>
>> On Dec 12, 2011, at 12:07 AM, "Periya.Data" 
>> wrote:
>>
>>  Hi,
>>I am trying to create Hive tables on an EC2 instance. I get this
>> strange error about URI schema and log4j properties not found. I do not
>> know how to fix this.
>>
>> On EC2 instance : Ubuntu 10.04, Hive-0.7.1-cdh3u2.
>>
>> Initially I did not have an entry for hive.metastore.uris property in my
>> hive-default.xml file. So, I created one.  Still, I get the errors as
>> pasted below. I was under the assumption that if we leave the uris value
>> blank, it is will assume the local metastore.
>>
>> 
>>   hive.metastore.local> name>
>>   true
>>   controls whether to connect to remove metastore server or
>> open a new metastore server in Hive Client JVM
>> 
>>
>> 
>>   hive.metastore.uris
>>   /home/users/jtv/CDH3/hive/conf/metastore_db
>> 
>>
>>
>> root@ip-10-114-18-63:/home/users/jtv# hive -f ./scripts/log25.q
>> hive-log4j.properties not found
>> Hive history file=/tmp/root/hive_job_log_root_201112120332_1795396613.txt
>> 11/12/12 03:32:03 INFO exec.HiveHistory: Hive history
>> file=/tmp/root/hive_job_log_root_201112120332_1795396613.txt
>> 11/12/12 03:32:03 INFO parse.ParseDriver: Parsing command: CREATE TABLE
>> log25_tbl (OperationEvent STRING, HostIP STRING, StartTime STRING,
>> SourceRepo STRING, SourceFolder STRING, DestRepo STRING, DestFolder STRING,
>> EntityOrObject STRING, BytesSent STRING, TotalTimeInSecs STRING) COMMENT
>> 'This is the Log_25 Table'
>> 11/12/12 03:32:04 INFO parse.ParseDriver: Parse Completed
>> 11/12/12 03:32:04 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
>> 11/12/12 03:32:04 INFO parse.SemanticAnalyzer: Creating table log25_tbl
>> position=13
>> 11/12/12 03:32:04 INFO ql.Driver: Semantic Analysis Completed
>> 11/12/12 03:32:04 INFO ql.Driver: Returning Hive schema:
>> Schema(fieldSchemas:null, properties:null)
>> 11/12/12 03:32:04 INFO ql.Driver: Starting command: CREATE TABLE
>> log25_tbl (OperationEvent STRING, HostIP STRING, StartTime STRING,
>> SourceRepo STRING, SourceFolder STRING, DestRepo STRING, DestFolder STRING,
>> EntityOrObject STRING, BytesSent STRING, TotalTimeInSecs STRING) COMMENT
>> 'This is the Log_25 Table'
>> 11/12/12 03:32:04 INFO exec.DDLTask: Default to LazySimpleSerDe for table
>> log25_tbl
>> 11/12/12 03:32:04 INFO hive.log: DDL: struct log25_tbl { string
>> operationevent, string hostip, string starttime, string sourcerepo, string
>> sourcefolder, string destrepo, string destfolder, string entityorobject,
>> string bytessent, string totaltimeinsecs}
>> FAILED: Error in metadata: java.lang.IllegalArgumentException: URI:  does
>> not have a scheme
>> 11/12/12 03:32:04 ERROR exec.DDLTask: FAILED: Error in metadata:
>> java.lang.IllegalArgumentException: URI:  does not have a scheme
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> java.lang.IllegalArgumentException: URI:  does not have a scheme
>> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:476)
>> at
>> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3176)
>> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:213)
>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
>> at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
>> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:209)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:286)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:310)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:317)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:490)
>> at sun.reflect.N

Re: Hive ODBC driver - Cloudera not contributing

[Carl Steinbach]

Hi Li,

The Cloudera ODBC driver for Apache Hive is free to use, and anyone can
download it from the Cloudera support site:
https://ccp.cloudera.com/display/SUPPORT/Downloads


> Why Cloudera didn't contribute to open source?
>

Actually, Cloudera has contributed a lot to open source and will continue
to do so in the future. Over the past 18 months we have open sourced Apache
Sqoop, Apache Flume, Crunch, Recordbreaker, Hue, Apache Bigtop, and Apache
Whirr, and our engineers have contributed hundreds of patches to Apache
Hadoop, HBase, Hive, etc. Meanwhile, we make projects like the ODBC driver
and SCM Express available for free for everyone to use.

> But in this case there was already Apache open source code for Hive ODBC
driver and Cloudera chose not to contribute. Why?

The Apache open source code for the Hive ODBC driver depends on the
unixODBC driver toolkit, and consequently it inherits unixODBC's GPL
license. This makes it incompatible with the ASF license. Working around
this licensing problem was our primary motivation for rewriting the driver
from scratch. Had this not been the case we would not have done a closed
source implementation, and would have instead worked to improve the
original version of the driver.

Thanks.

Carl

Re: Organizing a Hive Meetup for Hadoop World NYC

[Carl Steinbach]

Yes, there's going to be a Hive Meetup in NYC on Wednesday, November 9th.
More details can be found on the Meetup page located here:

http://www.meetup.com/Hive-User-Group-Meeting/events/36439952/

Space is limited, so please RSVP if you plan to attend the event.

Thanks.

Carl

On Wed, Oct 12, 2011 at 10:31 AM, Chalcy Raja  wrote:

>  There is a sqoop user group meet on the 7th which I am planning to
> attend.  I am really interested in hive group meet up.  I have the same
> question as Bennie and if yes when?
>
> ** **
>
> Thank you,
>
> Chalcy
>
> ** **
>
> *From:* Bennie Schut [mailto:bsc...@ebuddy.com]
> *Sent:* Wednesday, October 12, 2011 9:58 AM
> *To:* user@hive.apache.org
> *Subject:* Re: Organizing a Hive Meetup for Hadoop World NYC
>
> ** **
>
> I'll be at hadoop world. Is the hive meetup still happening?
>
> On 08/29/2011 10:03 PM, Carl Steinbach wrote: 
>
> Hi Ed, 
>
> ** **
>
> This is a one-time event targeted at Hadoop World attendees, though
>
> others are welcome to attend as well.
>
> ** **
>
> Thanks.
>
> ** **
>
> Carl
>
> ** **
>
> On Mon, Aug 29, 2011 at 12:09 PM, Edward Capriolo 
> wrote:
>
> Carl,
>
> ** **
>
> Do you mean a one time Hive meetup or do you mean a recurring one?
>
> ** **
>
> I ask because the hadoop-nyc meetup is slowing down alot.
> http://www.meetup.com/Hadoop-NYC/. So supporting a hadoop and specific
> hive meetup seem difficult.
>
> ** **
>
> Edward
>
> ** **
>
> On Mon, Aug 29, 2011 at 2:36 PM, Carl Steinbach  wrote:
>
> 
>
> Dear Hive users,
>
> Hadoop World 2011 (http://hadoopworld.com/) will be held November 8th
> and 9th in NYC. This year we're also planning to organize a Hive Meetup.
> These events are a good place for users to interact with each other
> and with the Hive development team.
>
> In order to help with organization, I set up a form with a few
> questions about what kind of meetup the community wants, and which
> evening is best:
>
> ** **
>
>
> https://docs.google.com/spreadsheet/viewform?formkey=dENBelpZaDc3X1gxbmpFem01MzJPT0E6MQ
> 
>
> ** **
>
> Please fill this out, and feel free to contact me directly if you have
> any questions.
>
> Thanks!
>
> ** **
>
> Carl
>
> ** **
>
> ** **
>
> ** **
>

Hive User Group Meeting, NYC Edition

[Carl Steinbach]

 The next Hive User Group Meeting is happening the last night of the
Hadoop World Conference, Wednesday, November 9th, in New York
City. The format will be a series of short (15 min) talks followed by
un-conference style sessions and networking. The meetup is being
hosted by Palantir at their NYC office.

More information as well as updates about this event can be found on
the meetup page:

http://www.meetup.com/Hive-User-Group-Meeting/events/36439952/

An agenda will be posted as we near the event. If you have a topic you
would like to present please contact the meetup organizer, Carl
Steinbach (cws at apache org).


• 6:30p Arrival & Refreshments

• 6:45p Administrivia & Introductions

• 7:00p Talk 1

• 7:15p Talk 2

• 7:30p Talk 3

• 7:45p Talk 4

• 8:00p Breakouts

• 8:30p Take it to a local watering-hole

When:
Wednesday, November 9th @ 6:30pm

Where:
Palantir Offices
15 Little W. 12th Street
New York City, NY 10014

Re: urldecode hive column

[Carl Steinbach]

It's possible to call Java static methods directly using the built-in
'reflect' UDF. The performance will not be as good as writing your
own UDF, but you'll get results a lot faster.

On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja
wrote:

>  Thank you! Loren.  So far I have only using hive as is.  Looks like I
> have to start extending J
>
> ** **
>
> --Chalcy
>
> ** **
>
> *From:* Loren Siebert [mailto:lo...@siebert.org]
> *Sent:* Thursday, September 15, 2011 12:28 PM
> *To:* user@hive.apache.org
> *Subject:* Re: urldecode hive column
>
> ** **
>
> You need to write a UDF, like this person did:
>
> ** **
>
> http://search-hadoop.com/m/HFWE32CYs6x/v=plain
>
> ** **
>
> On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:
>
>
>
> 
>
> Hi,
>
>  
>
> I have a situation where I need to do urldecode on one particular column.
> Is there any hive built in function available?
>
>  
>
> Thank you,
>
> Chalcy
>
> ** **
>

Re: Organizing a Hive Meetup for Hadoop World NYC

[Carl Steinbach]

Hi Ed,

This is a one-time event targeted at Hadoop World attendees, though
others are welcome to attend as well.

Thanks.

Carl

On Mon, Aug 29, 2011 at 12:09 PM, Edward Capriolo wrote:

> Carl,
>
> Do you mean a one time Hive meetup or do you mean a recurring one?
>
> I ask because the hadoop-nyc meetup is slowing down alot.
> http://www.meetup.com/Hadoop-NYC/. So supporting a hadoop and specific
> hive meetup seem difficult.
>
> Edward
>
> On Mon, Aug 29, 2011 at 2:36 PM, Carl Steinbach  wrote:
>
>> Dear Hive users,
>>
>> Hadoop World 2011 (http://hadoopworld.com/) will be held November 8th
>> and 9th in NYC. This year we're also planning to organize a Hive Meetup.
>> These events are a good place for users to interact with each other
>> and with the Hive development team.
>>
>> In order to help with organization, I set up a form with a few
>> questions about what kind of meetup the community wants, and which
>> evening is best:
>>
>>
>> https://docs.google.com/spreadsheet/viewform?formkey=dENBelpZaDc3X1gxbmpFem01MzJPT0E6MQ
>>
>> Please fill this out, and feel free to contact me directly if you have
>> any questions.
>>
>> Thanks!
>>
>> Carl
>>
>
>

Organizing a Hive Meetup for Hadoop World NYC

[Carl Steinbach]

Dear Hive users,

Hadoop World 2011 (http://hadoopworld.com/) will be held November 8th
and 9th in NYC. This year we're also planning to organize a Hive Meetup.
These events are a good place for users to interact with each other
and with the Hive development team.

In order to help with organization, I set up a form with a few
questions about what kind of meetup the community wants, and which
evening is best:

https://docs.google.com/spreadsheet/viewform?formkey=dENBelpZaDc3X1gxbmpFem01MzJPT0E6MQ

Please fill this out, and feel free to contact me directly if you have
any questions.

Thanks!

Carl

Re: Hive DDL issue

[Carl Steinbach]

Adding to what Ed said, we don't run regression tests on Cygwin, so Hive on
Cygwin is
de facto unmaintained.

On Thu, Aug 18, 2011 at 12:37 PM, Edward Capriolo wrote:

> It did work with cygwin at one point but since it is rarely used in that
> environment it is not well supported. Your best bet is QEMU or Vmware
> emulating a linux environment.
>
>
> On Thu, Aug 18, 2011 at 3:14 PM, Siddharth Tiwari <
> siddharth.tiw...@live.com> wrote:
>
>>  hey carl,
>>
>> Isint there any way to enable it, if not, what is this error about ? what
>> is the problem ?
>>
>> ****
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>>
>>
>> --
>> From: c...@cloudera.com
>> Date: Thu, 18 Aug 2011 11:34:03 -0700
>> Subject: Re: Hive DDL issue
>> To: user@hive.apache.org
>>
>> Hive does not work on Cygwin.
>>
>> On Wed, Aug 17, 2011 at 3:38 PM, Siddharth Tiwari <
>> siddharth.tiw...@live.com> wrote:
>>
>>
>> encountering following issur pls help, on cygwin windows
>>
>> hive> show tables;
>> FAILED: Hive Internal Error:
>> java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative
>> path in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_
>> 04-08-25_850_5502285238716420526)
>> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
>> path in absolute URI:
>> file:C:/cygwin/tmp//siddharth/hive_2011-08-18_04-08-25_850_550228523871642
>> 0526
>> at org.apache.hadoop.fs.Path.initialize(Path.java:140)
>> at org.apache.hadoop.fs.Path.(Path.java:132)
>> at
>> org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142)
>> at
>> org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168)
>> at
>> org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282)
>> at
>> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205)
>> at
>> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
>> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
>> file:C:/cygwin/tmp//siddharth/hive_2011-08-18_04-08-25_850_5502285238716420526
>> at java.net.URI.checkPath(URI.java:1787)
>> at java.net.URI.(URI.java:735)
>> at org.apache.hadoop.fs.Path.initialize(Path.java:137)
>> ... 16 more
>>
>>
>> ****
>> *Cheers !!!*
>> *Siddharth Tiwari*
>> Have a refreshing day !!!
>>
>>
>>
>

Re: Hive DDL issue

[Carl Steinbach]

Hive does not work on Cygwin.

On Wed, Aug 17, 2011 at 3:38 PM, Siddharth Tiwari  wrote:

>
> encountering following issur pls help, on cygwin windows
>
> hive> show tables;
> FAILED: Hive Internal Error:
> java.lang.IllegalArgumentException(java.net.URISyntaxException: Relative
> path in absolute URI: file:C:/cygwin/tmp//siddharth/hive_2011-08-18_
> 04-08-25_850_5502285238716420526)
> java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative
> path in absolute URI:
> file:C:/cygwin/tmp//siddharth/hive_2011-08-18_04-08-25_850_550228523871642
> 0526
> at org.apache.hadoop.fs.Path.initialize(Path.java:140)
> at org.apache.hadoop.fs.Path.(Path.java:132)
> at
> org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:142)
> at
> org.apache.hadoop.hive.ql.Context.getLocalScratchDir(Context.java:168)
> at
> org.apache.hadoop.hive.ql.Context.getLocalTmpFileURI(Context.java:282)
> at
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:205)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> file:C:/cygwin/tmp//siddharth/hive_2011-08-18_04-08-25_850_5502285238716420526
> at java.net.URI.checkPath(URI.java:1787)
> at java.net.URI.(URI.java:735)
> at org.apache.hadoop.fs.Path.initialize(Path.java:137)
> ... 16 more
>
>
> ****
> *Cheers !!!*
> *Siddharth Tiwari*
> Have a refreshing day !!!
>

Re: Hive crashing after an upgrade - issue with existing larger tables

[Carl Steinbach]

Hi,

The original CDH3U1 release of Hive contained a configuration bug which we
recently fixed in an update. You can get the update by refreshing your Hive
packages. Afterwards please verify that you are using the following Hive
package: hive-0.7.1+42.9

You can also fix the problem by modifying your hive-site.xml file to include
the following setting:

mapred.max.split.size=25600

Thanks.

Carl

On Thu, Aug 18, 2011 at 8:48 AM,  wrote:

> A small correction to my previous post. The CDH version is CDH u1 not u0
> Sorry for the confusion
>
> Regards
> Bejoy K S
> --
> *From: * Bejoy Ks 
> *Date: *Thu, 18 Aug 2011 05:51:58 -0700 (PDT)
> *To: *hive user group
> *ReplyTo: * user@hive.apache.org
> *Subject: *Hive crashing after an upgrade - issue with existing larger
> tables
>
> Hi Experts
> I was working on hive with larger volume data  with hive 0.7 .
> Recently my hive installation was upgraded to 0.7.1 . After the upgrade I'm
> having a lot of issues with queries that were already working fine with
> larger data. The queries that took seconds to return results is now taking
> hours, for most larger tables even the map reduce jobs are not getting
> triggered. Queries like Select * and describe are working fine since they
> don't involve any map reduce jobs. For the jobs that didn't even get
> triggered I got the following error from job tracker
>
> Job initialization failed: java.io.IOException: Split metadata size
> exceeded 1000.
> Aborting job job_201106061630_6993 at
> org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:48)
>
> at
> org.apache.hadoop.mapred.JobInProgress.createSplits(JobInProgress.java:807)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:701)
>
> at org.apache.hadoop.mapred.JobTracker.initJob(JobTracker.java:4013)
> at
> org.apache.hadoop.mapred.EagerTaskInitializationListener$InitJob.run(EagerTaskInitializationListener.java:79)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:619)
>
>
> Looks like some metadata issue. My cluster is on CDH3-u0 . Has anyone faced
> similar issues before. Please share your thoughts what could be the probable
> cause of the error.
>
> Thank You
>