Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Carl Steinbach
Hi Viral, Hive 0.5.0 and 0.6.0 use the same version of libthrift, so the problem is more likely related to some difference in the way 0.5.0 and 0.6.0 configure/initialize Thrift, or to some other issue related to the way the Thrift connection is managed on the client or server side (though it look

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Viral Bajaria
Carl, Do you think this issue was not there before 0.6 ? We run our thrift servers for hours and have never faced this issue. I don't think I have restarted any of my thrift servers for days. My hive wrapper does have logic to handle timeouts, it reconnects whenever it sees that the thrift connec

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Ayush Gupta
Thanks Carl, I'll check that. But, surely, I cant be the only one running Hive queries which last more than 10 minutes over a thrift client! The hive model is somewhat intended to work with large data sets and long running queries should be expected. I wonder why there is no discussion around this

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Carl Steinbach
I filed a JIRA ticket to track the task of making the Thrift socket timeout configurable: https://issues.apache.org/jira/browse/HIVE-2006 On Thu, Feb 24, 2011 at 11:21 PM, Carl Steinbach wrote: > Hi Ayush, > > I suspect you're running into Thrift's default socket timeout setting. I > recommend

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Carl Steinbach
Hi Ayush, I suspect you're running into Thrift's default socket timeout setting. I recommend checking out a copy of the Hive source code, and modifying the Thrift setup code in HiveServer.java to explicitly set the socket timeout on the TServerSocket, e.g. in HiveServer.main() change TServerTrans

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Ayush Gupta
On Fri, Feb 25, 2011 at 12:17 PM, Viral Bajaria wrote: > What do the logs of the thrift server say ?? If it does not give any > relevant information, I would enable DEBUG level logging on the console. the hiveserver is pretty quiet, the connection appears to be terminated silently. I'll up the lo

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Ayush Gupta
Yes, the hiveserver server was started and running before the client program was run. -ayush On Fri, Feb 25, 2011 at 12:14 PM, Adarsh Sharma wrote: > Did you start hiverserver service before running the client Program. > > > Cheers, Adarsh > > > > Ayush Gupta wrote: > > Probing this further rev

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Viral Bajaria
What do the logs of the thrift server say ?? If it does not give any relevant information, I would enable DEBUG level logging on the console. Also a point to remember is the single-threaded nature of the hive thrift server (atleast upto v0.5) But looking at the logs is what will be the first thin

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Adarsh Sharma
Did you start hiverserver service before running the client Program. Cheers, Adarsh Ayush Gupta wrote: Probing this further reveals that the connection is reset by the server in exactly 10 minutes every time. I'm running Hive 0.6. I do not see anything relevant at http://wiki.apache.org/ha

Re: Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Ayush Gupta
Probing this further reveals that the connection is reset by the server in exactly 10 minutes every time. I'm running Hive 0.6. I do not see anything relevant at http://wiki.apache.org/hadoop/Hive/AdminManual/Configuration but is there some configuration property which controls this? -ayush On F

Re: FW: Query Regarding HIVE-1535.

2011-02-24 Thread Carl Steinbach
Hi Mohit, The fix for HIVE-1535 did not include a testcase. See the discussion in the ticket for an explanation of why this was the case. The steps you outlined in your email seem to indicate that HIVE-1535 was not actually fixed, or that the problem was reintroduced later. Please file a JIRA tic

FW: Query Regarding HIVE-1535.

2011-02-24 Thread Mohit
*** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any

Thrift Java Client - TTransportException (SocketException: Connection reset)

2011-02-24 Thread Ayush Gupta
Hi! I'm having some trouble running queries from a java client against a remote Thrift Hive server. Its all setup and quicker queries do run through fine. But queries which run longer than about 10 minutes disconnect the client with a "TTransportException: Connection reset" exception.. The query c

RE: Specifying a double precision in HiveQL

2011-02-24 Thread Paul Yang
Hacky, but maybe something like select concat( cast(num as int), '.' , cast(abs(num)*100 as int) % 100) from (select 1.234 as num from src limit 1) a; ? -Original Message- From: Aurora Skarra-Gallagher [mailto:aur...@yahoo-inc.com] Sent: Thursday, February 24, 2011 11:31 AM To: user@hi

Specifying a double precision in HiveQL

2011-02-24 Thread Aurora Skarra-Gallagher
Hi, I have a Hive query that has a statement like this "(sum(itemcount) / count(item))". I want to specify only two digits of precision (i.e. 53.55). The result is stored inside of a string, not its own column, so I'd need to set the precision in the statement. Is this possible? Thanks, Aurora

Re: Altering partition fileformat

2011-02-24 Thread Edward Capriolo
On Thu, Feb 24, 2011 at 10:26 AM, charlie w wrote: > I am trying to query against a partitioned Hive table where the input > format of different partitions may be different.  I'd like to change > the partition file format, and reading the language manual at > http://wiki.apache.org/hadoop/Hive/Lan

Altering partition fileformat

2011-02-24 Thread charlie w
I am trying to query against a partitioned Hive table where the input format of different partitions may be different. I'd like to change the partition file format, and reading the language manual at http://wiki.apache.org/hadoop/Hive/LanguageManual, it seems to indicate that I should be able to d

Re: How to add hours/minutes to a timestamp column in Hive Query

2011-02-24 Thread Thiruvel Thirumoolan
You can use unix_timestamp(), do the math and convert the result to timestamp. something like from_unixtime(unix_timestamp(Arrival) + n). Use the proper units though. Will that not work for you? On Feb 24, 2011, at 7:57 PM, Bejoy Ks wrote: Hi Experts Could some one please help me out wit

Re: How to add hours/minutes to a timestamp column in Hive Query

2011-02-24 Thread Ajo Fod
What I normally do is use python in a map phase for this sort of stuff ... if a UDF is not available. Any scripting language would do the trick as well. -Ajo On Thu, Feb 24, 2011 at 6:27 AM, Bejoy Ks wrote: > Hi Experts > Could some one please help me out with this? Any similar situations

Re: Database/Schema , INTERVAL and SQL IN usages in Hive

2011-02-24 Thread Bejoy Ks
Yes Ajo time was the concern. I just got a chance though the in memory join implementation within hive and it is great. Sorry for the confusion :) From: Ajo Fod To: user@hive.apache.org Sent: Wed, February 23, 2011 8:01:07 PM Subject: Re: Database/Schema , INT

Re: How to add hours/minutes to a timestamp column in Hive Query

2011-02-24 Thread Bejoy Ks
Hi Experts Could some one please help me out with this? Any similar situations faced by any one in your real time hive queries? If I have to go ahead with an User Defined Function for this, then how could I implement and use the same in Hive? Does Hive have any plans/JIRAs in progress with

FW: Query Regarding HIVE-1535.

2011-02-24 Thread Mohit
*** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any

Re: metastore with mysql

2011-02-24 Thread hive1
Hello, thank you for your quick responses! Seems my root mysql user wasn't really 'root'. "GRANT ALL PRIVILEGES ..." with a new user got it running. however I don't understand it, because my root user has the same privileges as the new one... but whatever. Malte -- NEU: FreePhone - kostenlos