Re: Closing long running hive query forcefully
' select /* myid bla bla*/ x,y,z ' I can't run above command using cli nor hive -f, could you explain how to add comment in hive query? 2013/2/8 Edward Capriolo edlinuxg...@gmail.com That is a good way to do it. We do it with comment sometimes. select /* myid bla bla*/ x,y,z Edward On Thu, Feb 7, 2013 at 11:12 AM, Gabor Makrai makrai.l...@gmail.com wrote: Hi, We solved this problem in the following way (this is really not a simple solution): - start hive query in a different thread - we generated an unique id for the query and used the SET key=value; command (before the long query command) to give this unique id to the MR jobs related to the query - main thread waits for the completion or the cancellation sign (comes from the gui or anywhere else) - if cancellation sign comes, then we will kill the running query through the JobTracker - list all jobs and search the job which contains the previously configured unique id - kill that job - that will cause exception on the query thread It's a little bit ugly, but thrift doesn't support async communication and so statement.cancel() function can't be supported/implemented. Gabor On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar nitinpawar...@gmail.com wrote: hive waits till the hadoop job is completed. so unless you kill the job of jdbc connection is dropped I don't see any other way to reduce the load on application. best when you think its long enough, you will need to find a way to kill the hadoop job. That will automatically release the resources in the pipeline On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran abimar...@hsenidmobile.com wrote: Hi All, Are there anyways to close the long running hive query through hive-jdbc? Since when ever Hive hangs, my application also hang, So I want to close the hive connection forcefully after a certain time. -- Thanks, With Regards, Abimaran -- Nitin Pawar
Re: Closing long running hive query forcefully
Hi All, I have filed a feature enhancement bug on jira : https://issues.apache.org/jira/browse/HIVE-4017 On Thu, Feb 7, 2013 at 10:01 PM, Edward Capriolo edlinuxg...@gmail.comwrote: That is a good way to do it. We do it with comment sometimes. select /* myid bla bla*/ x,y,z Edward On Thu, Feb 7, 2013 at 11:12 AM, Gabor Makrai makrai.l...@gmail.com wrote: Hi, We solved this problem in the following way (this is really not a simple solution): - start hive query in a different thread - we generated an unique id for the query and used the SET key=value; command (before the long query command) to give this unique id to the MR jobs related to the query - main thread waits for the completion or the cancellation sign (comes from the gui or anywhere else) - if cancellation sign comes, then we will kill the running query through the JobTracker - list all jobs and search the job which contains the previously configured unique id - kill that job - that will cause exception on the query thread It's a little bit ugly, but thrift doesn't support async communication and so statement.cancel() function can't be supported/implemented. Gabor On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar nitinpawar...@gmail.com wrote: hive waits till the hadoop job is completed. so unless you kill the job of jdbc connection is dropped I don't see any other way to reduce the load on application. best when you think its long enough, you will need to find a way to kill the hadoop job. That will automatically release the resources in the pipeline On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran abimar...@hsenidmobile.com wrote: Hi All, Are there anyways to close the long running hive query through hive-jdbc? Since when ever Hive hangs, my application also hang, So I want to close the hive connection forcefully after a certain time. -- Thanks, With Regards, Abimaran -- Nitin Pawar -- Thanks, With Regards, Abimaran
Re: Closing long running hive query forcefully
hive waits till the hadoop job is completed. so unless you kill the job of jdbc connection is dropped I don't see any other way to reduce the load on application. best when you think its long enough, you will need to find a way to kill the hadoop job. That will automatically release the resources in the pipeline On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran abimar...@hsenidmobile.com wrote: Hi All, Are there anyways to close the long running hive query through hive-jdbc? Since when ever Hive hangs, my application also hang, So I want to close the hive connection forcefully after a certain time. -- Thanks, With Regards, Abimaran -- Nitin Pawar
Re: Closing long running hive query forcefully
Hi, We solved this problem in the following way (this is really not a simple solution): - start hive query in a different thread - we generated an unique id for the query and used the SET key=value; command (before the long query command) to give this unique id to the MR jobs related to the query - main thread waits for the completion or the cancellation sign (comes from the gui or anywhere else) - if cancellation sign comes, then we will kill the running query through the JobTracker - list all jobs and search the job which contains the previously configured unique id - kill that job - that will cause exception on the query thread It's a little bit ugly, but thrift doesn't support async communication and so statement.cancel() function can't be supported/implemented. Gabor On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar nitinpawar...@gmail.com wrote: hive waits till the hadoop job is completed. so unless you kill the job of jdbc connection is dropped I don't see any other way to reduce the load on application. best when you think its long enough, you will need to find a way to kill the hadoop job. That will automatically release the resources in the pipeline On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran abimar...@hsenidmobile.com wrote: Hi All, Are there anyways to close the long running hive query through hive-jdbc? Since when ever Hive hangs, my application also hang, So I want to close the hive connection forcefully after a certain time. -- Thanks, With Regards, Abimaran -- Nitin Pawar
Re: Closing long running hive query forcefully
That is a good way to do it. We do it with comment sometimes. select /* myid bla bla*/ x,y,z Edward On Thu, Feb 7, 2013 at 11:12 AM, Gabor Makrai makrai.l...@gmail.com wrote: Hi, We solved this problem in the following way (this is really not a simple solution): - start hive query in a different thread - we generated an unique id for the query and used the SET key=value; command (before the long query command) to give this unique id to the MR jobs related to the query - main thread waits for the completion or the cancellation sign (comes from the gui or anywhere else) - if cancellation sign comes, then we will kill the running query through the JobTracker - list all jobs and search the job which contains the previously configured unique id - kill that job - that will cause exception on the query thread It's a little bit ugly, but thrift doesn't support async communication and so statement.cancel() function can't be supported/implemented. Gabor On Thu, Feb 7, 2013 at 2:21 PM, Nitin Pawar nitinpawar...@gmail.com wrote: hive waits till the hadoop job is completed. so unless you kill the job of jdbc connection is dropped I don't see any other way to reduce the load on application. best when you think its long enough, you will need to find a way to kill the hadoop job. That will automatically release the resources in the pipeline On Thu, Feb 7, 2013 at 6:35 PM, Kugathasan Abimaran abimar...@hsenidmobile.com wrote: Hi All, Are there anyways to close the long running hive query through hive-jdbc? Since when ever Hive hangs, my application also hang, So I want to close the hive connection forcefully after a certain time. -- Thanks, With Regards, Abimaran -- Nitin Pawar
