Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109959
---

Ship it!


Please update testing done with latest patch.

- Amareshwari Sriramadasu


On Dec. 11, 2015, 12:55 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 11, 2015, 12:55 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/
---

(Updated Dec. 11, 2015, 12:55 p.m.)


Review request for lens.


Changes
---

Added persistence for opHandleToSessionMap.


Repository: lens


Description
---

OperationHandle-HiveSession mapping is maintained for all active operations. 
Now, hive session will only be closed when there are no active operations 
running on that session.


Diffs (updated)
-

  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
a84c679 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 ab5ada9 
  lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
f6693aa 

Diff: https://reviews.apache.org/r/39911/diff/


Testing
---

Yes.

1. Created session.
2. Fired a query.
3. Deleted the session before query finishes. 
4. Query keeps running and finishes eventually.


Thanks,

Deepak Barr

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Dec. 11, 2015, 7:11 a.m., Amareshwari Sriramadasu wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1295
> > 
> >
> > Should we not persist opHandleToSessionHandle map as well? Otherwise, 
> > orphaned sessions will be still orphaned upon lens server restart.

I agree. My bad.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109927
---


On Dec. 11, 2015, 12:55 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 11, 2015, 12:55 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Dec. 10, 2015, 4:41 a.m., Puneet Gupta wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 535
> > 
> >
> > If closeQuery() is not called before this, which happens for (result == 
> > null || !(result instanceof HiveInMemoryResultSet),  how will the oprphan 
> > session be closed? 
> > As of now closeQuery() is doing that also.

Looking at the execute method and its comment[assuming this is only called for 
executing explain/insert/set/delete/etc... queries which don't ask to fetch 
data.),I think this method is only called for queries that do not return 
result,so I assumed closeQuery() will be called.Looking at the code, I can see 
this method is called for  "add jar" and "explain" queries.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109474
---


On Dec. 2, 2015, 10:40 a.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 2, 2015, 10:40 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Dec. 10, 2015, 4:41 a.m., Puneet Gupta wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > lines 803-804
> > 
> >
> > Please log the closing of session .

Not required in latest diff. Session close on driver close is removed.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109474
---


On Dec. 10, 2015, 10:19 a.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 10, 2015, 10:19 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109927
---


Deepak, can you mark the review issues as fixed/dropped accordingly ?


lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 1284)


Should we not persist opHandleToSessionHandle map as well? Otherwise, 
orphaned sessions will be still orphaned upon lens server restart.


- Amareshwari Sriramadasu


On Dec. 10, 2015, 10:19 a.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 10, 2015, 10:19 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

> On Dec. 7, 2015, 7:56 a.m., Rajat Khandelwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 803
> > 
> >
> > If sessions are getting closed at query finish, there shouldn't be any 
> > orphans now. Let's log the orphans and the cause(if possible) for their 
> > being orphans.
> 
> Deepak Barr wrote:
> The close() method is never called except for some test classes. So I am 
> not quite sure if there will be orphan sessions. For example, if a query is 
> running on a orphaned session and driver.close() is called,  then we would 
> want to close the orphaned sessions ( which will also kill the query running 
> on that session)

Agree with Deepak here. driver.close should not close the sessions, and i think 
right now, driver.close itself is not called. We can remove session close from 
here to be consistent with semantics.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review109130
---


On Dec. 2, 2015, 10:40 a.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Dec. 2, 2015, 10:40 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> a84c679 
>   
> lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
>  ab5ada9 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> f6693aa 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/
---

(Updated Dec. 2, 2015, 10:40 a.m.)


Review request for lens.


Changes
---

Proper clean up of orphaned hive sessions


Repository: lens


Description
---

OperationHandle-HiveSession mapping is maintained for all active operations. 
Now, hive session will only be closed when there are no active operations 
running on that session.


Diffs (updated)
-

  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
a84c679 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 ab5ada9 
  lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
f6693aa 

Diff: https://reviews.apache.org/r/39911/diff/


Testing
---

Yes.

1. Created session.
2. Fired a query.
3. Deleted the session before query finishes. 
4. Query keeps running and finishes eventually.


Thanks,

Deepak Barr

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review108103
---


Deepak Barr, Is there any update on this for review comments?

- Amareshwari Sriramadasu


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 13, 2015, 6:57 a.m., Pranav Agarwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1258
> > 
> >
> > If we indeed decide to change the semantic of this method then we 
> > should minimally communicate back to the user that the session was not 
> > really closed, mere logging is not sufficient. This will have a cascading 
> > impact to the closeSession API exposed to the user.

Closing a session will close the lens session but not the underlying hive 
session. I dont think it should be ok to tell the user that the session is 
closed because session can not be used again anyway. What do you think ?


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review106370
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 26, 2015, 9:14 a.m., Amareshwari Sriramadasu wrote:
> > Deepak Barr, Is there any update on this for review comments?

Sorry, I was occupied with something else. Will resume work on this. From the 
review comments, I am thinking we should do the following -

1. For sync queries (CLI), closing the session will kill the queries . For 
async, queries wont be killed.
2. We will clean up the orphaned hive sessions periodically when the queries on 
the hive session finishes. I imagined hive might clean up the session after 
certain timeout but the functionaliy is available version 0.14 onwards 
(HIVE-5799).

Let me know if its fine with everyone.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review108103
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

> On Nov. 4, 2015, 3:51 p.m., Puneet Gupta wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1249
> > 
> >
> > What happens to skipped(open) sessions ? do we need to close them later 
> > once query execution is over ?
> 
> Deepak Barr wrote:
> I think we can rely of hive server to clean it up eventually.
> 
> Puneet Gupta wrote:
> Not sure if that will be OK. Can someone comment form Hive perspective..
> Will lot open(skipped) hive sessions form lens create issues while 
> creating new Hive sessions?
> What is the timeout for Hive sessions in production? 
> 
> Incase its Not Ok form Hive prespective, we might have to inititiate 
> periodic cleanup on lens side. (If not periodic, it can also be triggeregd 
> when any query is purged. Not very relaible though)
> Only other problem would be server restart. Where do we store the open 
> hive sessions details ? We  might have to serailize that info too. 
> 
> We an take this up separately also if required and close this issue.

Hive does not do any cleanup of sessions. This will lead to session leaks and 
Memory pile up on hive. We should take care of closing the hive sessions when 
all its queries are completed.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105069
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Pranav Agarwal]

> On Nov. 13, 2015, 6:57 a.m., Pranav Agarwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1245
> > 
> >
> > In my opinion we shouldn't change the default behavior of this API. 
> > Since the user asked to close the session this semantic implies that the 
> > underying queries should be killed.
> > 
> > If we want to address the usecase of closing the session that are 
> > inactive and are not running any queries then driver should have an API to 
> > return the running queries for a session. And the caller should make a call 
> > whether they want to first check if there are any running queries in the 
> > driver before calling closeSession.
> 
> Puneet Gupta wrote:
> I was wondering when user has issued an "asynchronous" query, why should 
> the user wait till the query finishes (sometimes more than a day). He can 
> close the cli/session and come back later and check the status or kill the 
> query or do other operations available via query handle. Keeping the session 
> open when a user doesn't intend to do any further oprations other than just 
> waiting for the async query result, can be wastage of resources for lens 
> server. When a sync query is issued, the behaviour can be different, if 
> really required.

In my opinion - If the queries are running, even after explictly closing the 
session then it's counter intutive. Today the user know's if the session is 
closed then all the queries fired from that session will be killed and that 
makes good sense to me.

I do understand the issue about async long running queries to continue even 
after the session is closed. To handle that we can probably accept an additonal 
parameter say "noHangUp" at the time of query submission and the sessions 
running queries with noHangUp shouldn't be closed in Hive but can be removed 
from LensServer. I would think only users querying via CLI will need this 
option, as the session close for rest of the apps can be managed by the client 
code.


- Pranav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review106370
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Puneet Gupta]

> On Nov. 13, 2015, 6:57 a.m., Pranav Agarwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1245
> > 
> >
> > In my opinion we shouldn't change the default behavior of this API. 
> > Since the user asked to close the session this semantic implies that the 
> > underying queries should be killed.
> > 
> > If we want to address the usecase of closing the session that are 
> > inactive and are not running any queries then driver should have an API to 
> > return the running queries for a session. And the caller should make a call 
> > whether they want to first check if there are any running queries in the 
> > driver before calling closeSession.

I was wondering when user has issued an "asynchronous" query, why should the 
user wait till the query finishes (sometimes more than a day). He can close the 
cli/session and come back later and check the status or kill the query or do 
other operations available via query handle. Keeping the session open when a 
user doesn't intend to do any further oprations other than just waiting for the 
async query result, can be wastage of resources for lens server. When a sync 
query is issued, the behaviour can be different, if really required.


- Puneet


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review106370
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 9, 2015, 12:57 p.m., Rajat Khandelwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1260
> > 
> >
> > Add test cases for not closeable?

didn't get you ? I added a test case that closes the lens session but wont 
close hivesession because of running query.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105671
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105671
---



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 1260)


Add test cases for not closeable?


- Rajat Khandelwal


On Nov. 9, 2015, 12:28 a.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 9, 2015, 12:28 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 4, 2015, 3:51 p.m., Puneet Gupta wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1249
> > 
> >
> > What happens to skipped(open) sessions ? do we need to close them later 
> > once query execution is over ?

I think we can rely of hive server to clean it up eventually.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105069
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Puneet Gupta]

> On Nov. 4, 2015, 3:51 p.m., Puneet Gupta wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1249
> > 
> >
> > What happens to skipped(open) sessions ? do we need to close them later 
> > once query execution is over ?
> 
> Deepak Barr wrote:
> I think we can rely of hive server to clean it up eventually.

Not sure if that will be OK. Can someone comment form Hive perspective..
Will lot open(skipped) hive sessions form lens create issues while creating new 
Hive sessions?
What is the timeout for Hive sessions in production? 

Incase its Not Ok form Hive prespective, we might have to inititiate periodic 
cleanup on lens side. (If not periodic, it can also be triggeregd when any 
query is purged. Not very relaible though)
Only other problem would be server restart. Where do we store the open hive 
sessions details ? We  might have to serailize that info too. 

We an take this up separately also if required and close this issue.


- Puneet


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105069
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105069
---



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 516)


Should this be done in case on Execptions also, as in should we do this in 
finally clause ?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 1249)


What happens to skipped(open) sessions ? do we need to close them later 
once query execution is over ?



lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
(line 1254)


Can we print session id also here?


- Puneet Gupta


On Nov. 3, 2015, 6:43 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 3, 2015, 6:43 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review105035
---


Can you add unit test?

- Amareshwari Sriramadasu


On Nov. 3, 2015, 6:43 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 3, 2015, 6:43 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>