[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Srikanth Sundarrajan (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029348#comment-15029348
 ] 

Srikanth Sundarrajan commented on LENS-884:
---

We can append the version using the version property in the assembly descriptor 
as opposed to attaching the shade goal to top level pom (which would shade 
artifacts of every module)

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 39842: Ability to load different instances of same driver class

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39842/
---

(Updated Nov. 27, 2015, 5:05 a.m.)


Review request for lens, Amareshwari Sriramadasu and Rajat Khandelwal.


Changes
---

2 Chnages--

1. Patch Appy failed . Merged with latest code 
2. Updated Test case TestQueryConstraints to use multiple drivers support . 
Also fixed a bug in this test case which was causing intermittent failure due 
to assertValidity() logic.
org.apache.lens.server.query.TestQueryConstraints.assertValidity()
if (count.running == 4) {
assertEquals(count.queued, 0);
  } else {
assertEquals(count.waiting, 0);
  }
Assuming … all candidate queries have moved to waiting from queued and 4 
queries are running. Lets say the 4th query finishes before we call 
queryService.getQueryCountSnapshot() from the test case assertValidity(), so 
running queries =3.  Now we call   assertEquals(count.waiting, 0), but the 
query service thread has still not pushed a waiting query to launched . In this 
case we will see waiting queries >0 even though running queries != 4


Bugs: LENS-123
https://issues.apache.org/jira/browse/LENS-123


Repository: lens


Description
---

Ability to load multiple drivers on lens server. 

As of now only one driver instance of each type (hive,jdbc,es,etc) can be 
loaded by lens server. Hence lens can not support for example, two jdbc 
instances, one for MySql and one for Vertica or just two different MySQl 
deployments. This can be a big limitation for some deployments.  

Made below changes in fix 
1. Based on the approaches discussed in JIRA(LENS-123), the below folder 
structure was picked.  (Please refer to the discussion on JIRA if required - 
last few comments )
-conf
--drivers 
---hive 
driver1
-hivedriver-site.xml
driver2
-hivedriver-site.xml
---jdbc
driver1
-jdbcdriver-site.xml
driver2
-jdbcdriver-site.xml
Note: drivers configuration is read from "drivers" directory under conf 
location. Conf loaction is set while starting the server as -Dconfig.location = 
"" 

2. Added an abstract class  
"org.apache.lens.server.api.driver.AbstractLensDriver". This has common 
implemenation for getting lens driver's fully qualified name and some methods 
for getting driver specific resource paths. All existing driver implementations 
extend this class. In future, we can also add common functionality here(if 
required) without affecting existing driver implementations that extend this 
abstract class.

3. Updated driver interface 
-configure method now takes driver name and type configuration parameters.
-added getFullyQualifiedName() method to identify the driver uniquely. 

4. Value of lens.server.drivers property in lens-site.xml (and 
default-lens-site) updated to include driver type and driver class name 
(earlier only driver class name was present).
hive:org.apache.lens.driver.hive.HiveDriver,jdbc:org.apache.lens.driver.jdbc.JDBCDriver

5. Updated LensServerDAO to use driver qualified name instead of driver class 
name. The table column name is chnaged to drivername form driverclass 

6. The driver instance was earlier referred to by its class name 
(Example:org.apache.lens.driver.hive.HiveDriver) everywhere in code. This is 
now changed to driver's fully qualified name which has the driver type and 
driver name (Example :hive/hive1)

7. All metrics were also relying on driver class name. This is also changed to 
driver's fully qualified name.

8. Updated driver folder struture for test cases and lens server build (as per 
point1).

9. Query execution service will fail to start if no drivers are found. (This is 
a change from previous flow . We can discuss this if req.)  


Pending ( will upload this soon)
1. Adding test cases to test multiple drivers
2. updating documentation

Future Enhancements(that can be taken up as a separate JIRA)
1.Multiple versions of a datasource to be supported ( say Hive and Hive on 
spark use different versions of Hive. This will not work as of now as we have 
common Hive jars as part of war classpath. We need to have driver specific jars 
which can have different version. For this, the jars and other hive version 
specific resources should be picked from driver's folder by a separate class 
loader).


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/query/LensPreparedQuery.java 
9595ce9 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 204ecee 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
83b919b 
  lens-cli/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
PRE-CREATION 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 
  lens-client/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
PRE-CREATION 
  

[jira] [Commented] (LENS-123) Ability to load different instances of same driver class

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029482#comment-15029482
 ] 

Hadoop QA commented on LENS-123:


Applied patch: 
[lens-123-revision8.patch|https://issues.apache.org/jira/secure/attachment/12774576/lens-123-revision8.patch]
 and ran command: mvn clean install -fae. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/203/

> Ability to load different instances of same driver class
> 
>
> Key: LENS-123
> URL: https://issues.apache.org/jira/browse/LENS-123
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: server
>Reporter: Amareshwari Sriramadasu
>Assignee: Puneet Gupta
> Attachments: lens-123-revision8.patch
>
>
> Currently we are loading only one driver instance per class. We should be 
> able to load multiple different instances of a driver class.
> For example, There can be multiple HiveDrivers talking to different 
> HiveServers sitting on each colo. Or different JDBCDrivers talking to 
> different dbs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 39842: Ability to load different instances of same driver class

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39842/#review108191
---



lens-server/src/test/resources/drivers/mockHive/mockHive1/hivedriver-site.xml 
(lines 75 - 78)


Do we need this for this test?



lens-server/src/test/resources/drivers/mockHive/mockHive1/hivedriver-site.xml 
(lines 80 - 83)


Isn't name auto-inferred?


- Rajat Khandelwal


On Nov. 27, 2015, 10:35 a.m., Puneet Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39842/
> ---
> 
> (Updated Nov. 27, 2015, 10:35 a.m.)
> 
> 
> Review request for lens, Amareshwari Sriramadasu and Rajat Khandelwal.
> 
> 
> Bugs: LENS-123
> https://issues.apache.org/jira/browse/LENS-123
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Ability to load multiple drivers on lens server. 
> 
> As of now only one driver instance of each type (hive,jdbc,es,etc) can be 
> loaded by lens server. Hence lens can not support for example, two jdbc 
> instances, one for MySql and one for Vertica or just two different MySQl 
> deployments. This can be a big limitation for some deployments.  
> 
> Made below changes in fix 
> 1. Based on the approaches discussed in JIRA(LENS-123), the below folder 
> structure was picked.  (Please refer to the discussion on JIRA if required - 
> last few comments )
> -conf
> --drivers 
> ---hive 
> driver1
> -hivedriver-site.xml
> driver2
> -hivedriver-site.xml
> ---jdbc
> driver1
> -jdbcdriver-site.xml
> driver2
> -jdbcdriver-site.xml
> Note: drivers configuration is read from "drivers" directory under conf 
> location. Conf loaction is set while starting the server as -Dconfig.location 
> = "" 
> 
> 2. Added an abstract class  
> "org.apache.lens.server.api.driver.AbstractLensDriver". This has common 
> implemenation for getting lens driver's fully qualified name and some methods 
> for getting driver specific resource paths. All existing driver 
> implementations extend this class. In future, we can also add common 
> functionality here(if required) without affecting existing driver 
> implementations that extend this abstract class.
> 
> 3. Updated driver interface 
> -configure method now takes driver name and type configuration parameters.
> -added getFullyQualifiedName() method to identify the driver uniquely. 
> 
> 4. Value of lens.server.drivers property in lens-site.xml (and 
> default-lens-site) updated to include driver type and driver class name 
> (earlier only driver class name was present).
> hive:org.apache.lens.driver.hive.HiveDriver,jdbc:org.apache.lens.driver.jdbc.JDBCDriver
> 
> 5. Updated LensServerDAO to use driver qualified name instead of driver class 
> name. The table column name is chnaged to drivername form driverclass 
> 
> 6. The driver instance was earlier referred to by its class name 
> (Example:org.apache.lens.driver.hive.HiveDriver) everywhere in code. This is 
> now changed to driver's fully qualified name which has the driver type and 
> driver name (Example :hive/hive1)
> 
> 7. All metrics were also relying on driver class name. This is also changed 
> to driver's fully qualified name.
> 
> 8. Updated driver folder struture for test cases and lens server build (as 
> per point1).
> 
> 9. Query execution service will fail to start if no drivers are found. (This 
> is a change from previous flow . We can discuss this if req.)  
> 
> 
> Pending ( will upload this soon)
> 1. Adding test cases to test multiple drivers
> 2. updating documentation
> 
> Future Enhancements(that can be taken up as a separate JIRA)
> 1.Multiple versions of a datasource to be supported ( say Hive and Hive on 
> spark use different versions of Hive. This will not work as of now as we have 
> common Hive jars as part of war classpath. We need to have driver specific 
> jars which can have different version. For this, the jars and other hive 
> version specific resources should be picked from driver's folder by a 
> separate class loader).
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/LensPreparedQuery.java 
> 9595ce9 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 204ecee 
>   lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
> 83b919b 
>   lens-cli/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
> PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 
>   lens-client/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
> PRE-CREATION 
>   
> 

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029499#comment-15029499
 ] 

Rajat Khandelwal commented on LENS-884:
---

after the change, 

{noformat}
$ jar -tvf lens-api/target/lens-api-2.5.0-beta-SNAPSHOT.jar  | grep -v lens | 
grep -v META-INF
 0 Fri Nov 27 11:47:46 IST 2015 org/
 0 Fri Nov 27 11:47:46 IST 2015 org/apache/
 54251 Fri Nov 27 11:47:42 IST 2015 cube-0.1.xsd
rajat.khandelwal@IM1258-X0:~/Git/lens [0]
$
{noformat}

While lens-api has at lombok, guava etc as dependencies. 

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029498#comment-15029498
 ] 

Rajat Khandelwal commented on LENS-884:
---

Yes, it's not building shaded jars for all modules, we're good :)



> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40638: Base framework for scheduler.

[Rajat Khandelwal]

> On Nov. 26, 2015, 12:21 p.m., Rajat Khandelwal wrote:
> > lens-api/src/main/resources/example-job.xml, lines 27-35
> > 
> >
> > Feels like keeping two conf is only useful when they are present on 
> > different heirarchies. 
> > 
> > If we are creating session just to run a single query, then there is no 
> > point of having session level and query level configurations. One 
> > configuration should be enough
> > 
> > Thinking in those terms, an execution shouldn't be a `query`, it should 
> > be a session. Inside that session there can be multiple actions, which 
> > might decide the next actions. Any of those actions can be a `query` 
> > action. 
> > 
> > Thoughts?
> 
> Amareshwari Sriramadasu wrote:
> I agree there can be many actions in session. But we cannot support each 
> action through xsd. I think what we can support is List of REST Actions, we 
> would be generic. Whether we want to support Lens REST actions to be 
> schedulable or not is something debatable. But can be done later.
> 
> We can start with query as one execution and minimally run on a session 
> in a db, as it is put up now. 
> 
> Coming back to merging session conf and query conf into single one, it 
> should be fine as long as it is single query, but we should not take away the 
> fact that query conf can be different for each query when there are multiple 
> queries. So, I'm fine merging them into one in this jira.

Considering the important distinction of session conf and query conf, would the 
following make more sense?

```
   

http://192.168.30.30:5000/

sessionKey1
sessionValue1



select name from name_table;

queryKey1
queryValue1




```

As a sequence of two actions, open a session with a conf and run a query with a 
conf. 

The following might also make sense:

```
   

session1
http://192.168.30.30:5000/

sessionKey1
sessionValue1



session1
select name from name_table;

queryKey1
queryValue1




```

With this you can open multiple sessions and associate any other action to take 
place in any of the sessions. Also, it's more explicit. Here, there will be an 
implicit parallel execution of actions of a different sessions. 


Third option is, we note that any execution will happen in a session, so:

```
   

http://192.168.30.30:5000/

sessionKey1
sessionValue1



session1
select name from name_table;

queryKey1
queryValue1




```
Here, opening multiple sessions will happen thorugh multiple `execution` tags. 


Would like to hear the views from implementation and use-case standpoints.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108090
---


On Nov. 25, 2015, 9:27 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 9:27 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java

Re: Review Request 40638: Base framework for scheduler.

[Rajat Khandelwal]

> On Nov. 26, 2015, 12:21 p.m., Rajat Khandelwal wrote:
> > lens-api/src/main/resources/example-job.xml, lines 27-35
> > 
> >
> > Feels like keeping two conf is only useful when they are present on 
> > different heirarchies. 
> > 
> > If we are creating session just to run a single query, then there is no 
> > point of having session level and query level configurations. One 
> > configuration should be enough
> > 
> > Thinking in those terms, an execution shouldn't be a `query`, it should 
> > be a session. Inside that session there can be multiple actions, which 
> > might decide the next actions. Any of those actions can be a `query` 
> > action. 
> > 
> > Thoughts?
> 
> Amareshwari Sriramadasu wrote:
> I agree there can be many actions in session. But we cannot support each 
> action through xsd. I think what we can support is List of REST Actions, we 
> would be generic. Whether we want to support Lens REST actions to be 
> schedulable or not is something debatable. But can be done later.
> 
> We can start with query as one execution and minimally run on a session 
> in a db, as it is put up now. 
> 
> Coming back to merging session conf and query conf into single one, it 
> should be fine as long as it is single query, but we should not take away the 
> fact that query conf can be different for each query when there are multiple 
> queries. So, I'm fine merging them into one in this jira.
> 
> Rajat Khandelwal wrote:
> Considering the important distinction of session conf and query conf, 
> would the following make more sense?
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> As a sequence of two actions, open a session with a conf and run a query 
> with a conf. 
> 
> The following might also make sense:
> 
> ```
>
> 
> session1
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> With this you can open multiple sessions and associate any other action 
> to take place in any of the sessions. Also, it's more explicit. Here, there 
> will be an implicit parallel execution of actions of a different sessions. 
> 
> 
> Third option is, we note that any execution will happen in a session, so:
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> Here, opening multiple sessions will happen thorugh multiple `execution` 
> tags. 
> 
> 
> Would like to hear the views from implementation and use-case standpoints.

Come to think of it, first and third are functionaly equivalent.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108090
---


On Nov. 25, 2015, 9:27 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 9:27 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   

[jira] [Commented] (LENS-832) Queued cube queries on JDBCDriver are not getting restored correctly

[Amareshwari Sriramadasu (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029511#comment-15029511
 ] 

Amareshwari Sriramadasu commented on LENS-832:
--

I just ran the above test on local, but did not find any issue :
 {noformat}
lens-shell>query list --state QUEUED
070d0cf1-4ded-48e8-8985-4cafda9ee1ae
07df7d8d-5910-4bec-a8cc-d52fa71e6dc6
a32deff9-5b96-43d4-a528-ef0798d6cb9e
79152fe6-3817-4a4c-92af-a090cccd181d
9d020a18-8c5f-4574-910b-48f29e54f2e8
e56cbe20-69d4-4686-bbac-f971bb05ddb7
57399640-7e47-4157-a05f-4b0fa01e1864
039b61e0-7d41-4034-bbd5-ed3f3dae754f
41c9dbb3-964c-4c47-b79e-eefbbb51afae
94a15a5b-048b-438d-bdea-a7d730bb4653
107482ff-dee4-4238-87f7-357078886ba1
5aed5c2b-7770-49b5-9d39-6e3f8b6ab3bd
534fdd32-bc04-4716-b979-a828e85e917e
69a2df2e-0195-4cb6-a8f5-43c552b9551a
dc6416a1-2112-49c4-b947-645c6ad93a4c
04448f1b-b293-4384-932f-5f92c38be5bf
90fc7a4a-1421-4d16-8c2c-0426920962be
36df14ea-bb1b-4bea-976f-9dfc77b7de04
Total number of queries: 18
lens-shell>query list --state QUEUED
Command failed javax.ws.rs.ProcessingException: java.net.ConnectException: 
Connection refused
lens-shell>query list --state QUEUED
Command failed javax.ws.rs.ProcessingException: java.net.ConnectException: 
Connection refused
lens-shell>query list --state QUEUED
07df7d8d-5910-4bec-a8cc-d52fa71e6dc6
a32deff9-5b96-43d4-a528-ef0798d6cb9e
e56cbe20-69d4-4686-bbac-f971bb05ddb7
9d020a18-8c5f-4574-910b-48f29e54f2e8
41c9dbb3-964c-4c47-b79e-eefbbb51afae
039b61e0-7d41-4034-bbd5-ed3f3dae754f
107482ff-dee4-4238-87f7-357078886ba1
5aed5c2b-7770-49b5-9d39-6e3f8b6ab3bd
69a2df2e-0195-4cb6-a8f5-43c552b9551a
dc6416a1-2112-49c4-b947-645c6ad93a4c
04448f1b-b293-4384-932f-5f92c38be5bf
90fc7a4a-1421-4d16-8c2c-0426920962be
Total number of queries: 12
lens-shell>query status 070d0cf1-4ded-48e8-8985-4cafda9ee1ae
 Status : FAILED
 Message : INTERNAL_SERVER_ERROR
 Progress : 0.0

lens-shell>query status 07df7d8d-5910-4bec-a8cc-d52fa71e6dc6
 Status : FAILED
 Message : INTERNAL_SERVER_ERROR
 Progress : 0.0
{noformat}


> Queued cube queries on JDBCDriver are not getting restored correctly
> 
>
> Key: LENS-832
> URL: https://issues.apache.org/jira/browse/LENS-832
> Project: Apache Lens
>  Issue Type: Bug
>  Components: api, server
>Affects Versions: 2.3
>Reporter: Amareshwari Sriramadasu
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.5
>
> Attachments: LENS-832.patch
>
>
> The query that is getting launched on JDBCDriver after restart is cube select 
> query - not the rewritten query.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029530#comment-15029530
 ] 

Hadoop QA commented on LENS-884:


Applied patch: 
[LENS-884.01.patch|https://issues.apache.org/jira/secure/attachment/12774583/LENS-884.01.patch]
 and ran command: mvn clean install -fae. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/204/

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-884) version is not associated with lens ship jar

[Amareshwari Sriramadasu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-884:
-
   Resolution: Fixed
Fix Version/s: 2.5
   Status: Resolved  (was: Patch Available)

Committed. Thanks [~prongs]

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Fix For: 2.5
>
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-123) Ability to load different instances of same driver class

[Puneet Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puneet Gupta updated LENS-123:
--
Attachment: (was: lens-123-revision7.patch)

> Ability to load different instances of same driver class
> 
>
> Key: LENS-123
> URL: https://issues.apache.org/jira/browse/LENS-123
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: server
>Reporter: Amareshwari Sriramadasu
>Assignee: Puneet Gupta
>
> Currently we are loading only one driver instance per class. We should be 
> able to load multiple different instances of a driver class.
> For example, There can be multiple HiveDrivers talking to different 
> HiveServers sitting on each colo. Or different JDBCDrivers talking to 
> different dbs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40736: LENS-884: version is not associated with lens ship jar

[Srikanth Sundarrajan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40736/#review108187
---

Ship it!


Ship It!

- Srikanth Sundarrajan


On Nov. 26, 2015, 9:03 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40736/
> ---
> 
> (Updated Nov. 26, 2015, 9:03 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-884
> https://issues.apache.org/jira/browse/LENS-884
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Lens ship jar is generated without version
> lens-ship-jars.jar
> 
> 
> Diffs
> -
> 
>   lens-dist/src/main/assembly/bin-dist.xml 
> 535e9911f743ca1f8370ad67f3c1c2603f09ad82 
>   lens-ship-jars/pom.xml b150980ef597f5b064debf68b590243d3ad5e4c6 
>   pom.xml b19857f1610434ba9d99e398d7905d8fbb7bb5ec 
> 
> Diff: https://reviews.apache.org/r/40736/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Amareshwari Sriramadasu (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029451#comment-15029451
 ] 

Amareshwari Sriramadasu commented on LENS-884:
--

shade-plugin is added only to the plugin-management in main pom. AFAIK, that 
should be fine and if it is added to build, only then shaded jar will be 
created. [~prongs] Can you confirm?

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40638: Base framework for scheduler.

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108188
---



lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java (line 
42)


I feel we should call the class ScheduleHandle instead of 
SchedulerJobHandler. 

and similarly others. what do you think?



lens-api/src/main/resources/example-job.xml (line 1)


Seems forgot to remove the class from lens-api.



lens-examples/src/main/resources/example-job.xml (line 31)


session_db here is the db name : like default or db1 or mydb or any



lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
 (line 29)


Is this the class returned to client. It should be moved to lens-api module



lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
 (line 27)


move to lens-api module


- Amareshwari Sriramadasu


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

[jira] [Updated] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-884:
--
Attachment: LENS-884.01.patch

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-884) version is not associated with lens ship jar

[Amareshwari Sriramadasu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-884:
-
Affects Version/s: 2.4
  Component/s: (was: server)
   build
   Issue Type: Improvement  (was: Bug)

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 2.4
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Fix For: 2.5
>
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-884) version is not associated with lens ship jar

[Amareshwari Sriramadasu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-884:
-
Assignee: Rajat Khandelwal  (was: Amareshwari Sriramadasu)

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-883) httpresultset api should return 404 when resultset file does not exist

[Rajat Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-883:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> httpresultset api should return 404 when resultset file does not exist
> --
>
> Key: LENS-883
> URL: https://issues.apache.org/jira/browse/LENS-883
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Deepak Barr
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-883.patch
>
>
> In case the result file is purged by QueryResultPurger, /httpresultset api 
> returns 500. It should return 404.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-883) httpresultset api should return 404 when resultset file does not exist

[Rajat Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-883:
--
Fix Version/s: 2.5

> httpresultset api should return 404 when resultset file does not exist
> --
>
> Key: LENS-883
> URL: https://issues.apache.org/jira/browse/LENS-883
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Deepak Barr
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-883.patch
>
>
> In case the result file is purged by QueryResultPurger, /httpresultset api 
> returns 500. It should return 404.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-883) httpresultset api should return 404 when resultset file does not exist

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028353#comment-15028353
 ] 

Rajat Khandelwal commented on LENS-883:
---

Committed. Thanks [~deepak.barr]

> httpresultset api should return 404 when resultset file does not exist
> --
>
> Key: LENS-883
> URL: https://issues.apache.org/jira/browse/LENS-883
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Deepak Barr
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-883.patch
>
>
> In case the result file is purged by QueryResultPurger, /httpresultset api 
> returns 500. It should return 404.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 39842: Ability to load different instances of same driver class

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39842/
---

(Updated Nov. 26, 2015, 8:33 a.m.)


Review request for lens, Amareshwari Sriramadasu and Rajat Khandelwal.


Changes
---

Fixed intermittent test failure in 
org.apache.lens.server.TestServerRestart.testHiveServerRestart()
- Updated thread safe type for LensSessionImpl.ResourceEntry.restoreCount
- Updated Test case
- Added Driver Qualified name to some some logs in HiveDriver and 
QueryExecutionServiceImpl


Bugs: LENS-123
https://issues.apache.org/jira/browse/LENS-123


Repository: lens


Description
---

Ability to load multiple drivers on lens server. 

As of now only one driver instance of each type (hive,jdbc,es,etc) can be 
loaded by lens server. Hence lens can not support for example, two jdbc 
instances, one for MySql and one for Vertica or just two different MySQl 
deployments. This can be a big limitation for some deployments.  

Made below changes in fix 
1. Based on the approaches discussed in JIRA(LENS-123), the below folder 
structure was picked.  (Please refer to the discussion on JIRA if required - 
last few comments )
-conf
--drivers 
---hive 
driver1
-hivedriver-site.xml
driver2
-hivedriver-site.xml
---jdbc
driver1
-jdbcdriver-site.xml
driver2
-jdbcdriver-site.xml
Note: drivers configuration is read from "drivers" directory under conf 
location. Conf loaction is set while starting the server as -Dconfig.location = 
"" 

2. Added an abstract class  
"org.apache.lens.server.api.driver.AbstractLensDriver". This has common 
implemenation for getting lens driver's fully qualified name and some methods 
for getting driver specific resource paths. All existing driver implementations 
extend this class. In future, we can also add common functionality here(if 
required) without affecting existing driver implementations that extend this 
abstract class.

3. Updated driver interface 
-configure method now takes driver name and type configuration parameters.
-added getFullyQualifiedName() method to identify the driver uniquely. 

4. Value of lens.server.drivers property in lens-site.xml (and 
default-lens-site) updated to include driver type and driver class name 
(earlier only driver class name was present).
hive:org.apache.lens.driver.hive.HiveDriver,jdbc:org.apache.lens.driver.jdbc.JDBCDriver

5. Updated LensServerDAO to use driver qualified name instead of driver class 
name. The table column name is chnaged to drivername form driverclass 

6. The driver instance was earlier referred to by its class name 
(Example:org.apache.lens.driver.hive.HiveDriver) everywhere in code. This is 
now changed to driver's fully qualified name which has the driver type and 
driver name (Example :hive/hive1)

7. All metrics were also relying on driver class name. This is also changed to 
driver's fully qualified name.

8. Updated driver folder struture for test cases and lens server build (as per 
point1).

9. Query execution service will fail to start if no drivers are found. (This is 
a change from previous flow . We can discuss this if req.)  


Pending ( will upload this soon)
1. Adding test cases to test multiple drivers
2. updating documentation

Future Enhancements(that can be taken up as a separate JIRA)
1.Multiple versions of a datasource to be supported ( say Hive and Hive on 
spark use different versions of Hive. This will not work as of now as we have 
common Hive jars as part of war classpath. We need to have driver specific jars 
which can have different version. For this, the jars and other hive version 
specific resources should be picked from driver's folder by a separate class 
loader).


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/query/LensPreparedQuery.java 
9595ce9 
  lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 204ecee 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
83b919b 
  lens-cli/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
PRE-CREATION 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 
  lens-client/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
72f1497 
  lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 14d9f99 
  lens-driver-es/src/test/java/org/apache/lens/driver/es/ESDriverTest.java 
f453416 
  lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
c96ef20 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestHiveDriver.java 
fc57c94 
  
lens-driver-hive/src/test/java/org/apache/lens/driver/hive/TestRemoteHiveDriver.java
 98edc28 
  lens-driver-hive/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
PRE-CREATION 
  

Re: Review Request 39842: Ability to load different instances of same driver class

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39842/#review108098
---

Ship it!


Ship It!

- Amareshwari Sriramadasu


On Nov. 26, 2015, 8:33 a.m., Puneet Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39842/
> ---
> 
> (Updated Nov. 26, 2015, 8:33 a.m.)
> 
> 
> Review request for lens, Amareshwari Sriramadasu and Rajat Khandelwal.
> 
> 
> Bugs: LENS-123
> https://issues.apache.org/jira/browse/LENS-123
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Ability to load multiple drivers on lens server. 
> 
> As of now only one driver instance of each type (hive,jdbc,es,etc) can be 
> loaded by lens server. Hence lens can not support for example, two jdbc 
> instances, one for MySql and one for Vertica or just two different MySQl 
> deployments. This can be a big limitation for some deployments.  
> 
> Made below changes in fix 
> 1. Based on the approaches discussed in JIRA(LENS-123), the below folder 
> structure was picked.  (Please refer to the discussion on JIRA if required - 
> last few comments )
> -conf
> --drivers 
> ---hive 
> driver1
> -hivedriver-site.xml
> driver2
> -hivedriver-site.xml
> ---jdbc
> driver1
> -jdbcdriver-site.xml
> driver2
> -jdbcdriver-site.xml
> Note: drivers configuration is read from "drivers" directory under conf 
> location. Conf loaction is set while starting the server as -Dconfig.location 
> = "" 
> 
> 2. Added an abstract class  
> "org.apache.lens.server.api.driver.AbstractLensDriver". This has common 
> implemenation for getting lens driver's fully qualified name and some methods 
> for getting driver specific resource paths. All existing driver 
> implementations extend this class. In future, we can also add common 
> functionality here(if required) without affecting existing driver 
> implementations that extend this abstract class.
> 
> 3. Updated driver interface 
> -configure method now takes driver name and type configuration parameters.
> -added getFullyQualifiedName() method to identify the driver uniquely. 
> 
> 4. Value of lens.server.drivers property in lens-site.xml (and 
> default-lens-site) updated to include driver type and driver class name 
> (earlier only driver class name was present).
> hive:org.apache.lens.driver.hive.HiveDriver,jdbc:org.apache.lens.driver.jdbc.JDBCDriver
> 
> 5. Updated LensServerDAO to use driver qualified name instead of driver class 
> name. The table column name is chnaged to drivername form driverclass 
> 
> 6. The driver instance was earlier referred to by its class name 
> (Example:org.apache.lens.driver.hive.HiveDriver) everywhere in code. This is 
> now changed to driver's fully qualified name which has the driver type and 
> driver name (Example :hive/hive1)
> 
> 7. All metrics were also relying on driver class name. This is also changed 
> to driver's fully qualified name.
> 
> 8. Updated driver folder struture for test cases and lens server build (as 
> per point1).
> 
> 9. Query execution service will fail to start if no drivers are found. (This 
> is a change from previous flow . We can discuss this if req.)  
> 
> 
> Pending ( will upload this soon)
> 1. Adding test cases to test multiple drivers
> 2. updating documentation
> 
> Future Enhancements(that can be taken up as a separate JIRA)
> 1.Multiple versions of a datasource to be supported ( say Hive and Hive on 
> spark use different versions of Hive. This will not work as of now as we have 
> common Hive jars as part of war classpath. We need to have driver specific 
> jars which can have different version. For this, the jars and other hive 
> version specific resources should be picked from driver's folder by a 
> separate class loader).
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/LensPreparedQuery.java 
> 9595ce9 
>   lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java 204ecee 
>   lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
> 83b919b 
>   lens-cli/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
> PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 
>   lens-client/src/test/resources/drivers/hive/hive1/hivedriver-site.xml 
> PRE-CREATION 
>   
> lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
> 72f1497 
>   lens-driver-es/src/main/java/org/apache/lens/driver/es/ESDriver.java 
> 14d9f99 
>   lens-driver-es/src/test/java/org/apache/lens/driver/es/ESDriverTest.java 
> f453416 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> 

[jira] [Updated] (LENS-123) Ability to load different instances of same driver class

[Puneet Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puneet Gupta updated LENS-123:
--
Attachment: lens-123-revision7.patch

Fixed intermittent test failure in 
org.apache.lens.server.TestServerRestart.testHiveServerRestart()

> Ability to load different instances of same driver class
> 
>
> Key: LENS-123
> URL: https://issues.apache.org/jira/browse/LENS-123
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: server
>Reporter: Amareshwari Sriramadasu
>Assignee: Puneet Gupta
> Attachments: lens-123-revision7.patch
>
>
> Currently we are loading only one driver instance per class. We should be 
> able to load multiple different instances of a driver class.
> For example, There can be multiple HiveDrivers talking to different 
> HiveServers sitting on each colo. Or different JDBCDrivers talking to 
> different dbs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 40736: LENS-884: version is not associated with lens ship jar

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40736/
---

Review request for lens.


Bugs: LENS-884
https://issues.apache.org/jira/browse/LENS-884


Repository: lens


Description
---

Lens ship jar is generated without version
lens-ship-jars.jar


Diffs
-

  lens-dist/src/main/assembly/bin-dist.xml 
535e9911f743ca1f8370ad67f3c1c2603f09ad82 
  lens-ship-jars/pom.xml b150980ef597f5b064debf68b590243d3ad5e4c6 
  pom.xml b19857f1610434ba9d99e398d7905d8fbb7bb5ec 

Diff: https://reviews.apache.org/r/40736/diff/


Testing
---


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028387#comment-15028387
 ] 

Rajat Khandelwal commented on LENS-884:
---

Created https://reviews.apache.org/r/40736/

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-884 started by Rajat Khandelwal.
-
> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-883) httpresultset api should return 404 when resultset file does not exist

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028390#comment-15028390
 ] 

Hudson commented on LENS-883:
-

SUCCESS: Integrated in Lens-Commit-Java8 #56 (See 
[https://builds.apache.org/job/Lens-Commit-Java8/56/])
LENS-883 : httpresultset api should return 404 when resultset file does 
(rajatgupta59: rev 942f071f9925b1265a67a1f7b2ab437127485a09)
* 
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java


> httpresultset api should return 404 when resultset file does not exist
> --
>
> Key: LENS-883
> URL: https://issues.apache.org/jira/browse/LENS-883
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Deepak Barr
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-883.patch
>
>
> In case the result file is purged by QueryResultPurger, /httpresultset api 
> returns 500. It should return 404.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028391#comment-15028391
 ] 

Rajat Khandelwal commented on LENS-884:
---

It's not generated without version. When it's copied in server installation 
folder, version is removed. Have raised the review request. But I suspect it 
might have been a conscious decision. [~sriksun] can maybe explain. 

> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Archana H
>Assignee: Rajat Khandelwal
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40736: LENS-884: version is not associated with lens ship jar

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40736/#review108101
---

Ship it!


Ship It!

- Amareshwari Sriramadasu


On Nov. 26, 2015, 9:03 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40736/
> ---
> 
> (Updated Nov. 26, 2015, 9:03 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-884
> https://issues.apache.org/jira/browse/LENS-884
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Lens ship jar is generated without version
> lens-ship-jars.jar
> 
> 
> Diffs
> -
> 
>   lens-dist/src/main/assembly/bin-dist.xml 
> 535e9911f743ca1f8370ad67f3c1c2603f09ad82 
>   lens-ship-jars/pom.xml b150980ef597f5b064debf68b590243d3ad5e4c6 
>   pom.xml b19857f1610434ba9d99e398d7905d8fbb7bb5ec 
> 
> Diff: https://reviews.apache.org/r/40736/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Amareshwari Sriramadasu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review108103
---


Deepak Barr, Is there any update on this for review comments?

- Amareshwari Sriramadasu


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

[jira] [Commented] (LENS-883) httpresultset api should return 404 when resultset file does not exist

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028402#comment-15028402
 ] 

Hudson commented on LENS-883:
-

UNSTABLE: Integrated in Lens-Commit #1143 (See 
[https://builds.apache.org/job/Lens-Commit/1143/])
LENS-883 : httpresultset api should return 404 when resultset file does 
(rajatgupta59: rev 942f071f9925b1265a67a1f7b2ab437127485a09)
* 
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java


> httpresultset api should return 404 when resultset file does not exist
> --
>
> Key: LENS-883
> URL: https://issues.apache.org/jira/browse/LENS-883
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Deepak Barr
>Assignee: Deepak Barr
> Fix For: 2.5
>
> Attachments: LENS-883.patch
>
>
> In case the result file is purged by QueryResultPurger, /httpresultset api 
> returns 500. It should return 404.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-123) Ability to load different instances of same driver class

[Puneet Gupta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Puneet Gupta updated LENS-123:
--
Status: Patch Available  (was: Open)

> Ability to load different instances of same driver class
> 
>
> Key: LENS-123
> URL: https://issues.apache.org/jira/browse/LENS-123
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: server
>Reporter: Amareshwari Sriramadasu
>Assignee: Puneet Gupta
> Attachments: lens-123-revision7.patch
>
>
> Currently we are loading only one driver instance per class. We should be 
> able to load multiple different instances of a driver class.
> For example, There can be multiple HiveDrivers talking to different 
> HiveServers sitting on each colo. Or different JDBCDrivers talking to 
> different dbs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (LENS-832) Queued cube queries on JDBCDriver are not getting restored correctly

[Archana H (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028454#comment-15028454
 ] 

Archana H edited comment on LENS-832 at 11/26/15 9:56 AM:
--

Still failing
Scenario:
There were 5 JDBC queries in QUEUED state and lens was restarted.
After restart its saying query not found for these queries.

lens-shell>query status --query_handle 5a5612ef-94ec-4e48-a8e5-18f9b1e7298
Command failed java.lang.IllegalStateException: Failed to get query status, 
cause:HTTP 404 Not Found

lens-shell>show logs 5a5612ef-94ec-4e48-a8e5-18f9b1e7298b
last line in the output : 
[QuerySubmitter] INFO  org.apache.lens.server.query.QueryExecutionServiceImpl - 
Added to waiting queries. QueryId:5a5612ef-94ec-4e48-a8e5-18f9b1e7298b


was (Author: archana.h):
Still failing
Scenario:
There were 5 JDBC queries in QUEUED state and lens was restarted.
After restart its saying query not found for these queries.

lens-shell>show logs 5a5612ef-94ec-4e48-a8e5-18f9b1e7298b
last line in the output : 
[QuerySubmitter] INFO  org.apache.lens.server.query.QueryExecutionServiceImpl - 
Added to waiting queries. QueryId:5a5612ef-94ec-4e48-a8e5-18f9b1e7298b

> Queued cube queries on JDBCDriver are not getting restored correctly
> 
>
> Key: LENS-832
> URL: https://issues.apache.org/jira/browse/LENS-832
> Project: Apache Lens
>  Issue Type: Bug
>  Components: api, server
>Affects Versions: 2.3
>Reporter: Amareshwari Sriramadasu
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.5
>
> Attachments: LENS-832.patch
>
>
> The query that is getting launched on JDBCDriver after restart is cube select 
> query - not the rewritten query.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-832) Queued cube queries on JDBCDriver are not getting restored correctly

[Archana H (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028454#comment-15028454
 ] 

Archana H commented on LENS-832:


Still failing
Scenario:
There were 5 JDBC queries in QUEUED state and lens was restarted.
After restart its saying query not found for these queries.

lens-shell>show logs 5a5612ef-94ec-4e48-a8e5-18f9b1e7298b
last line in the output : 
[QuerySubmitter] INFO  org.apache.lens.server.query.QueryExecutionServiceImpl - 
Added to waiting queries. QueryId:5a5612ef-94ec-4e48-a8e5-18f9b1e7298b

> Queued cube queries on JDBCDriver are not getting restored correctly
> 
>
> Key: LENS-832
> URL: https://issues.apache.org/jira/browse/LENS-832
> Project: Apache Lens
>  Issue Type: Bug
>  Components: api, server
>Affects Versions: 2.3
>Reporter: Amareshwari Sriramadasu
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.5
>
> Attachments: LENS-832.patch
>
>
> The query that is getting launched on JDBCDriver after restart is cube select 
> query - not the rewritten query.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (LENS-885) Cleanup of Cube test cases

[Rajat Khandelwal (JIRA)]

Rajat Khandelwal created LENS-885:
-

 Summary: Cleanup of Cube test cases
 Key: LENS-885
 URL: https://issues.apache.org/jira/browse/LENS-885
 Project: Apache Lens
  Issue Type: Sub-task
Reporter: Rajat Khandelwal
Assignee: Rajat Khandelwal


* Some functions are wrongly placed. e.g. DateUtil.getCeilDate(date, 
updatePeriod) can be better expressed as updatePeriod.getCeilDate(date).
* Date parsing from query can be cached
* There is a bloat of Date variables in CubeTestSetup, organize them and only 
keep named variables for the most used ones
* Unify time range clause creation across test cases. Right now, tests create 
time ranges as and when needed. Remove repetition and redundancies. 
* Correct names to variables





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 40739: LENS-885: Cleanup of Cube test cases

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40739/
---

Review request for lens.


Bugs: LENS-885
https://issues.apache.org/jira/browse/LENS-885


Repository: lens


Description
---

* Some functions are wrongly placed. e.g. DateUtil.getCeilDate(date, 
updatePeriod) can be better expressed as updatePeriod.getCeilDate(date).
* Date parsing from query can be cached
* There is a bloat of Date variables in CubeTestSetup, organize them and only 
keep named variables for the most used ones
* Unify time range clause creation across test cases. Right now, tests create 
time ranges as and when needed. Remove repetition and redundancies. 
* Correct names to variables


Diffs
-

  lens-cube/src/main/java/org/apache/lens/cube/metadata/UpdatePeriod.java 
4c76a69d0e2986d450d81b8cbed877995696ba0d 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DateUtil.java 
5e17eac62a6b12ef04d1d4e3254efbda54402eb6 
  
lens-cube/src/test/java/org/apache/lens/cube/metadata/TestCubeMetastoreClient.java
 e415a5adeb5a8af16bad56c96b9f9a32cb7afcbb 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
1357035c8dfc6a5115c02d8fdc7cd8bb341eb6dd 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/FieldsCannotBeQueriedTogetherTest.java
 0fea9f1bb4a266b680a252786ebd9e931d1b0c09 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestBetweenTimeRangeWriter.java
 9a2493c306938a0823d4b7d2bd264f915143ea64 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
fea70b72cbde190d69d46f68d163fd9e541c53f8 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 d16ea4c56a792f1ac100e398bfdc6c8d1eab4490 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestORTimeRangeWriter.java 
4a23818313482d420721d62f908af4cc52f8311d 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeExtractor.java 
a4317173f9544ebcab05e290c3129479afd3bd6d 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriter.java 
0248409123f0588cbd5b45d0894bc9bf3503c888 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestTimeRangeWriterWithQuery.java
 7bd7b6b962fd9be83309ea3fc870dee9aefc2895 

Diff: https://reviews.apache.org/r/40739/diff/


Testing
---


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-885) Cleanup of Cube test cases

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028541#comment-15028541
 ] 

Rajat Khandelwal commented on LENS-885:
---

Created https://reviews.apache.org/r/40739/

> Cleanup of Cube test cases
> --
>
> Key: LENS-885
> URL: https://issues.apache.org/jira/browse/LENS-885
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: build
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>
> * Some functions are wrongly placed. e.g. DateUtil.getCeilDate(date, 
> updatePeriod) can be better expressed as updatePeriod.getCeilDate(date).
> * Date parsing from query can be cached
> * There is a bloat of Date variables in CubeTestSetup, organize them and only 
> keep named variables for the most used ones
> * Unify time range clause creation across test cases. Right now, tests create 
> time ranges as and when needed. Remove repetition and redundancies. 
> * Correct names to variables



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (LENS-885) Cleanup of Cube test cases

[Rajat Khandelwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-885 started by Rajat Khandelwal.
-
> Cleanup of Cube test cases
> --
>
> Key: LENS-885
> URL: https://issues.apache.org/jira/browse/LENS-885
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: build
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>
> * Some functions are wrongly placed. e.g. DateUtil.getCeilDate(date, 
> updatePeriod) can be better expressed as updatePeriod.getCeilDate(date).
> * Date parsing from query can be cached
> * There is a bloat of Date variables in CubeTestSetup, organize them and only 
> keep named variables for the most used ones
> * Unify time range clause creation across test cases. Right now, tests create 
> time ranges as and when needed. Remove repetition and redundancies. 
> * Correct names to variables



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 40751: LENS - 882: provide option from CLI as well to list queries by driver

[Deepak Barr]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40751/
---

Review request for lens.


Bugs: LENS-882
https://issues.apache.org/jira/browse/LENS-882


Repository: lens


Description
---

After this change,  a CLI user will be able to find a list of queries that ran 
on a particular driver.
eg. lens-shell>query list --driver org.apache.lens.driver.hive.HiveDriver


Diffs
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
83b919b 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 48b3ebb 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 5fd04c4 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 

Diff: https://reviews.apache.org/r/40751/diff/


Testing
---

Yes.

INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules .. SUCCESS [  2.401 s]
[INFO] Lens ... SUCCESS [  4.075 s]
[INFO] Lens API ... SUCCESS [ 19.117 s]
[INFO] Lens API for server and extensions . SUCCESS [ 16.010 s]
[INFO] Lens Cube .. SUCCESS [04:54 min]
[INFO] Lens DB storage  SUCCESS [ 15.088 s]
[INFO] Lens Query Library . SUCCESS [ 11.770 s]
[INFO] Lens Hive Driver ... SUCCESS [02:24 min]
[INFO] Lens Driver for JDBC ... SUCCESS [ 29.353 s]
[INFO] Lens Elastic Search Driver . SUCCESS [ 14.007 s]
[INFO] Lens Server  SUCCESS [12:05 min]
[INFO] Lens client  SUCCESS [ 29.389 s]
[INFO] Lens CLI ... SUCCESS [ 44.601 s]
[INFO] Lens Examples .. SUCCESS [  6.650 s]
[INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  0.863 s]
[INFO] Lens Distribution .. SUCCESS [ 10.363 s]
[INFO] Lens ML Lib  SUCCESS [01:21 min]
[INFO] Lens ML Ext Distribution ... SUCCESS [  3.111 s]
[INFO] Lens Regression  SUCCESS [ 10.240 s]
[INFO] Lens UI  SUCCESS [  3.515 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 24:26 min
[INFO] Finished at: 2015-11-26T18:04:55+05:30
[INFO] Final Memory: 143M/857M
[INFO] 


Thanks,

Deepak Barr

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 13, 2015, 6:57 a.m., Pranav Agarwal wrote:
> > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java, 
> > line 1258
> > 
> >
> > If we indeed decide to change the semantic of this method then we 
> > should minimally communicate back to the user that the session was not 
> > really closed, mere logging is not sufficient. This will have a cascading 
> > impact to the closeSession API exposed to the user.

Closing a session will close the lens session but not the underlying hive 
session. I dont think it should be ok to tell the user that the session is 
closed because session can not be used again anyway. What do you think ?


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review106370
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 39911: LENS-760 : Session close should not result in running query failures

[Deepak Barr]

> On Nov. 26, 2015, 9:14 a.m., Amareshwari Sriramadasu wrote:
> > Deepak Barr, Is there any update on this for review comments?

Sorry, I was occupied with something else. Will resume work on this. From the 
review comments, I am thinking we should do the following -

1. For sync queries (CLI), closing the session will kill the queries . For 
async, queries wont be killed.
2. We will clean up the orphaned hive sessions periodically when the queries on 
the hive session finishes. I imagined hive might clean up the session after 
certain timeout but the functionaliy is available version 0.14 onwards 
(HIVE-5799).

Let me know if its fine with everyone.


- Deepak


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39911/#review108103
---


On Nov. 8, 2015, 6:58 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39911/
> ---
> 
> (Updated Nov. 8, 2015, 6:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> OperationHandle-HiveSession mapping is maintained for all active operations. 
> Now, hive session will only be closed when there are no active operations 
> running on that session.
> 
> 
> Diffs
> -
> 
>   lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java 
> c96ef20 
>   
> lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java 
> c37b0ed 
> 
> Diff: https://reviews.apache.org/r/39911/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 1. Created session.
> 2. Fired a query.
> 3. Deleted the session before query finishes. 
> 4. Query keeps running and finishes eventually.
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Re: Review Request 40638: Base framework for scheduler.

[Ajay Yadava]

> On Nov. 26, 2015, 6:51 a.m., Rajat Khandelwal wrote:
> > lens-api/src/main/resources/example-job.xml, lines 27-35
> > 
> >
> > Feels like keeping two conf is only useful when they are present on 
> > different heirarchies. 
> > 
> > If we are creating session just to run a single query, then there is no 
> > point of having session level and query level configurations. One 
> > configuration should be enough
> > 
> > Thinking in those terms, an execution shouldn't be a `query`, it should 
> > be a session. Inside that session there can be multiple actions, which 
> > might decide the next actions. Any of those actions can be a `query` 
> > action. 
> > 
> > Thoughts?
> 
> Amareshwari Sriramadasu wrote:
> I agree there can be many actions in session. But we cannot support each 
> action through xsd. I think what we can support is List of REST Actions, we 
> would be generic. Whether we want to support Lens REST actions to be 
> schedulable or not is something debatable. But can be done later.
> 
> We can start with query as one execution and minimally run on a session 
> in a db, as it is put up now. 
> 
> Coming back to merging session conf and query conf into single one, it 
> should be fine as long as it is single query, but we should not take away the 
> fact that query conf can be different for each query when there are multiple 
> queries. So, I'm fine merging them into one in this jira.
> 
> Rajat Khandelwal wrote:
> Considering the important distinction of session conf and query conf, 
> would the following make more sense?
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> As a sequence of two actions, open a session with a conf and run a query 
> with a conf. 
> 
> The following might also make sense:
> 
> ```
>
> 
> session1
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> With this you can open multiple sessions and associate any other action 
> to take place in any of the sessions. Also, it's more explicit. Here, there 
> will be an implicit parallel execution of actions of a different sessions. 
> 
> 
> Third option is, we note that any execution will happen in a session, so:
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> Here, opening multiple sessions will happen thorugh multiple `execution` 
> tags. 
> 
> 
> Would like to hear the views from implementation and use-case standpoints.
> 
> Rajat Khandelwal wrote:
> Come to think of it, first and third are functionaly equivalent.

I am still not clear on why multiple sessions are required, won't they all be 
from same user?


- Ajay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108090
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml