[jira] [Updated] (LENS-879) Base framework for Scheduler.

[Ajay Yadava (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LENS-879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajay Yadava updated LENS-879:
-
Status: Patch Available  (was: Open)

> Base framework for Scheduler.
> -
>
> Key: LENS-879
> URL: https://issues.apache.org/jira/browse/LENS-879
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: server
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
> Attachments: LENS-879-v1.patch, LENS-879-v2.patch, LENS-879.patch
>
>
> Add interfaces for scheduler service and xsd for job.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40638: Base framework for scheduler.

[Ajay Yadava]

> On Nov. 25, 2015, 7:11 a.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/resources/job-0.1.xsd, line 35
> > 
> >
> > Please add  for all elements and 
> > attributes.
> > 
> > See cube-0.1.xsd, for example.
> 
> Ajay Yadava wrote:
>  is added at the definition of type not at the usage of 
> type, it is already there for this type(and most others). I will again do a 
> thorough check again if I have missed something.
> 
> Amareshwari Sriramadasu wrote:
> I feel xs:annotation should be added for elements, because type 
> documentation will be very much generic and context will be misisng. For ex: 
> map_type is saying key, value pairs. session_conf which is a map_type element 
> needs further explanation wrt what the configuration is for. Same applies to 
> session_db and so on.

Sorry I didn't understand what you meant the first time. Got it. Will add 
documentation.


- Ajay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review107921
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

Re: Review Request 40638: Base framework for scheduler.

[Ajay Yadava]

> On Nov. 27, 2015, 11:03 a.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java,
> >  line 27
> > 
> >
> > `SchedulerJobStats`?

Will rename.


- Ajay


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108210
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

Re: Review Request 40638: Base framework for scheduler.

[Ajay Yadava]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/
---

(Updated Nov. 27, 2015, 7:46 p.m.)


Review request for lens.


Changes
---

Addressed the review comments.


Bugs: LENS-879
https://issues.apache.org/jira/browse/LENS-879


Repository: lens


Description
---

Base framework for lens scheduler service and job definition.


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
PRE-CREATION 
  lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInfo.java 
PRE-CREATION 
  
lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
 PRE-CREATION 
  
lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceInfo.java 
PRE-CREATION 
  lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
  lens-examples/src/main/resources/example-job.xml PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
 9f48d27 
  
lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobStats.java
 PRE-CREATION 
  
lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
 PRE-CREATION 
  
lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
 3389aba 
  
lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
 24949bd 
  
lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
 PRE-CREATION 
  lens-server/src/main/resources/lensserver-default.xml 5f268cb 
  
lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
 e11413e 
  pom.xml b19857f 
  src/site/apt/admin/config.apt 88c1489 

Diff: https://reviews.apache.org/r/40638/diff/


Testing
---


Thanks,

Ajay Yadava

Re: Review Request 40638: Base framework for scheduler.

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108210
---



lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
 (line 27)


`SchedulerJobStats`?


- Rajat Khandelwal


On Nov. 25, 2015, 9:27 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 9:27 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

Re: Review Request 40638: Base framework for scheduler.

[Amareshwari Sriramadasu]

> On Nov. 26, 2015, 6:51 a.m., Rajat Khandelwal wrote:
> > lens-api/src/main/resources/example-job.xml, lines 27-35
> > 
> >
> > Feels like keeping two conf is only useful when they are present on 
> > different heirarchies. 
> > 
> > If we are creating session just to run a single query, then there is no 
> > point of having session level and query level configurations. One 
> > configuration should be enough
> > 
> > Thinking in those terms, an execution shouldn't be a `query`, it should 
> > be a session. Inside that session there can be multiple actions, which 
> > might decide the next actions. Any of those actions can be a `query` 
> > action. 
> > 
> > Thoughts?
> 
> Amareshwari Sriramadasu wrote:
> I agree there can be many actions in session. But we cannot support each 
> action through xsd. I think what we can support is List of REST Actions, we 
> would be generic. Whether we want to support Lens REST actions to be 
> schedulable or not is something debatable. But can be done later.
> 
> We can start with query as one execution and minimally run on a session 
> in a db, as it is put up now. 
> 
> Coming back to merging session conf and query conf into single one, it 
> should be fine as long as it is single query, but we should not take away the 
> fact that query conf can be different for each query when there are multiple 
> queries. So, I'm fine merging them into one in this jira.
> 
> Rajat Khandelwal wrote:
> Considering the important distinction of session conf and query conf, 
> would the following make more sense?
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> As a sequence of two actions, open a session with a conf and run a query 
> with a conf. 
> 
> The following might also make sense:
> 
> ```
>
> 
> session1
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> 
> With this you can open multiple sessions and associate any other action 
> to take place in any of the sessions. Also, it's more explicit. Here, there 
> will be an implicit parallel execution of actions of a different sessions. 
> 
> 
> Third option is, we note that any execution will happen in a session, so:
> 
> ```
>
> 
> http://192.168.30.30:5000/
> 
> sessionKey1
> sessionValue1
> 
> 
> 
> session1
> select name from name_table;
> 
> queryKey1
> queryValue1
> 
> 
> 
> 
> ```
> Here, opening multiple sessions will happen thorugh multiple `execution` 
> tags. 
> 
> 
> Would like to hear the views from implementation and use-case standpoints.
> 
> Rajat Khandelwal wrote:
> Come to think of it, first and third are functionaly equivalent.
> 
> Ajay Yadava wrote:
> I am still not clear on why multiple sessions are required, won't they 
> all be from same user?
> 
> Amareshwari Sriramadasu wrote:
> I agree with Ajay, one schedule would only work with one session, I don't 
> think we need to support multiple sessions at all. 
> 
> Let us keep the version1 simple with one session and one query.
> 
> Ajay Yadava wrote:
> Is it possible that we can unblock this JIRA and discuss the use case and 
> requirements for multiple actions/sessions in parallel? I will be happy to 
> incorporate any changes which might result as part of that discussion.

Rajat, are you suggesting any more changes in this jira? If not, can this be 
dropped?


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108090
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> 

Re: Review Request 40751: LENS - 882: provide option from CLI as well to list queries by driver

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40751/#review108252
---


Will require some changes if LENS-123 is merged . The drivers will be referred 
by their fully qualified names (and not class names as there can be multiple 
instances of HiveDriver/other drivers)

- Puneet Gupta


On Nov. 26, 2015, 3:04 p.m., Deepak Barr wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40751/
> ---
> 
> (Updated Nov. 26, 2015, 3:04 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-882
> https://issues.apache.org/jira/browse/LENS-882
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> After this change,  a CLI user will be able to find a list of queries that 
> ran on a particular driver.
> eg. lens-shell>query list --driver org.apache.lens.driver.hive.HiveDriver
> 
> 
> Diffs
> -
> 
>   lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
> 83b919b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 
> 48b3ebb 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java 5fd04c4 
>   lens-client/src/main/java/org/apache/lens/client/LensStatement.java 0a511f0 
> 
> Diff: https://reviews.apache.org/r/40751/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> INFO] 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.401 
> s]
> [INFO] Lens ... SUCCESS [  4.075 
> s]
> [INFO] Lens API ... SUCCESS [ 19.117 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 16.010 
> s]
> [INFO] Lens Cube .. SUCCESS [04:54 
> min]
> [INFO] Lens DB storage  SUCCESS [ 15.088 
> s]
> [INFO] Lens Query Library . SUCCESS [ 11.770 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [02:24 
> min]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 29.353 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 14.007 
> s]
> [INFO] Lens Server  SUCCESS [12:05 
> min]
> [INFO] Lens client  SUCCESS [ 29.389 
> s]
> [INFO] Lens CLI ... SUCCESS [ 44.601 
> s]
> [INFO] Lens Examples .. SUCCESS [  6.650 
> s]
> [INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  0.863 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 10.363 
> s]
> [INFO] Lens ML Lib  SUCCESS [01:21 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  3.111 
> s]
> [INFO] Lens Regression  SUCCESS [ 10.240 
> s]
> [INFO] Lens UI  SUCCESS [  3.515 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 24:26 min
> [INFO] Finished at: 2015-11-26T18:04:55+05:30
> [INFO] Final Memory: 143M/857M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Deepak Barr
> 
>

Review Request 40761: Query Completion email missing for some queries

[Puneet Gupta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40761/
---

Review request for lens.


Bugs: LENS-887
https://issues.apache.org/jira/browse/LENS-887


Repository: lens


Description (updated)
---

Initial Changes (as discussed in JIRA) 
+ disabled core pool timeout for AsyncEventListeners (since pool had only 
single thread)


Diffs (updated)
-

  
lens-server-api/src/main/java/org/apache/lens/server/api/events/AsyncEventListener.java
 547c008 
  lens-server/src/main/java/org/apache/lens/server/EventServiceImpl.java 
a276828 

Diff: https://reviews.apache.org/r/40761/diff/


Testing (updated)
---

> Verified Build .
> Verified new Thread Names while debugging.

[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [2.085s]
[INFO] Lens .. SUCCESS [2.889s]
[INFO] Lens API .. SUCCESS [25.083s]
[INFO] Lens API for server and extensions  SUCCESS [23.336s]
[INFO] Lens Cube . SUCCESS [5:57.544s]
[INFO] Lens DB storage ... SUCCESS [20.762s]
[INFO] Lens Query Library  SUCCESS [15.393s]
[INFO] Lens Hive Driver .. SUCCESS [2:50.461s]
[INFO] Lens Driver for JDBC .. SUCCESS [35.760s]
[INFO] Lens Elastic Search Driver  SUCCESS [17.248s]
[INFO] Lens Server ... SUCCESS [9:50.223s]
[INFO] Lens client ... SUCCESS [36.590s]
[INFO] Lens CLI .. SUCCESS [51.185s]
[INFO] Lens Examples . SUCCESS [9.779s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.037s]
[INFO] Lens Distribution . SUCCESS [8.573s]
[INFO] Lens ML Lib ... SUCCESS [1:20.846s]
[INFO] Lens ML Ext Distribution .. SUCCESS [2.013s]
[INFO] Lens Regression ... SUCCESS [12.349s]
[INFO] Lens UI ... SUCCESS [27.294s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 24:51.394s
[INFO] Finished at: Sat Nov 28 04:07:51 UTC 2015
[INFO] Final Memory: 199M/2040M
[INFO] 


Thanks,

Puneet Gupta

Review Request 40772: LENS-888: Limit of 4000 on total string length of dim attribute names of derived cube

[Rajat Khandelwal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40772/
---

Review request for lens.


Bugs: LENS-888
https://issues.apache.org/jira/browse/LENS-888


Repository: lens


Description
---


Diffs
-

  lens-cube/src/main/java/org/apache/lens/cube/metadata/DerivedCube.java 
3c30f78516c8c5fd9ca45aa58d4187758c66d0bf 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
4b57d95180a700fa65b213e1c7a2a036ec509579 

Diff: https://reviews.apache.org/r/40772/diff/


Testing
---


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-888) Limit of 4000 on total string length of dim attribute names of derived cube

[Rajat Khandelwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029967#comment-15029967
 ] 

Rajat Khandelwal commented on LENS-888:
---

Created https://reviews.apache.org/r/40772/

> Limit of 4000 on total string length of dim attribute names of derived cube
> ---
>
> Key: LENS-888
> URL: https://issues.apache.org/jira/browse/LENS-888
> Project: Apache Lens
>  Issue Type: Bug
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (LENS-888) Limit of 4000 on total string length of dim attribute names of derived cube

[Rajat Khandelwal (JIRA)]

Rajat Khandelwal created LENS-888:
-

 Summary: Limit of 4000 on total string length of dim attribute 
names of derived cube
 Key: LENS-888
 URL: https://issues.apache.org/jira/browse/LENS-888
 Project: Apache Lens
  Issue Type: Bug
Reporter: Rajat Khandelwal
Assignee: Rajat Khandelwal






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-879) Base framework for Scheduler.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15030216#comment-15030216
 ] 

Hadoop QA commented on LENS-879:


Applied patch: 
[LENS-879-v2.patch|https://issues.apache.org/jira/secure/attachment/12774662/LENS-879-v2.patch]
 and ran command: mvn clean install -fae. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/205/

> Base framework for Scheduler.
> -
>
> Key: LENS-879
> URL: https://issues.apache.org/jira/browse/LENS-879
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: server
>Reporter: Ajay Yadava
>Assignee: Ajay Yadava
> Attachments: LENS-879-v1.patch, LENS-879-v2.patch, LENS-879.patch
>
>
> Add interfaces for scheduler service and xsd for job.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-887) Query Completion email missing for some queries.

[Puneet Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029682#comment-15029682
 ] 

Puneet Gupta commented on LENS-887:
---

QueryEndNotifier extends AsyncEventListener . The AsyncEventListener has a 
ThreadPool to handle all events asynchronously .
For QueryEndNotifier, the thread pool is created with one fixed thread 
(coorPoolSize =1 and maximumPoolSize=1).
Exceptions from sendMail() are not being caught. This can cause some the worker 
thread in the pool to be removed. But even in this case the pool has mechanism 
to recreate workers if some of them are discarded. Pool should not be a problem 
here.
Not able to see any other causes as of now from code other than some exception 
that is thrown by QueryEndNotifier.process() code each time its called (before 
sendMail() is called) . Since such Exceptions are not being logged, we may be 
unaware of them .
Will do the following changes 
1. Add Naming for threads, so that we are sure all pool threads are healthy 
2. Catch Exceptions in AsyncEventListener.onEvent().
We can also decide on the Pool size . 1 may not be enough when many queries are 
executing parallely. Also ResultFormat extends AsyncEventListener. We will have 
to increase the pool size for result formatting also.

> Query Completion email missing for some queries.
> 
>
> Key: LENS-887
> URL: https://issues.apache.org/jira/browse/LENS-887
> Project: Apache Lens
>  Issue Type: Bug
>Reporter: Puneet Gupta
>Assignee: Puneet Gupta
>
> Noticed that query completion mails suddenly stops coming after a few days of 
> operation.  Only Server Restart resolves the problem. 
> Not much information in logs. QueryEndNotifier log "Sending completion email 
> for query handle: {}" is missing even though queries have completed 
> successfully.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40638: Base framework for scheduler.

[Amareshwari Sriramadasu]

> On Nov. 25, 2015, 7:11 a.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/resources/job-0.1.xsd, line 35
> > 
> >
> > Please add  for all elements and 
> > attributes.
> > 
> > See cube-0.1.xsd, for example.
> 
> Ajay Yadava wrote:
>  is added at the definition of type not at the usage of 
> type, it is already there for this type(and most others). I will again do a 
> thorough check again if I have missed something.

I feel xs:annotation should be added for elements, because type documentation 
will be very much generic and context will be misisng. For ex: map_type is 
saying key, value pairs. session_conf which is a map_type element needs further 
explanation wrt what the configuration is for. Same applies to session_db and 
so on.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review107921
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

Re: Review Request 40638: Base framework for scheduler.

[Amareshwari Sriramadasu]

> On Nov. 27, 2015, 5:25 a.m., Amareshwari Sriramadasu wrote:
> > lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java, 
> > line 42
> > 
> >
> > I feel we should call the class ScheduleHandle instead of 
> > SchedulerJobHandler. 
> > 
> > and similarly others. what do you think?
> 
> Ajay Yadava wrote:
> Do you mean to use Schedule instead of SchedulerJob everywhere? I had 
> given my reasons above for not incorporating those changes earlier. If it 
> still looks out of place to you then I am happy to change it. Just let me 
> know what would you like for the following:
> SchdulerJob -> ?
> SchedulerJobInstance -> ?

OK. Thinking again, current naming seems fine. Dropping my comment.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40638/#review108188
---


On Nov. 25, 2015, 3:57 p.m., Ajay Yadava wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40638/
> ---
> 
> (Updated Nov. 25, 2015, 3:57 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-879
> https://issues.apache.org/jira/browse/LENS-879
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Base framework for lens scheduler service and job definition.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobHandle.java 
> PRE-CREATION 
>   
> lens-api/src/main/java/org/apache/lens/api/query/SchedulerJobInstanceHandle.java
>  PRE-CREATION 
>   lens-api/src/main/resources/example-job.xml PRE-CREATION 
>   lens-api/src/main/resources/scheduler-job-0.1.xsd PRE-CREATION 
>   lens-examples/src/main/resources/example-job.xml PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/JobStats.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/QuerySchedulerService.java
>  9f48d27 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerJobInstanceInfo.java
>  PRE-CREATION 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/scheduler/SchedulerService.java
>  PRE-CREATION 
>   
> lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java
>  3389aba 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/QuerySchedulerServiceImpl.java
>  24949bd 
>   
> lens-server/src/main/java/org/apache/lens/server/scheduler/SchedulerServiceImpl.java
>  PRE-CREATION 
>   lens-server/src/main/resources/lensserver-default.xml 5f268cb 
>   
> lens-server/src/test/java/org/apache/lens/server/healthcheck/TestHealthChecks.java
>  e11413e 
>   pom.xml b19857f 
>   src/site/apt/admin/config.apt 88c1489 
> 
> Diff: https://reviews.apache.org/r/40638/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ajay Yadava
> 
>

[jira] [Created] (LENS-887) Query Completion email missing for some queries.

[Puneet Gupta (JIRA)]

Puneet Gupta created LENS-887:
-

 Summary: Query Completion email missing for some queries.
 Key: LENS-887
 URL: https://issues.apache.org/jira/browse/LENS-887
 Project: Apache Lens
  Issue Type: Bug
Reporter: Puneet Gupta


Noticed that query completion mails suddenly stops coming after a few days of 
operation.  Only Server Restart resolves the problem. 

Not much information in logs. QueryEndNotifier log "Sending completion email 
for query handle: {}" is missing even though queries have completed 
successfully.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029586#comment-15029586
 ] 

Hudson commented on LENS-884:
-

SUCCESS: Integrated in Lens-Commit-Java8 #57 (See 
[https://builds.apache.org/job/Lens-Commit-Java8/57/])
LENS-884 : Fix lens-ship-jars.jar to have version in the assembly as 
(amareshwari: rev 404d4518d9cf96d1a2c41a73da598fc27fece992)
* lens-ship-jars/pom.xml
* pom.xml
* lens-dist/src/main/assembly/bin-dist.xml


> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 2.4
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Fix For: 2.5
>
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-884) version is not associated with lens ship jar

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LENS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029595#comment-15029595
 ] 

Hudson commented on LENS-884:
-

SUCCESS: Integrated in Lens-Commit #1144 (See 
[https://builds.apache.org/job/Lens-Commit/1144/])
LENS-884 : Fix lens-ship-jars.jar to have version in the assembly as 
(amareshwari: rev 404d4518d9cf96d1a2c41a73da598fc27fece992)
* pom.xml
* lens-ship-jars/pom.xml
* lens-dist/src/main/assembly/bin-dist.xml


> version is not associated with lens ship jar
> 
>
> Key: LENS-884
> URL: https://issues.apache.org/jira/browse/LENS-884
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 2.4
>Reporter: Archana H
>Assignee: Rajat Khandelwal
> Fix For: 2.5
>
> Attachments: LENS-884.01.patch
>
>
> Lens ship jar is generated without version
> lens-ship-jars.jar



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)