[jira] [Created] (LENS-1158) Thread names with hive2 build are growing huge

2016-05-26 Thread Amareshwari Sriramadasu (JIRA) 
Amareshwari Sriramadasu created LENS-1158:
-

 Summary: Thread names with hive2 build are growing huge
 Key: LENS-1158
 URL: https://issues.apache.org/jira/browse/LENS-1158
 Project: Apache Lens
  Issue Type: Bug
Affects Versions: 2.6
Reporter: Amareshwari Sriramadasu
Assignee: Rajat Khandelwal
 Fix For: 2.6


One of the log looks the following:

27 May 2016 04:10:24 [bf1c4dce-f578-4237-8035-b3ba6846319e] 
[cbfb48cc-e1eb-4b58-9db4-b5b09d15a55d 36caf230-8e8a-4b4d-a0ef-857ec94c0996 
7e180c99-0d54-42a0-86e7-d9f88e7e5942 f302b519-9397-4b82-8f50-c1fc93da58d8 
097a39a7-b948-4804-ad77-f6beab75a017 1461e1a1-cb77-4c7a-8b20-85969dbcbc9e 
656edc21-6050-4174-aa2b-d6ea9d1f71de 276d7420-c9c3-43b2-94f1-e26282b9d24c 
grizzly-http-server-32] ERROR hive.ql.metadata.Table - Unable to get field from 
serde:



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-1143) Fix test cases to incorporate LENS-1019 changes

2016-05-26 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302008#comment-15302008
 ] 

Hadoop QA commented on LENS-1143:
-

Applied patch: 
[0001-Test-case-fix.patch|https://issues.apache.org/jira/secure/attachment/12806344/0001-Test-case-fix.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/467/

> Fix test cases to incorporate LENS-1019 changes
> ---
>
> Key: LENS-1143
> URL: https://issues.apache.org/jira/browse/LENS-1143
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
>Priority: Blocker
> Fix For: 2.6
>
> Attachments: 0001-Fixing-Text-case-for-duplicate-queries.patch, 
> 0001-Test-case-fix.patch
>
>
> Test cases should keep lens.server.duplicate.query.allowed true by default. 
> Because some of the test cases might be submitting the same query and 
> expecting the queue length to increase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1143) Fix test cases to incorporate LENS-1019 changes

2016-05-26 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1143:
--
Status: Patch Available  (was: Reopened)

> Fix test cases to incorporate LENS-1019 changes
> ---
>
> Key: LENS-1143
> URL: https://issues.apache.org/jira/browse/LENS-1143
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
>Priority: Blocker
> Fix For: 2.6
>
> Attachments: 0001-Fixing-Text-case-for-duplicate-queries.patch, 
> 0001-Test-case-fix.patch
>
>
> Test cases should keep lens.server.duplicate.query.allowed true by default. 
> Because some of the test cases might be submitting the same query and 
> expecting the queue length to increase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1143) Fix test cases to incorporate LENS-1019 changes

2016-05-26 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1143:
--
Attachment: 0001-Test-case-fix.patch

> Fix test cases to incorporate LENS-1019 changes
> ---
>
> Key: LENS-1143
> URL: https://issues.apache.org/jira/browse/LENS-1143
> Project: Apache Lens
>  Issue Type: Bug
>  Components: server
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
>Priority: Blocker
> Fix For: 2.6
>
> Attachments: 0001-Fixing-Text-case-for-duplicate-queries.patch, 
> 0001-Test-case-fix.patch
>
>
> Test cases should keep lens.server.duplicate.query.allowed true by default. 
> Because some of the test cases might be submitting the same query and 
> expecting the queue length to increase. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47450: Lens-1143

2016-05-26 Thread Lavkesh Lahngir 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47450/
---

(Updated May 26, 2016, 11:17 a.m.)


Review request for lens.


Bugs: LENS-1143
https://issues.apache.org/jira/browse/LENS-1143


Repository: lens


Description
---

Fixing test cases:
default value of lens.server.duplicate.query.allowed is false in the lens-site, 
which should be true in test cases. 
This patch fixes this and provides a separate test case for duplicate queries.


Diffs (updated)
-

  
lens-server/src/test/java/org/apache/lens/server/query/TestDuplicateQueries.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/query/TestQueryServiceDuplicate.java
 0828403 

Diff: https://reviews.apache.org/r/47450/diff/


Testing
---


Thanks,

Lavkesh Lahngir

Re: Review Request 47777: LENS-661: Added Weighted selector

2016-05-26 Thread Anshul Gupta 


> On May 25, 2016, 12:50 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  line 78
> > 
> >
> > Define constants for property name and default value in 
> > `LensConfConstants`.

Defined the constant. Default value is already being specified in the 
getDouble method.


> On May 25, 2016, 12:50 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  line 132
> > 
> >
> > Maybe we should rename it to `lens.driver.weight`. Having both `ratio` 
> > and `weight` might be confusing to the end-user. For end users, they need 
> > to set the selector class name in `lens-site.xml` and this property in 
> > driver's configuration files. The combination of  > lens.driver.weight> sounds more intuitive.

renamed to lens.driver.weight


> On May 25, 2016, 12:50 p.m., Rajat Khandelwal wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightCostSelector.java,
> >  line 32
> > 
> >
> > Rename to a more explicit name like `WeightedQueryCostDriverSelector`.

renamed


On May 25, 2016, 12:50 p.m., Anshul Gupta wrote:
> > The reactor summary is incomplete. It doesn't cover all the modules.

Updated reactor summary.


- Anshul


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4/#review134760
---


On May 26, 2016, 10:07 a.m., Anshul Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4/
> ---
> 
> (Updated May 26, 2016, 10:07 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-661
> https://issues.apache.org/jira/browse/LENS-661
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Added Weighted selector
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
> 82672294fcbf462db1fd75f00e796d5dae9d2c22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  85f54d1f835f86dd6122a987525853e0ffa9488c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightedQueryCostDriverSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestMinCostSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestWeightedCostSelector.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/4/diff/
> 
> 
> Testing
> ---
> 
> Here's the reactor summary after running mvn clean package
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Server  SUCCESS [11:32 
> min]
> [INFO] Lens client  SUCCESS [01:49 
> min]
> [INFO] Lens CLI ... SUCCESS [02:02 
> min]
> [INFO] Lens Examples .. SUCCESS [  8.430 
> s]
> [INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  2.801 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 21.429 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:28 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [ 20.690 
> s]
> [INFO] Lens Regression  SUCCESS [  9.313 
> s]
> [INFO] Lens UI  SUCCESS [ 14.163 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 19:10 min
> [INFO] Finished at: 2016-05-24T20:31:02+05:30
> [INFO] Final Memory: 103M/466M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Anshul Gupta
> 
>

Re: Review Request 47777: LENS-661: Added Weighted selector

2016-05-26 Thread Anshul Gupta 


> On May 25, 2016, 12:50 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  lines 28-34
> > 
> >
> > It's not using any of `lens-cube` dependencies, so this test class can 
> > move to `lens-server-api` module. Same should be possible with 
> > `TestMinCostSelector`. Let's move both there.

Classes moved to lens-server-api module.


> On May 25, 2016, 12:50 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  line 66
> > 
> >
> > This method is performing 2 tests sequentially. The problem with that 
> > is that if the first one fails, the second one never gets to run. This 
> > method should be split into multiple methods. Ideally, one method per test 
> > should be followed. 
> > 
> > 
> > 1. 3 Drivers with ratio -- Should distribute in ratio.
> > 2. 3 Drivers without ratio -- Should distribute equally.
> > 3. 2 Drivers, where cost of queries on one driver is always less than 
> > the other one -- The first one should always be picked.
> > 
> > For 1 and 2, let's have one extra driver which always gives a high 
> > cost. Assertions will be that that driver should never get picked.

Added three test cases along with another test case for the default 
MinCostDriver behavior.


- Anshul


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4/#review134760
---


On May 26, 2016, 10:07 a.m., Anshul Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4/
> ---
> 
> (Updated May 26, 2016, 10:07 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-661
> https://issues.apache.org/jira/browse/LENS-661
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Added Weighted selector
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
> 82672294fcbf462db1fd75f00e796d5dae9d2c22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  85f54d1f835f86dd6122a987525853e0ffa9488c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightedQueryCostDriverSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestMinCostSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestWeightedCostSelector.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/4/diff/
> 
> 
> Testing
> ---
> 
> Here's the reactor summary after running mvn clean package
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Server  SUCCESS [11:32 
> min]
> [INFO] Lens client  SUCCESS [01:49 
> min]
> [INFO] Lens CLI ... SUCCESS [02:02 
> min]
> [INFO] Lens Examples .. SUCCESS [  8.430 
> s]
> [INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  2.801 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 21.429 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:28 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [ 20.690 
> s]
> [INFO] Lens Regression  SUCCESS [  9.313 
> s]
> [INFO] Lens UI  SUCCESS [ 14.163 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 19:10 min
> [INFO] Finished at: 2016-05-24T20:31:02+05:30
> [INFO] Final Memory: 103M/466M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Anshul Gupta
> 
>

Re: Review Request 47777: LENS-661: Added Weighted selector

2016-05-26 Thread Anshul Gupta 


> On May 25, 2016, noon, Puneet Gupta wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  line 56
> > 
> >
> > argument drivers is never user.

Removed the method containing the drivers argument from both 
TestMinCostSelector & TestWeightedCostSelector


> On May 25, 2016, noon, Puneet Gupta wrote:
> > lens-cube/src/test/java/org/apache/lens/driver/cube/TestWeightedCostSelector.java,
> >  line 105
> > 
> >
> > Should wereduce the counter to 1000 to reduce the test run time ?

Reduced the counter to 1000. Changed the desired buffer to 5%


- Anshul


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4/#review134756
---


On May 26, 2016, 10:07 a.m., Anshul Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4/
> ---
> 
> (Updated May 26, 2016, 10:07 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-661
> https://issues.apache.org/jira/browse/LENS-661
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Added Weighted selector
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
> 82672294fcbf462db1fd75f00e796d5dae9d2c22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  85f54d1f835f86dd6122a987525853e0ffa9488c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightedQueryCostDriverSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestMinCostSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestWeightedCostSelector.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/4/diff/
> 
> 
> Testing
> ---
> 
> Here's the reactor summary after running mvn clean package
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Server  SUCCESS [11:32 
> min]
> [INFO] Lens client  SUCCESS [01:49 
> min]
> [INFO] Lens CLI ... SUCCESS [02:02 
> min]
> [INFO] Lens Examples .. SUCCESS [  8.430 
> s]
> [INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  2.801 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 21.429 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:28 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [ 20.690 
> s]
> [INFO] Lens Regression  SUCCESS [  9.313 
> s]
> [INFO] Lens UI  SUCCESS [ 14.163 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 19:10 min
> [INFO] Finished at: 2016-05-24T20:31:02+05:30
> [INFO] Final Memory: 103M/466M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Anshul Gupta
> 
>

Re: Review Request 47777: LENS-661: Added Weighted selector

2016-05-26 Thread Anshul Gupta 


> On May 25, 2016, 5:54 a.m., Puneet Gupta wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightCostSelector.java,
> >  lines 46-48
> > 
> >
> > Better to check explicitly is debug logs are enabled before going into 
> > this loop . In most cases this loop can be skipped.
> 
> Anshul Gupta wrote:
> Added check for whether debug logs are enabled.
> Added driver ratio in the debug logs.
> 
> Puneet Gupta wrote:
> Did not find the check in latest path . Please check

The earlier diff wasn't created properly.
Added at Line 51 of WeightedQueryCostDriverSelector


> On May 25, 2016, 5:54 a.m., Puneet Gupta wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightCostSelector.java,
> >  line 51
> > 
> >
> > Should we optimize in case only one driver is selected ? We can skip 
> > all the extra processing in this case
> 
> Anshul Gupta wrote:
> Optimized for the same.
> 
> Puneet Gupta wrote:
> Did not find it in latest patch.

Added


- Anshul


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4/#review134702
---


On May 26, 2016, 10:07 a.m., Anshul Gupta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4/
> ---
> 
> (Updated May 26, 2016, 10:07 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-661
> https://issues.apache.org/jira/browse/LENS-661
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Added Weighted selector
> 
> 
> Diffs
> -
> 
>   
> lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
> 82672294fcbf462db1fd75f00e796d5dae9d2c22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  85f54d1f835f86dd6122a987525853e0ffa9488c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightedQueryCostDriverSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestMinCostSelector.java
>  PRE-CREATION 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestWeightedCostSelector.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/4/diff/
> 
> 
> Testing
> ---
> 
> Here's the reactor summary after running mvn clean package
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Server  SUCCESS [11:32 
> min]
> [INFO] Lens client  SUCCESS [01:49 
> min]
> [INFO] Lens CLI ... SUCCESS [02:02 
> min]
> [INFO] Lens Examples .. SUCCESS [  8.430 
> s]
> [INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  2.801 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 21.429 
> s]
> [INFO] Lens ML Lib  SUCCESS [02:28 
> min]
> [INFO] Lens ML Ext Distribution ... SUCCESS [ 20.690 
> s]
> [INFO] Lens Regression  SUCCESS [  9.313 
> s]
> [INFO] Lens UI  SUCCESS [ 14.163 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 19:10 min
> [INFO] Finished at: 2016-05-24T20:31:02+05:30
> [INFO] Final Memory: 103M/466M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Anshul Gupta
> 
>

Re: Review Request 47777: LENS-661: Added Weighted selector

2016-05-26 Thread Anshul Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4/
---

(Updated May 26, 2016, 10:07 a.m.)


Review request for lens.


Changes
---

Added all the changes as discussed.


Summary (updated)
-

LENS-661: Added Weighted selector


Bugs: LENS-661
https://issues.apache.org/jira/browse/LENS-661


Repository: lens


Description
---

Added Weighted selector


Diffs (updated)
-

  lens-cube/src/test/java/org/apache/lens/driver/cube/TestMinCostSelector.java 
82672294fcbf462db1fd75f00e796d5dae9d2c22 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
85f54d1f835f86dd6122a987525853e0ffa9488c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/driver/WeightedQueryCostDriverSelector.java
 PRE-CREATION 
  
lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestMinCostSelector.java
 PRE-CREATION 
  
lens-server-api/src/test/java/org/apache/lens/server/api/driver/TestWeightedCostSelector.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/4/diff/


Testing
---

Here's the reactor summary after running mvn clean package
[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Server  SUCCESS [11:32 min]
[INFO] Lens client  SUCCESS [01:49 min]
[INFO] Lens CLI ... SUCCESS [02:02 min]
[INFO] Lens Examples .. SUCCESS [  8.430 s]
[INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  2.801 s]
[INFO] Lens Distribution .. SUCCESS [ 21.429 s]
[INFO] Lens ML Lib  SUCCESS [02:28 min]
[INFO] Lens ML Ext Distribution ... SUCCESS [ 20.690 s]
[INFO] Lens Regression  SUCCESS [  9.313 s]
[INFO] Lens UI  SUCCESS [ 14.163 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 19:10 min
[INFO] Finished at: 2016-05-24T20:31:02+05:30
[INFO] Final Memory: 103M/466M
[INFO] 


Thanks,

Anshul Gupta

[jira] [Work started] (LENS-661) Add Weighted Driver selector for selecting driver among hive and spark drivers

2016-05-26 Thread Anshul Gupta (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-661 started by Anshul Gupta.
-
> Add Weighted Driver selector for selecting driver among hive and spark drivers
> --
>
> Key: LENS-661
> URL: https://issues.apache.org/jira/browse/LENS-661
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: driver-hive
>Reporter: Raju Bairishetti
>Assignee: Anshul Gupta
>
> Spark Driver also uses hive thrift server as an execution engine. Both spark 
> and hive drivers are eligible for executing the same cube query. Right now, 
> Jdbc has Query cost 0.0 and Hive driver has 1.0. Spark driver also uses same 
> query cost as hive driver which is 1.0.
> We can introduce a weighted driver selector which will submit configured % of 
> queries to spark driver for execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-661) Add Weighted Driver selector for selecting driver among hive and spark drivers

2016-05-26 Thread Anshul Gupta (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301890#comment-15301890
 ] 

Anshul Gupta commented on LENS-661:
---

https://reviews.apache.org/r/4/

> Add Weighted Driver selector for selecting driver among hive and spark drivers
> --
>
> Key: LENS-661
> URL: https://issues.apache.org/jira/browse/LENS-661
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: driver-hive
>Reporter: Raju Bairishetti
>Assignee: Anshul Gupta
>
> Spark Driver also uses hive thrift server as an execution engine. Both spark 
> and hive drivers are eligible for executing the same cube query. Right now, 
> Jdbc has Query cost 0.0 and Hive driver has 1.0. Spark driver also uses same 
> query cost as hive driver which is 1.0.
> We can introduce a weighted driver selector which will submit configured % of 
> queries to spark driver for execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (LENS-1157) Execute with timeout fails to read metadata when user requests streamed results

2016-05-26 Thread Puneet Gupta (JIRA) 
Puneet Gupta created LENS-1157:
--

 Summary: Execute with timeout fails to read metadata when user 
requests streamed results
 Key: LENS-1157
 URL: https://issues.apache.org/jira/browse/LENS-1157
 Project: Apache Lens
  Issue Type: Bug
Reporter: Puneet Gupta
Assignee: Puneet Gupta


This happens when 
1. EXECUTE_WITH_TIMEOUT options is used to execute a query
2. server persistence is enabled 
3. Query returns zero rows (within timeout)

{noformat}
apache.lens.driver.jdbc.JDBCResultSet - Error getting JDBC type information: 
INTERNAL_SERVER_ERROR
1308 org.apache.lens.server.api.error.LensException: INTERNAL_SERVER_ERROR
1309 at 
org.apache.lens.driver.jdbc.JDBCResultSet.getRsMetadata(JDBCResultSet.java:89) 
~[lens-driver-jdbc-2.6.5-inm.jar:2.6.5-inm]
1310 at 
org.apache.lens.driver.jdbc.JDBCResultSet.getMetadata(JDBCResultSet.java:112) 
~[lens-driver-jdbc-2.6.5-inm.jar:2.6.5-inm]
1311 at 
org.apache.lens.server.api.driver.PartiallyFetchedInMemoryResultSet.getMetadata(PartiallyFetchedInMemoryResultSet.java:154)
 [lens-server-api-2.6.5-inm.jar:2.6.5-inm]
1312 at 
org.apache.lens.server.query.QueryExecutionServiceImpl.executeTimeoutInternal(QueryExecutionServiceImpl.java:2068)
 [classes/:na]
1313 at 
org.apache.lens.server.query.QueryExecutionServiceImpl.execute(QueryExecutionServiceImpl.java:2009)
 [classes/:na]
1314 at 
org.apache.lens.server.query.QueryServiceResource.query(QueryServiceResource.java:234)
 [classes/:na]
1315 at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source) 
~[na:na]
1316 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[na:1.8.0_72]
1317 at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_72]
1318 at 
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
 [jersey-server-2.22.1.jar:na]
1319 at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
 [jersey-server-2.22.1.jar:na]
1320 at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
 [jersey-server-2.22.1.jar:na]
1321 at 
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
 [jersey-server-2.22.1.jar:na]
1322 at 
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
 [jersey-server-2.22.1.jar:na]
1323 at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
 [jersey-server-2.22.1.jar:na]
1324 at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
 [jersey-server-2.22.1.jar:na]
1325 at 
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
 [jersey-server-2.22.1.jar:na]
1326 at 
org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) 
[jersey-server-2.22.1.jar:na]
1327 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) 
[jersey-common-2.22.1.jar:na]
1328 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) 
[jersey-common-2.22.1.jar:na]
1329 at org.glassfish.jersey.internal.Errors.process(Errors.java:315) 
[jersey-common-2.22.1.jar:na]
1330 at org.glassfish.jersey.internal.Errors.process(Errors.java:297) 
[jersey-common-2.22.1.jar:na]
1331 at org.glassfish.jersey.internal.Errors.process(Errors.java:267) 
[jersey-common-2.22.1.jar:na]
1332 at 
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
 [jersey-common-2.22.1.jar:na]
1333 at 
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) 
[jersey-server-2.22.1.jar:na]
1334 at 
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
 [jersey-server-2.22.1.jar:na]
1335 at 
org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384)
 [jersey-container-grizzly2-http-2.22.1.jar:na]
1336 at 
org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) 
[grizzly-http-server-2.3.23.jar:2.3.23]
1337 at 
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:591)
 [grizzly-framework-2.3.23.jar:2.3.23]
1338 at 
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:571)
 [grizzly-framework-2.3.23.jar:2.3.23]
1339 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
1340 Caused by: java.sql.S

Re: Review Request 47776: LENS-1151: More info on Queries moving between waiting and queued queries data structures

2016-05-26 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47776/#review134941
---


Ship it!




Ship It!

- Amareshwari Sriramadasu


On May 24, 2016, 2:02 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47776/
> ---
> 
> (Updated May 24, 2016, 2:02 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-1151
> https://issues.apache.org/jira/browse/LENS-1151
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Current logs don't give much info and for debugging why a query took so much 
> time in queue would be helpful to understand whether or not the wait time of 
> a query is valid.
> 
> 
> Diffs
> -
> 
>   
> lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
>  1b3a7c0f6dd9949d313d7c5920d0f0f1dcd18c0b 
>   
> lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java
>  2decf42ec9f06ef9019af24bf64c749edd484cff 
> 
> Diff: https://reviews.apache.org/r/47776/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>