Re: Review Request 41039: LENS-890: Allow per-queue driver max launched queries constraints

2015-12-07 Thread Rajat Khandelwal 


> On Dec. 8, 2015, 9:08 a.m., Puneet Gupta wrote:
> > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java,
> >  line 48
> > 
> >
> > Shall we make this be a separate constraint and not part of this 
> > constraint? 
> > 
> > MaxConcurrentQueriesPerQueueConstraint will only be used if its 
> > configured ( say for Hive Driver and not for JDBC)

I did not want to add another confusing classname for users to set in driver 
properties. One class can handle all 4 combinations of 2 constraints being 
present/absent.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41039/#review109253
---


On Dec. 7, 2015, 8:28 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41039/
> ---
> 
> (Updated Dec. 7, 2015, 8:28 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-890
> https://issues.apache.org/jira/browse/LENS-890
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/util/CommonUtils.java 
> 38d58c710a2b917c34cda5de96a36dc49b6f15f0 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
>  2f20113a1a30f902a1db4b25d5a78da0126a1b0c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
>  bae2e6490aafbfd55c9983c94d9cc892af384f7b 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java
>  b6e6c2f1bd2e4f3c0acdcdaae57dcd812e6bb435 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java
>  55a2eea110f9001c962faede01cef396e8c91b6c 
>   
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java
>  9138f8ea2bb0b87bfcd2c9a4770bc732856fa680 
> 
> Diff: https://reviews.apache.org/r/41039/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 41033: LENS-836: Query commands in CLI should take default value for query handle as the last executed query

2015-12-07 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41033/
---

(Updated Dec. 8, 2015, 12:20 p.m.)


Review request for lens.


Bugs: LENS-836
https://issues.apache.org/jira/browse/LENS-836


Repository: lens


Description
---


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
fe9b84dd5574b922c6b8a16a1b8c5b7518aa400a 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 
5fd04c4591264ac2e5082f1a60281b15fac9 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 
40e6d76d20dbb1359d4d6dc64ff7c41f324c7662 

Diff: https://reviews.apache.org/r/41033/diff/


Testing (updated)
---

http://showterm.io/ec9c4280efd6a72e03143


Thanks,

Rajat Khandelwal

Re: Review Request 41033: LENS-836: Query commands in CLI should take default value for query handle as the last executed query

2015-12-07 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41033/
---

(Updated Dec. 8, 2015, 12:58 p.m.)


Review request for lens.


Bugs: LENS-836
https://issues.apache.org/jira/browse/LENS-836


Repository: lens


Description
---


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
fe9b84dd5574b922c6b8a16a1b8c5b7518aa400a 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 
5fd04c4591264ac2e5082f1a60281b15fac9 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 
40e6d76d20dbb1359d4d6dc64ff7c41f324c7662 
  
lens-cube/src/test/java/org/apache/lens/cube/metadata/timeline/TestPartitionTimelines.java
 3f9e1258536da5955ed6a846fbe25ce074547625 

Diff: https://reviews.apache.org/r/41033/diff/


Testing (updated)
---

lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java


Thanks,

Rajat Khandelwal

[jira] [Work started] (LENS-890) Allow per-queue driver max launched queries constraints

2015-12-07 Thread Rajat Khandelwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-890 started by Rajat Khandelwal.
-
> Allow per-queue driver max launched queries constraints
> ---
>
> Key: LENS-890
> URL: https://issues.apache.org/jira/browse/LENS-890
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (LENS-890) Allow per-queue driver max launched queries constraints

2015-12-07 Thread Rajat Khandelwal (JIRA) 
Rajat Khandelwal created LENS-890:
-

 Summary: Allow per-queue driver max launched queries constraints
 Key: LENS-890
 URL: https://issues.apache.org/jira/browse/LENS-890
 Project: Apache Lens
  Issue Type: Improvement
Reporter: Rajat Khandelwal
Assignee: Rajat Khandelwal






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 41039: LENS-890: Allow per-queue driver max launched queries constraints

2015-12-07 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41039/
---

Review request for lens.


Bugs: LENS-890
https://issues.apache.org/jira/browse/LENS-890


Repository: lens


Description
---


Diffs
-

  lens-api/src/main/java/org/apache/lens/api/util/CommonUtils.java 
38d58c710a2b917c34cda5de96a36dc49b6f15f0 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
 2f20113a1a30f902a1db4b25d5a78da0126a1b0c 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
 bae2e6490aafbfd55c9983c94d9cc892af384f7b 
  
lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java
 b6e6c2f1bd2e4f3c0acdcdaae57dcd812e6bb435 
  
lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java
 55a2eea110f9001c962faede01cef396e8c91b6c 
  
lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java
 9138f8ea2bb0b87bfcd2c9a4770bc732856fa680 

Diff: https://reviews.apache.org/r/41039/diff/


Testing
---


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-890) Allow per-queue driver max launched queries constraints

2015-12-07 Thread Rajat Khandelwal (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045041#comment-15045041
 ] 

Rajat Khandelwal commented on LENS-890:
---

Created https://reviews.apache.org/r/41039/

> Allow per-queue driver max launched queries constraints
> ---
>
> Key: LENS-890
> URL: https://issues.apache.org/jira/browse/LENS-890
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 41035: LENS-791: CLI format json strips away all commas, including useful ones

2015-12-07 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41035/
---

(Updated Dec. 7, 2015, 6:57 p.m.)


Review request for lens.


Bugs: LENS-791
https://issues.apache.org/jira/browse/LENS-791


Repository: lens


Description
---


Diffs (updated)
-

  lens-api/src/main/java/org/apache/lens/api/LensConf.java 
ff965d6aabf052371e0b3b4644a5f60d1210be3d 
  lens-cli/src/main/java/org/apache/lens/cli/commands/BaseLensCommand.java 
643772596c472425c0688a93ed621b4dba45 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensCRUDCommand.java 
a0c05b41650cd8303879c6c370d892c544adf6ca 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LensConnectionCommands.java 
0aa51a8eeea2ad3e0b43d7f011c50b83b0ba30e4 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LensDimensionCommands.java 
dc8349e322adcb031ef503850e012603f5865088 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
fe9b84dd5574b922c6b8a16a1b8c5b7518aa400a 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LogicalTableCrudCommand.java
 a04d5c836a31b625e0efcd5509d999d6e483cbb6 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 
5fd04c4591264ac2e5082f1a60281b15fac9 

Diff: https://reviews.apache.org/r/41035/diff/


Testing (updated)
---

lens-shell>describe dimtable dim_table

columns :
column :
   name : id  type : int  comment : ID
   name : name  type : string  comment : name
   name : detail  type : string  comment : more details
   name : d2id  type : int  comment : d2 ID


properties :
property :
   name : cube.table.type  value : DIM_TABLE
   name : dimtble.dim_table.local.dumpperiod  value : HOURLY
   name : dim1.prop  value : d1
   name : cube.table.dim_table.weight  value : 100.0
   name : dimtble.dim_table.storages  value : local
   name : transient_lastDdlTime  value : 1449487545
   name : dimtble.dim_table.dim.name  value : sample_dim


storageTables :
storageTable :
   updatePeriods :   updatePeriod :  HOURLY
storageName : local  tableDesc :   partCols :   column :
   name : dt  type : string  comment : Time column
tableParameters :   property :
   name : EXTERNAL  value : TRUE name : cube.storagetable.time.partcols  
value : dt name : transient_lastDdlTime  value : 1449487545
serdeParameters :   property :
   name : serialization.format  value :  name : field.delim  value :
timePartCols :  dt
  external : true  tableLocation : file:/tmp/examples/dim1  inputFormat : 
org.apache.hadoop.mapred.TextInputFormat  outputFormat : 
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat  fieldDelimiter :   
serdeClassName : org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe  
storageHandlerName :   numBuckets : 0  compressed : false


dimensionName : sample_dim
tableName : dim_table
weight : 100.0

lens-shell>set lens.cli.json.pretty=true
Client side Set lens.cli.json.pretty=true
lens-shell>describe dimtable dim_table
{
"columns" : {
"column" : [ {  "name" : "id",  "type" : "int",  "comment" : "ID"
}, {  "name" : "name",  "type" : "string",  "comment" : "name"
}, {  "name" : "detail",  "type" : "string",  "comment" : "more details"
}, {  "name" : "d2id",  "type" : "int",  "comment" : "d2 ID"
} ]
},
"properties" : {
"property" : [ {  "name" : "cube.table.type",  "value" : "DIM_TABLE"
}, {  "name" : "dimtble.dim_table.local.dumpperiod",  "value" : "HOURLY"
}, {  "name" : "dim1.prop",  "value" : "d1"
}, {  "name" : "cube.table.dim_table.weight",  "value" : "100.0"
}, {  "name" : "dimtble.dim_table.storages",  "value" : "local"
}, {  "name" : "transient_lastDdlTime",  "value" : "1449487545"
}, {  "name" : "dimtble.dim_table.dim.name",  "value" : "sample_dim"
} ]
},
"storageTables" : {
"storageTable" : [ {  "updatePeriods" : {  "updatePeriod" : [ "HOURLY" ]  },  
"storageName" : "local",  "tableDesc" : {  "partCols" : {  "column" : [ {  
"name" : "dt",  "type" : "string",  "comment" : "Time column"  } ]  },  
"tableParameters" : {  "property" : [ {  "name" : "EXTERNAL",  "value" : "TRUE" 
 }, {  "name" : "cube.storagetable.time.partcols",  "value" : "dt"  }, {  
"name" : "transient_lastDdlTime",  "value" : "1449487545"  } ]  },  
"serdeParameters" : {  "property" : [ {  "name" : "serialization.format",  
"value" : ","  }, {  "name" : "field.delim",  "value" : ","  } ]  },  
"timePartCols" : [ "dt" ],  "external" : true,  "tableLocation" : 
"file:/tmp/examples/dim1",  "inputFormat" : 
"org.apache.hadoop.mapred.TextInputFormat",  "outputFormat" : 
"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",  "fieldDelimiter" 
: ",",  "serdeClassName" : 
"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe",  "storageHandlerName" : 
"",  "numBuckets" : 0,  "compressed" : false  }
} ]
},
"dimensionName" : "sample_dim",
"tableName" : "dim_table",
"weight" : 100.0
}


[INFO]

Review Request 41035: LENS-791: CLI format json strips away all commas, including useful ones

2015-12-07 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41035/
---

Review request for lens.


Bugs: LENS-791
https://issues.apache.org/jira/browse/LENS-791


Repository: lens


Description
---


Diffs
-

  lens-api/src/main/java/org/apache/lens/api/LensConf.java 
ff965d6aabf052371e0b3b4644a5f60d1210be3d 
  lens-cli/src/main/java/org/apache/lens/cli/commands/BaseLensCommand.java 
643772596c472425c0688a93ed621b4dba45 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensCRUDCommand.java 
a0c05b41650cd8303879c6c370d892c544adf6ca 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LensConnectionCommands.java 
0aa51a8eeea2ad3e0b43d7f011c50b83b0ba30e4 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LensDimensionCommands.java 
dc8349e322adcb031ef503850e012603f5865088 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
fe9b84dd5574b922c6b8a16a1b8c5b7518aa400a 
  
lens-cli/src/main/java/org/apache/lens/cli/commands/LogicalTableCrudCommand.java
 a04d5c836a31b625e0efcd5509d999d6e483cbb6 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 
5fd04c4591264ac2e5082f1a60281b15fac9 

Diff: https://reviews.apache.org/r/41035/diff/


Testing
---


Thanks,

Rajat Khandelwal

[jira] [Commented] (LENS-791) CLI format json strips away all commas, including useful ones

2015-12-07 Thread Rajat Khandelwal (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044900#comment-15044900
 ] 

Rajat Khandelwal commented on LENS-791:
---

Created https://reviews.apache.org/r/41035/

> CLI format json strips away all commas, including useful ones
> -
>
> Key: LENS-791
> URL: https://issues.apache.org/jira/browse/LENS-791
> Project: Apache Lens
>  Issue Type: Bug
>  Components: client
>Reporter: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 40751: LENS - 882: provide option from CLI as well to list queries by driver

2015-12-07 Thread Deepak Barr 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40751/
---

(Updated Dec. 7, 2015, 8:41 p.m.)


Review request for lens.


Changes
---

Updated Cli doc.


Bugs: LENS-882
https://issues.apache.org/jira/browse/LENS-882


Repository: lens


Description
---

After this change,  a CLI user will be able to find a list of queries that ran 
on a particular driver.
eg. lens-shell>query list --driver org.apache.lens.driver.hive.HiveDriver


Diffs (updated)
-

  lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
fe9b84d 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensQueryCommands.java 48b3ebb 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java 5fd04c4 
  lens-client/src/main/java/org/apache/lens/client/LensStatement.java 40e6d76 
  src/site/apt/user/cli.apt c266c6d 

Diff: https://reviews.apache.org/r/40751/diff/


Testing
---

Yes.

INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules .. SUCCESS [  2.401 s]
[INFO] Lens ... SUCCESS [  4.075 s]
[INFO] Lens API ... SUCCESS [ 19.117 s]
[INFO] Lens API for server and extensions . SUCCESS [ 16.010 s]
[INFO] Lens Cube .. SUCCESS [04:54 min]
[INFO] Lens DB storage  SUCCESS [ 15.088 s]
[INFO] Lens Query Library . SUCCESS [ 11.770 s]
[INFO] Lens Hive Driver ... SUCCESS [02:24 min]
[INFO] Lens Driver for JDBC ... SUCCESS [ 29.353 s]
[INFO] Lens Elastic Search Driver . SUCCESS [ 14.007 s]
[INFO] Lens Server  SUCCESS [12:05 min]
[INFO] Lens client  SUCCESS [ 29.389 s]
[INFO] Lens CLI ... SUCCESS [ 44.601 s]
[INFO] Lens Examples .. SUCCESS [  6.650 s]
[INFO] Lens Ship Jars to Distributed Cache  SUCCESS [  0.863 s]
[INFO] Lens Distribution .. SUCCESS [ 10.363 s]
[INFO] Lens ML Lib  SUCCESS [01:21 min]
[INFO] Lens ML Ext Distribution ... SUCCESS [  3.111 s]
[INFO] Lens Regression  SUCCESS [ 10.240 s]
[INFO] Lens UI  SUCCESS [  3.515 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 24:26 min
[INFO] Finished at: 2015-11-26T18:04:55+05:30
[INFO] Final Memory: 143M/857M
[INFO] 


Thanks,

Deepak Barr

Re: Review Request 41039: LENS-890: Allow per-queue driver max launched queries constraints

2015-12-07 Thread Puneet Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41039/#review109253
---



lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
 (line 44)


Shall we make this be a separate constraint and not part of this 
constraint? 

MaxConcurrentQueriesPerQueueConstraint will only be used if its configured 
( say for Hive Driver and not for JDBC)


- Puneet Gupta


On Dec. 7, 2015, 2:58 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41039/
> ---
> 
> (Updated Dec. 7, 2015, 2:58 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-890
> https://issues.apache.org/jira/browse/LENS-890
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/java/org/apache/lens/api/util/CommonUtils.java 
> 38d58c710a2b917c34cda5de96a36dc49b6f15f0 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java
>  2f20113a1a30f902a1db4b25d5a78da0126a1b0c 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java
>  bae2e6490aafbfd55c9983c94d9cc892af384f7b 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java
>  b6e6c2f1bd2e4f3c0acdcdaae57dcd812e6bb435 
>   
> lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java
>  55a2eea110f9001c962faede01cef396e8c91b6c 
>   
> lens-server/src/test/java/org/apache/lens/server/query/constraint/ThreadSafeEstimatedQueryCollectionTest.java
>  9138f8ea2bb0b87bfcd2c9a4770bc732856fa680 
> 
> Diff: https://reviews.apache.org/r/41039/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 41033: LENS-836: Query commands in CLI should take default value for query handle as the last executed query

2015-12-07 Thread Puneet Gupta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41033/#review109248
---



lens-client/src/main/java/org/apache/lens/client/LensClient.java (line 61)


Some methods are not updating this variable. 

Example, if we execute  query1 followed by query2 and then call get status 
on query1, what will lastLesnStatement point to ? 

Should get status / get details /etc also update the last statement ?


- Puneet Gupta


On Dec. 7, 2015, 11:13 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41033/
> ---
> 
> (Updated Dec. 7, 2015, 11:13 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-836
> https://issues.apache.org/jira/browse/LENS-836
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java 
> fe9b84dd5574b922c6b8a16a1b8c5b7518aa400a 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java 
> 5fd04c4591264ac2e5082f1a60281b15fac9 
> 
> Diff: https://reviews.apache.org/r/41033/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>