Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-18 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/#review107005
---



lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java (line 
48)


Why is this RuntimeException?


- Amareshwari Sriramadasu


On Nov. 18, 2015, 9:28 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39895/
> ---
> 
> (Updated Nov. 18, 2015, 9:28 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-851
> https://issues.apache.org/jira/browse/LENS-851
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Will do further cleanup and refactoring. Putting this up for an early review.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/lens-errors.conf 
> 7526456bf77e26946179d8b52fe7cf75fb175e00 
>   lens-cube/src/main/java/org/apache/lens/cube/error/LensCubeErrorCode.java 
> 2119b6489dbd41967e2dcb0f03fa5c933de8ebf1 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> 87972c87126210a10fcb55c136409fb18deca8c8 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
> 450d17202d7a06214ecd997d4bab68fe8351ab3e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
> 16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
> 067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
>  15a98dd941bd7b4cb68369fb553d6722956430d6 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
> 90058260b36879bbc149c3c33198d3daf1179b36 
>   lens-cube/src/main/resources/olap-query-conf.xml 
> 4c7b7fa61ba5fd269a590547938fc6ab6101d5a3 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 92f50679f2d117b0b5ef3c08c56c1d34ffd5b90d 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
> 8da52635c72dfc5f669002f1c8dd1da1b9712b40 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> 4acd063f4746044afa0e0345faf81adbdda18e0c 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
>  64b1ac66566239f3a16e55fb5174467940fa0818 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java
>  9dcced06b533461c0f2e087709546fe7b66e89c2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
> 8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 
>   src/site/apt/user/olap-query-conf.apt 
> 6606d42084213ae5f4ad51c2552b47c89a6d9df0 
> 
> Diff: https://reviews.apache.org/r/39895/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules . SUCCESS [1.800s]
> [INFO] Lens .. SUCCESS [2.894s]
> [INFO] Lens API .. SUCCESS [23.557s]
> [INFO] Lens API for server and extensions  SUCCESS [22.292s]
> [INFO] Lens Cube . SUCCESS [5:44.876s]
> [INFO] Lens DB storage ... SUCCESS [20.799s]
> [INFO] Lens Query Library  SUCCESS [15.775s]
> [INFO] Lens Hive Driver .. SUCCESS [2:47.769s]
> [INFO] Lens Driver for JDBC .. SUCCESS [36.353s]
> [INFO] Lens Elastic Search Driver  SUCCESS [15.537s]
> [INFO] Lens Server ... SUCCESS [8:22.306s]
> [INFO] Lens client ... SUCCESS [35.382s]
> [INFO] Lens CLI .. SUCCESS [52.107s]
> [INFO] Lens Examples . SUCCESS [9.908s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.759s]
> [INFO] Lens Distribution . SUCCESS [9.286s]
> [INFO] Lens ML Lib ... SUCCESS [1:20.508s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [1.992s]
> [INFO] Lens Regression ... SUCCESS [13.003s]
> [INFO] Lens UI ... SUCCESS [27.497s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 23:06.330s
> [INFO] Finished at: Wed Nov 18 08:07:18 UTC 2015
> [INFO] Final

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-18 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 18, 2015, 4:40 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description (updated)
---

Explained in comments in the code.


Diffs (updated)
-

  lens-api/src/main/resources/lens-errors.conf 
7526456bf77e26946179d8b52fe7cf75fb175e00 
  lens-cube/src/main/java/org/apache/lens/cube/error/LensCubeErrorCode.java 
2119b6489dbd41967e2dcb0f03fa5c933de8ebf1 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
87972c87126210a10fcb55c136409fb18deca8c8 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/main/resources/olap-query-conf.xml 
4c7b7fa61ba5fd269a590547938fc6ab6101d5a3 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
92f50679f2d117b0b5ef3c08c56c1d34ffd5b90d 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
4acd063f4746044afa0e0345faf81adbdda18e0c 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 64b1ac66566239f3a16e55fb5174467940fa0818 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java 
9dcced06b533461c0f2e087709546fe7b66e89c2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 
  src/site/apt/user/olap-query-conf.apt 
6606d42084213ae5f4ad51c2552b47c89a6d9df0 

Diff: https://reviews.apache.org/r/39895/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [1.800s]
[INFO] Lens .. SUCCESS [2.894s]
[INFO] Lens API .. SUCCESS [23.557s]
[INFO] Lens API for server and extensions  SUCCESS [22.292s]
[INFO] Lens Cube . SUCCESS [5:44.876s]
[INFO] Lens DB storage ... SUCCESS [20.799s]
[INFO] Lens Query Library  SUCCESS [15.775s]
[INFO] Lens Hive Driver .. SUCCESS [2:47.769s]
[INFO] Lens Driver for JDBC .. SUCCESS [36.353s]
[INFO] Lens Elastic Search Driver  SUCCESS [15.537s]
[INFO] Lens Server ... SUCCESS [8:22.306s]
[INFO] Lens client ... SUCCESS [35.382s]
[INFO] Lens CLI .. SUCCESS [52.107s]
[INFO] Lens Examples . SUCCESS [9.908s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.759s]
[INFO] Lens Distribution . SUCCESS [9.286s]
[INFO] Lens ML Lib ... SUCCESS [1:20.508s]
[INFO] Lens ML Ext Distribution .. SUCCESS [1.992s]
[INFO] Lens Regression ... SUCCESS [13.003s]
[INFO] Lens UI ... SUCCESS [27.497s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 23:06.330s
[INFO] Finished at: Wed Nov 18 08:07:18 UTC 2015
[INFO] Final Memory: 188M/1161M
[INFO] 


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-18 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/#review106994
---



lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java (line 
54)


Document the config in olap-conf.xml



lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java (line 
422)


Can we fail if rewrite succeeds?


- Amareshwari Sriramadasu


On Nov. 18, 2015, 7:30 a.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39895/
> ---
> 
> (Updated Nov. 18, 2015, 7:30 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-851
> https://issues.apache.org/jira/browse/LENS-851
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Will do further cleanup and refactoring. Putting this up for an early review.
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/lens-errors.conf 
> a582dc266ae98c240489d2fe78fde0887601545e 
>   lens-cube/src/main/java/org/apache/lens/cube/error/LensCubeErrorCode.java 
> 73a584f507444ceceb265e8439fab45d0de7a1b3 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> 87972c87126210a10fcb55c136409fb18deca8c8 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
> 450d17202d7a06214ecd997d4bab68fe8351ab3e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
> 16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
> 067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
>  15a98dd941bd7b4cb68369fb553d6722956430d6 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
> 90058260b36879bbc149c3c33198d3daf1179b36 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 826f6b63cb4b2c3c3be7846bfbd8d0744d1069ad 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
> 8da52635c72dfc5f669002f1c8dd1da1b9712b40 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> 4acd063f4746044afa0e0345faf81adbdda18e0c 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
>  64b1ac66566239f3a16e55fb5174467940fa0818 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java
>  9dcced06b533461c0f2e087709546fe7b66e89c2 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
> 8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 
> 
> Diff: https://reviews.apache.org/r/39895/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules . SUCCESS [2.062s]
> [INFO] Lens .. SUCCESS [3.128s]
> [INFO] Lens API .. SUCCESS [29.618s]
> [INFO] Lens API for server and extensions  SUCCESS [23.581s]
> [INFO] Lens Cube . SUCCESS [5:44.708s]
> [INFO] Lens DB storage ... SUCCESS [20.474s]
> [INFO] Lens Query Library  SUCCESS [16.973s]
> [INFO] Lens Hive Driver .. SUCCESS [2:53.401s]
> [INFO] Lens Driver for JDBC .. SUCCESS [37.230s]
> [INFO] Lens Elastic Search Driver  SUCCESS [17.677s]
> [INFO] Lens Server ... SUCCESS [8:16.106s]
> [INFO] Lens client ... SUCCESS [38.470s]
> [INFO] Lens CLI .. SUCCESS [51.862s]
> [INFO] Lens Examples . SUCCESS [9.145s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.612s]
> [INFO] Lens Distribution . SUCCESS [9.499s]
> [INFO] Lens ML Lib ... SUCCESS [1:20.974s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [2.216s]
> [INFO] Lens Regression ... SUCCESS [14.834s]
> [INFO] Lens UI ... SUCCESS [27.308s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 23:21.839s
> [INFO] Finished at: Tue Nov 17 14:25:05 UTC 2015
> [INFO]

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-17 Thread Rajat Khandelwal 


> On Nov. 17, 2015, 5:27 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java,
> >  line 98
> > 
> >
> > Why does it required copyAST followed by new ASTNode() ?

new ASTNode is shortcut for creating just a single node(not the entire tree) 
based on the root of the constructor argument. This is just creating an 
`ASTNode(CommonToken(TOK_SELECT))`


> On Nov. 17, 2015, 5:27 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java,
> >  line 181
> > 
> >
> > Is intelligence required? :)

No. :)


> On Nov. 17, 2015, 5:27 p.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java, 
> > line 441
> > 
> >
> > Can you share cube query and expected union query in a comment, cannot 
> > make much from looking at testcase.

Signature should be intuitive to understand: 

```
public static String getExpectedUnionQuery(String cubeName, List 
storages, StoragePartitionProvider provider,
String outerSelectPart, String outerWhere, String outerPostWhere, String 
innerQuerySelectPart,
String innerWhere, String innerPostWhere) {
```

Wherever used, the query is supposed to be like: `outerselectpart from 
(innerselectpart from table1 where partitions and innerWhere innerPostWhere 
union all innerselectpart from table2 where partitions and innerWhere 
innerPostWhere) cubeName where outerWhere outerPostWhere`

postwhere is things like `group by`, `order by`, `having`, `limit` etc. 

Mentioning some complex queries here:

```
SELECT ( testcube . alias0 ) as `City ID` , (sum(( testcube . alias1 )) + max(( 
testcube . alias2 ))),  case  when (sum(( testcube . alias1 )) =  0 ) then  0  
else (sum(( testcube . alias3 )) / sum(( testcube . alias1 ))) end  as `Third 
measure`  FROM (SELECT ( testcube . cityid ) as `alias0` , sum(( testcube . 
msr2 )) as `alias1` , max(( testcube . msr3 )) as `alias2` , sum( case  when (( 
testcube . cityid ) =  'x' ) then ( testcube . msr21 ) else ( testcube . msr22 
) end ) as `alias3`  FROM TestQueryRewrite.c1_testfact testcube WHERE 
 testcube . dt ) =  
'2015-09-17-19' ) or (( testcube . dt ) =  '2015-09-17-20' )) or (( testcube . 
dt ) =  '2015-09-17-21' )) or (( testcube . dt ) =  '2015-09-17-22' )) or (( 
testcube . dt ) =  '2015-09-17-23' )) or (( testcube . dt ) =  '2015-09-18' )) 
or (( testcube . dt ) =  '2015-09-19' )) or (( testcube . dt ) =  '2015-09-20' 
)) or (( testcube . dt ) =  '2015-09-21' )) or (( testcube . dt ) 
 =  '2015-09-22' )) or (( testcube . dt ) =  '2015-09-23' )) or (( testcube . 
dt ) =  '2015-09-24' )) or (( testcube . dt ) =  '2015-09-25' )) or (( testcube 
. dt ) =  '2015-09-26' )) or (( testcube . dt ) =  '2015-09-27' )) or (( 
testcube . dt ) =  '2015-09-28' )) or (( testcube . dt ) =  '2015-09-29' )) or 
(( testcube . dt ) =  '2015-09-30' )) or (( testcube . dt ) =  '2015-11-01' )) 
or (( testcube . dt ) =  '2015-11-02' )) or (( testcube . dt ) =  '2015-11-03' 
)) or (( testcube . dt ) =  '2015-11-04' )) or (( testcube . dt ) =  
'2015-11-05' )) or (( testcube . dt ) =  '2015-11-06' )) or (( testcube . dt ) 
=  '2015-11-07' )) or (( testcube . dt ) =  '2015-11-08' )) or (( testcube . dt 
) =  '2015-11-09' )) or (( testcube . dt ) =  '2015-11-10' )) or (( testcube . 
dt ) =  '2015-11-11' )) or (( testcube . dt ) =  '2015-11-12' )) or (( testcube 
. dt ) =  '2015-11-13' )) or (( testcube . dt ) =  '2015-11-14' )) or (( 
testcube . dt ) =  '2015-11-15' )) or (( testcube . dt ) =  '2015-11-1
 6' )) or (( testcube . dt ) =  '2015-11-17-00' )) or (( testcube . dt ) =  
'2015-11-17-01' )) or (( testcube . dt ) =  '2015-11-17-02' )) or (( testcube . 
dt ) =  '2015-11-17-03' )) or (( testcube . dt ) =  '2015-11-17-04' )) or (( 
testcube . dt ) =  '2015-11-17-05' )) or (( testcube . dt ) =  '2015-11-17-06' 
)) or (( testcube . dt ) =  '2015-11-17-07' )) or (( testcube . dt ) =  
'2015-11-17-08' )) or (( testcube . dt ) =  '2015-11-17-09' )) or (( testcube . 
dt ) =  '2015-11-17-10' )) or (( testcube . dt ) =  '2015-11-17-11' )) or (( 
testcube . dt ) =  '2015-11-17-12' )) or (( testcube . dt ) =  '2015-11-17-13' 
)) or (( testcube . dt ) =  '2015-11-17-14' )) or (( testcube . dt ) =  
'2015-11-17-15' )) or (( testcube . dt ) =  '2015-11-17-16' )) or (( testcube . 
dt ) =  '2015-11-17-17' )) or (( testcube . dt ) =  '2015-11-17-18'  GROUP 
BY ( testcube . cityid ) UNION ALL SELECT ( testcube . cityid ) as `alias0` , 
sum(( testcube . msr2 )) as `alias1` , max(( testcube . msr3 )) as `a
 lias2` , sum( case  when (( testcube . cityid ) =  'x' ) then ( testcube . 
msr21

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-17 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 17, 2015, 8 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs
-

  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
826f6b63cb4b2c3c3be7846bfbd8d0744d1069ad 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
4acd063f4746044afa0e0345faf81adbdda18e0c 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 64b1ac66566239f3a16e55fb5174467940fa0818 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java 
9dcced06b533461c0f2e087709546fe7b66e89c2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 

Diff: https://reviews.apache.org/r/39895/diff/


Testing (updated)
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [2.062s]
[INFO] Lens .. SUCCESS [3.128s]
[INFO] Lens API .. SUCCESS [29.618s]
[INFO] Lens API for server and extensions  SUCCESS [23.581s]
[INFO] Lens Cube . SUCCESS [5:44.708s]
[INFO] Lens DB storage ... SUCCESS [20.474s]
[INFO] Lens Query Library  SUCCESS [16.973s]
[INFO] Lens Hive Driver .. SUCCESS [2:53.401s]
[INFO] Lens Driver for JDBC .. SUCCESS [37.230s]
[INFO] Lens Elastic Search Driver  SUCCESS [17.677s]
[INFO] Lens Server ... SUCCESS [8:16.106s]
[INFO] Lens client ... SUCCESS [38.470s]
[INFO] Lens CLI .. SUCCESS [51.862s]
[INFO] Lens Examples . SUCCESS [9.145s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.612s]
[INFO] Lens Distribution . SUCCESS [9.499s]
[INFO] Lens ML Lib ... SUCCESS [1:20.974s]
[INFO] Lens ML Ext Distribution .. SUCCESS [2.216s]
[INFO] Lens Regression ... SUCCESS [14.834s]
[INFO] Lens UI ... SUCCESS [27.308s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 23:21.839s
[INFO] Finished at: Tue Nov 17 14:25:05 UTC 2015
[INFO] Final Memory: 202M/2273M
[INFO] 


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-17 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 17, 2015, 7:31 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
826f6b63cb4b2c3c3be7846bfbd8d0744d1069ad 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
4acd063f4746044afa0e0345faf81adbdda18e0c 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 64b1ac66566239f3a16e55fb5174467940fa0818 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java 
9dcced06b533461c0f2e087709546fe7b66e89c2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 

Diff: https://reviews.apache.org/r/39895/diff/


Testing
---

Tested for some queries in the test cases themselves. Need to add proper 
asserts.


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-17 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 18, 2015, 1 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs (updated)
-

  lens-api/src/main/resources/lens-errors.conf 
a582dc266ae98c240489d2fe78fde0887601545e 
  lens-cube/src/main/java/org/apache/lens/cube/error/LensCubeErrorCode.java 
73a584f507444ceceb265e8439fab45d0de7a1b3 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
87972c87126210a10fcb55c136409fb18deca8c8 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
16e1aa3f3288426e1bc9b823ac8c3d805d54c42f 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
826f6b63cb4b2c3c3be7846bfbd8d0744d1069ad 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
4acd063f4746044afa0e0345faf81adbdda18e0c 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 64b1ac66566239f3a16e55fb5174467940fa0818 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java 
9dcced06b533461c0f2e087709546fe7b66e89c2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 

Diff: https://reviews.apache.org/r/39895/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [2.062s]
[INFO] Lens .. SUCCESS [3.128s]
[INFO] Lens API .. SUCCESS [29.618s]
[INFO] Lens API for server and extensions  SUCCESS [23.581s]
[INFO] Lens Cube . SUCCESS [5:44.708s]
[INFO] Lens DB storage ... SUCCESS [20.474s]
[INFO] Lens Query Library  SUCCESS [16.973s]
[INFO] Lens Hive Driver .. SUCCESS [2:53.401s]
[INFO] Lens Driver for JDBC .. SUCCESS [37.230s]
[INFO] Lens Elastic Search Driver  SUCCESS [17.677s]
[INFO] Lens Server ... SUCCESS [8:16.106s]
[INFO] Lens client ... SUCCESS [38.470s]
[INFO] Lens CLI .. SUCCESS [51.862s]
[INFO] Lens Examples . SUCCESS [9.145s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [1.612s]
[INFO] Lens Distribution . SUCCESS [9.499s]
[INFO] Lens ML Lib ... SUCCESS [1:20.974s]
[INFO] Lens ML Ext Distribution .. SUCCESS [2.216s]
[INFO] Lens Regression ... SUCCESS [14.834s]
[INFO] Lens UI ... SUCCESS [27.308s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 23:21.839s
[INFO] Finished at: Tue Nov 17 14:25:05 UTC 2015
[INFO] Final Memory: 202M/2273M
[INFO] 


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-09 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 9, 2015, 6:52 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
ea9badd8f0c20644ca2b6e82586fa33f599b781e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
dc11b4ce4598bf6d1dc2dc7ad814bd8757ee34fd 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
d7484d83dc0a3f7810ab2109ec882b19b940803b 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 

Diff: https://reviews.apache.org/r/39895/diff/


Testing
---

Tested for some queries in the test cases themselves. Need to add proper 
asserts.


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-09 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 10, 2015, 2:51 a.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
ea9badd8f0c20644ca2b6e82586fa33f599b781e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
dc11b4ce4598bf6d1dc2dc7ad814bd8757ee34fd 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestAggregateResolver.java 
8da52635c72dfc5f669002f1c8dd1da1b9712b40 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
d7484d83dc0a3f7810ab2109ec882b19b940803b 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestDenormalizationResolver.java
 64b1ac66566239f3a16e55fb5174467940fa0818 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/TestExpressionResolver.java 
9dcced06b533461c0f2e087709546fe7b66e89c2 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQuery.java 
8e35ea9bd9cd727cc25ff011a03d6a38a6f0229f 

Diff: https://reviews.apache.org/r/39895/diff/


Testing
---

Tested for some queries in the test cases themselves. Need to add proper 
asserts.


Thanks,

Rajat Khandelwal

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-04 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/#review105048
---



lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java (line 795)


Why is this change done?



lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 (lines 168 - 172)


What is done here? I see similar code in other methods also. Can be 
separated into another method ?


- Amareshwari Sriramadasu


On Nov. 3, 2015, 2 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39895/
> ---
> 
> (Updated Nov. 3, 2015, 2 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-851
> https://issues.apache.org/jira/browse/LENS-851
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Will do further cleanup and refactoring. Putting this up for an early review.
> 
> 
> Diffs
> -
> 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ColumnResolver.java 
> b95595ade6120c0073c286e073715800b6cdd915 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
> 450d17202d7a06214ecd997d4bab68fe8351ab3e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 
> 97088a146729d4d2fb6ed6c55c2d16bfa8c75339 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
> ea9badd8f0c20644ca2b6e82586fa33f599b781e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
> 067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
>  15a98dd941bd7b4cb68369fb553d6722956430d6 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
> 90058260b36879bbc149c3c33198d3daf1179b36 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> d7484d83dc0a3f7810ab2109ec882b19b940803b 
> 
> Diff: https://reviews.apache.org/r/39895/diff/
> 
> 
> Testing
> ---
> 
> Tested for some queries in the test cases themselves. Need to add proper 
> asserts.
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-04 Thread Amareshwari Sriramadasu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/#review105051
---


Can you add tests with some fields selected being expressions, functions around 
selected fields (for dimattributes) ?

- Amareshwari Sriramadasu


On Nov. 3, 2015, 2 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39895/
> ---
> 
> (Updated Nov. 3, 2015, 2 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-851
> https://issues.apache.org/jira/browse/LENS-851
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Will do further cleanup and refactoring. Putting this up for an early review.
> 
> 
> Diffs
> -
> 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/ColumnResolver.java 
> b95595ade6120c0073c286e073715800b6cdd915 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
> 450d17202d7a06214ecd997d4bab68fe8351ab3e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 
> 97088a146729d4d2fb6ed6c55c2d16bfa8c75339 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
> ea9badd8f0c20644ca2b6e82586fa33f599b781e 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
> 067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
>  15a98dd941bd7b4cb68369fb553d6722956430d6 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
> 90058260b36879bbc149c3c33198d3daf1179b36 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> d7484d83dc0a3f7810ab2109ec882b19b940803b 
> 
> Diff: https://reviews.apache.org/r/39895/diff/
> 
> 
> Testing
> ---
> 
> Tested for some queries in the test cases themselves. Need to add proper 
> asserts.
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

Re: Review Request 39895: LENS-851: queries where results of two storage tables of same fact are unioned, the rows should be aggregated

2015-11-03 Thread Rajat Khandelwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39895/
---

(Updated Nov. 3, 2015, 7:30 p.m.)


Review request for lens.


Bugs: LENS-851
https://issues.apache.org/jira/browse/LENS-851


Repository: lens


Description (updated)
---

Will do further cleanup and refactoring. Putting this up for an early review.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/parse/ColumnResolver.java 
b95595ade6120c0073c286e073715800b6cdd915 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java 
450d17202d7a06214ecd997d4bab68fe8351ab3e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/GroupbyResolver.java 
97088a146729d4d2fb6ed6c55c2d16bfa8c75339 
  lens-cube/src/main/java/org/apache/lens/cube/parse/HQLParser.java 
ea9badd8f0c20644ca2b6e82586fa33f599b781e 
  lens-cube/src/main/java/org/apache/lens/cube/parse/SimpleHQLContext.java 
067a37a29b536a9ff03aae12a396e7bb99f9ba0f 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java
 15a98dd941bd7b4cb68369fb553d6722956430d6 
  lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java 
90058260b36879bbc149c3c33198d3daf1179b36 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
d7484d83dc0a3f7810ab2109ec882b19b940803b 

Diff: https://reviews.apache.org/r/39895/diff/


Testing (updated)
---

Tested for some queries in the test cases themselves. Need to add proper 
asserts.


Thanks,

Rajat Khandelwal