> On May 11, 2015, 11:26 a.m., Rajat Khandelwal wrote: > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java, > > line 336 > > <https://reviews.apache.org/r/34033/diff/1/?file=955206#file955206line336> > > > > So if multiple storages have the same partition, they are unioned? > > Sushil Mohanty wrote: > Yes. Need to make the partitions across storage tables mutually exclusive > to avoid duplicate counting. > > Amareshwari Sriramadasu wrote: > We already pick minimal storage tables covering all partitions. That > should remove duplicate partitions. Can you check? > > Sushil Mohanty wrote: > yes. > > Amareshwari Sriramadasu wrote: > Seems this issue still exists with current changes put up. > > For example : > Consider following query with common partitions. > Range : X partitions : X1 + C (common) partitions are in S1 and X2 + C > partitions are in S2. Right now, where clause is written with common > partitions included. > Minimal storage tables covered would give : S1 covers X1+C and S2 covers > X2. If we write the query with this. where clause will correctly include > partitions in storage, without duplicates. > > Consider the following multi range query with multiple storages : > Range1 ( X ): X1 +C -> S1; X2 + C -> S2; > Range2 ( Y ): Y1 +C2 -> S1; Y2+C2 -> S2; > > Minimal storage cover will give : S1 -> X1+Y1+C+C1; S2 -> X2+Y2; Where > clause for range should look at intersection of minimal cover and range > parts. Then it will give > range1's where clause : S1 -> X1+c; S2- > X2; > range2's where clause : S1 -> Y1+c2; S2- > Y2;
Updated patch to handle the above scenario. - Sushil ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34033/#review83213 ----------------------------------------------------------- On May 18, 2015, 11:44 a.m., Sushil Mohanty wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34033/ > ----------------------------------------------------------- > > (Updated May 18, 2015, 11:44 a.m.) > > > Review request for lens and Amareshwari Sriramadasu. > > > Bugs: LENS-204 > https://issues.apache.org/jira/browse/LENS-204 > > > Repository: lens > > > Description > ------- > > Support for union across two or more tables in cube query rewriting. > > > Diffs > ----- > > lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java > 31eb113 > lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryContext.java > 230a2ee > lens-cube/src/main/java/org/apache/lens/cube/parse/DimOnlyHQLContext.java > 785b0e4 > > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactHQLContext.java > e14fcfa > > lens-cube/src/main/java/org/apache/lens/cube/parse/SingleFactMultiStorageHQLContext.java > PRE-CREATION > > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java > bdc9855 > lens-cube/src/main/java/org/apache/lens/cube/parse/UnionHQLContext.java > PRE-CREATION > lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java > 1fe4173 > lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java > 44826bd > > Diff: https://reviews.apache.org/r/34033/diff/ > > > Testing > ------- > > Updated unit tests wrt union support. mvn clean test is running fine. > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [8.026s] > [INFO] Lens .............................................. SUCCESS [0.299s] > [INFO] Lens API .......................................... SUCCESS [17.493s] > [INFO] Lens API for server and extensions ................ SUCCESS [15.690s] > [INFO] Lens Cube ......................................... SUCCESS [4:13.909s] > [INFO] Lens DB storage ................................... SUCCESS [16.124s] > [INFO] Lens Query Library ................................ SUCCESS [10.275s] > [INFO] Lens Hive Driver .................................. SUCCESS [3:55.868s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [27.174s] > [INFO] Lens Server ....................................... SUCCESS > [10:18.860s] > [INFO] Lens client ....................................... SUCCESS [35.250s] > [INFO] Lens CLI .......................................... SUCCESS [5:21.299s] > [INFO] Lens Examples ..................................... SUCCESS [4.412s] > [INFO] Lens Distribution ................................. SUCCESS [2.964s] > [INFO] Lens ML Lib ....................................... SUCCESS [2:22.502s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [6.153s] > [INFO] Lens Regression ................................... SUCCESS [0.909s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 28:38.764s > [INFO] Finished at: Mon May 11 13:59:03 GMT+05:30 2015 > [INFO] Final Memory: 85M/1078M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Sushil Mohanty > >
