> On Sept. 23, 2014, 6:15 a.m., Jinfeng Ni wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DirPathBuilder.java, > > line 48 > > <https://reviews.apache.org/r/25925/diff/1/?file=701928#file701928line48> > > > > Why do you choose 5 as a magic number here? > > > > Seems to me we will not run into problem, even if we do not specify the > > max number, since the code uses ArrayList() ?
I picked a number that seemed reasonable but probably should bump it up to 10 to accommodate dir0-dir9. I am using the arraylist's set(index) method, hence I create the arraylist with a predefined capacity. > On Sept. 23, 2014, 6:15 a.m., Jinfeng Ni wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DirPathBuilder.java, > > line 175 > > <https://reviews.apache.org/r/25925/diff/1/?file=701928#file701928line175> > > > > Since MAX_NESTED_SUBDIRS = 5, if query has dir7 = "yyy", will this > > query hit assertion error? Yes, but will bump it up. > On Sept. 23, 2014, 6:15 a.m., Jinfeng Ni wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushPartitionFilterIntoScan.java, > > line 83 > > <https://reviews.apache.org/r/25925/diff/1/?file=701929#file701929line83> > > > > Will it better to add a new method (i.e., supportPartitionFilter() ) to > > StoragePlugin, in stead of using the StoragePlugin's name as a criteria > > whether this rule will be fired or not? Agreed...I was thinking about that but took a shortcut. I will add a new method. - Aman ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25925/#review54250 ----------------------------------------------------------- On Sept. 23, 2014, 4:36 a.m., Aman Sinha wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/25925/ > ----------------------------------------------------------- > > (Updated Sept. 23, 2014, 4:36 a.m.) > > > Review request for drill and Jinfeng Ni. > > > Bugs: DRILL-1386 > https://issues.apache.org/jira/browse/DRILL-1386 > > > Repository: drill-git > > > Description > ------- > > > Suppose we have following directory hierarchy for either Parquet or JSON > format: > orders/2012/Jan...Dec, orders/2013/Jan...Dec, orders/2014/Jan...Dec} > In this patch we attempt to push the following types of filters (More types > of filters will be added in the future): > 1. SELECT * FROM <path>/orders WHERE o_custkey = 5 AND dir0 = '2014' > AND dir1 = 'June' > 2. SELECT * FROM <path>/orders WHERE (dir0 = '2013' AND dir1 = 'June') > OR (dir0 = '2014' AND dir1 = 'June') > For (1) dirPath = <path>/orders/2014/June > For (2) there are 2 dirPaths: {<path>/orders/2013/June, > <path>/orders/2014/June} > > A new rule DrillPushPartitionFilterIntoScan has been added which makes use of > a helper class DirPathBuilder. > There are couple of pending issues that need to be resolved based on testing. > > > > Diffs > ----- > > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractFileGroupScan.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/FileGroupScan.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScanRelBase.java > 0934818 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DirPathBuilder.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushPartitionFilterIntoScan.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java > dbb85b2 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java > 5a0cc08 > > exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyGroupScan.java > 8efcd2c > > exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java > 86e5224 > exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java > c54772f > > Diff: https://reviews.apache.org/r/25925/diff/ > > > Testing > ------- > > New manual tests in TestExampleQueries; these are not automated yet due to > the fact that directory hierarchies have to be created. > > > Thanks, > > Aman Sinha > >
