----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33836/ -----------------------------------------------------------
Review request for drill and Jacques Nadeau. Repository: drill-git Description ------- An implementation of Parquet pushdown filtering for Drill. More details can be found in the JIRA item (DRILL-1950). Diffs ----- contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePlugin.java c10b0abd0d79e94524a7f4ce0043cd63ace60a69 contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveStoragePlugin.java f4baf3b023f83452eea2fccc5b54feeaa262b53c contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoStoragePlugin.java e46d8ec589785d5db1b655dae0857464fa3c9159 exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java bd93206d531699ee4f3fcaff11ce0df143f3032f exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java ca2a0486d7f18fb27a592c0824dbef1a31cb3b70 exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java b98778d2a90eefc297df81739ce945beb1080021 exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 1a8559e54810bd9c678bece13b1f42aad89c00e2 exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractStoragePlugin.java b032fce8763d92e58993b78b48993444cb84e5c3 exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java ef5978c8b1b9c91d4d6a873ff798ae096b0bd1e7 exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java 5d0eed6a8446b02610ccb0e9f2b4a667142b3900 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java c5ca41b160782e6421c6d9f90dbfea555d5d14b8 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FormatPlugin.java 58d5b426937accee0c3ba1752e40bfec8df23e19 exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyFormatPlugin.java 6e1e0cc25109beabd119fbb6aac13a3fea670c55 exec/java-exec/src/main/java/org/apache/drill/exec/store/ischema/InfoSchemaStoragePlugin.java 77c6b9ad0c53fc29e7ed73d9d184f411e3f21308 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/EmptyRowGroupScan.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/EmptyScanBatchCreator.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetCompareFunctionProcessor.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFilterBuilder.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java e204a2c28a39c3a647686f09445af3682185fa5e exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java acac61f35448b2884c60a03830532ccd24bc638a exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetPushDownFilter.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetRowGroupScan.java fd40f417a65921efccae2deb5226e6adf86364c3 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetScanBatchCreator.java c1f815e68945bd454d0b6fd268a17a33b6eda00c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java 9f8357bef94d6bc00392d32a61bf35ca10a8feaa exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetRecordMaterializer.java 720e8be7f5befb3039dccd062230ca686cb5f32e exec/java-exec/src/main/java/parquet/hadoop/FilterPredicateSerializer.java PRE-CREATION exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestFilterPushdown.java PRE-CREATION exec/java-exec/src/test/java/parquet/hadoop/TestFilterPredicateSerializer.java PRE-CREATION exec/java-exec/src/test/resources/parquet/pushdown/0_0_0.parquet PRE-CREATION exec/java-exec/src/test/resources/parquet/pushdown/0_0_1.parquet PRE-CREATION protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java ac1bcbb8761b69dc944180234399c0badc517e4f Diff: https://reviews.apache.org/r/33836/diff/ Testing ------- I have created a number of test cases to test that the filter is correctly pushed down in various scenarios. This also ensures that the pushdown filter is correctly working as it must run through the row group filtering to estimate fewer rows scanned and for the optimizer to pick that as a better plan. Thanks, Adam Gilmore
