Rahul Challapalli created DRILL-3966: ----------------------------------------
Summary: Metadata Cache + Partition Pruning not hapenning when the partition column is of type boolean Key: DRILL-3966 URL: https://issues.apache.org/jira/browse/DRILL-3966 Project: Apache Drill Issue Type: Bug Components: Metadata, Query Planning & Optimization Reporter: Rahul Challapalli git.commit.id.abbrev=19b4b79 I have partitioned parquet files whose partition column is of type boolean. The below plan suggests that pruning did not take place when partitioned column is of type boolean and when metadata exists. However if I get rid of the metadata cache, partition pruning seems to be working fine. Query : {code} explain plan for select * from fewtypes_boolpartition where bool_col = false; 00-00 Screen 00-01 Project(*=[$0]) 00-02 Project(T11¦¦*=[$0]) 00-03 SelectionVectorRemover 00-04 Filter(condition=[=($1, false)]) 00-05 Project(T11¦¦*=[$0], bool_col=[$1]) 00-06 Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=maprfs:///drill/testdata/metadata_caching/fewtypes_boolpartition/0_0_2.parquet], ReadEntryWithPath [path=maprfs:///drill/testdata/metadata_caching/fewtypes_boolpartition/0_0_1.parquet]], selectionRoot=/drill/testdata/metadata_caching/fewtypes_boolpartition, numFiles=2, usedMetadataFile=true, columns=[`*`]]]) {code} Error from the log : {code} WARN o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune partition. java.lang.UnsupportedOperationException: Unsupported type: BIT at org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:451) ~[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96) ~[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:212) ~[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6] at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808) [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6] at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6] at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303) [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.handlers.ExplainHandler.getPlan(ExplainHandler.java:61) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244) [drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] {code} I attached the data sets required. Let me know if you need anything -- This message was sent by Atlassian JIRA (v6.3.4#6332)