[
https://issues.apache.org/jira/browse/DRILL-3639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944805#comment-14944805
]
Ivan Pavlov commented on DRILL-3639:
------------------------------------
Sorry, but I cannot confirm this. Using the latest master, for a table
partitioned by `month` which was casted from string to int, this query:
{noformat} explain plan for select count(*) from dfs.ivan.`drill/p2008` where
`month`=1; {noformat}
results in:
{noformat}
2015-10-06 02:38:49,013 [29ec6958-4361-7842-cb2e-7e1561f713a3:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 17 out of
17 using 16 threads. Time: 1158ms total, 898.990298ms avg, 1156ms max.
2015-10-06 02:38:49,014 [29ec6958-4361-7842-cb2e-7e1561f713a3:foreman] INFO
o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 17 out of
17 using 16 threads. Earliest start: 369.254000 ?s, Latest start: 161749.298000
?s, Average start: 13271.456235 ?s .
2015-10-06 02:38:49,028 [29ec6958-4361-7842-cb2e-7e1561f713a3:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
StringIndexOutOfBoundsException: String index out of range: -6
[Error Id: 3bf20517-422f-4823-87cb-19e560176932 on localhost:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
StringIndexOutOfBoundsException: String index out of range: -6
[Error Id: 3bf20517-422f-4823-87cb-19e560176932 on localhost:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:742)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:841)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:786)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73)
[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:788)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:894)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:255)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_79]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_79]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Internal error: Error while applying
rule PruneScanRule:Filter_On_Scan_Parquet, args
[rel#1089:DrillFilterRel.LOGICAL.ANY([]).[](input=rel#1086:Subset#0.LOGICAL.ANY([]).[],condition==($1,
1)), rel#1102:DrillScanRel.LOGICAL.ANY([]).[](table=[dfs, ivan,
drill/p2008],groupscan=ParquetGroupScan [entries=[ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_4.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_5.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_6.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_7.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_4.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_5.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_6.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_4.parquet]],
selectionRoot=hdfs://localhost:9000/ivan/drill/p2008, numFiles=17,
usedMetadataFile=false, columns=[`*`]])]
... 4 common frames omitted
Caused by: java.lang.AssertionError: Internal error: Error while applying rule
PruneScanRule:Filter_On_Scan_Parquet, args
[rel#1089:DrillFilterRel.LOGICAL.ANY([]).[](input=rel#1086:Subset#0.LOGICAL.ANY([]).[],condition==($1,
1)), rel#1102:DrillScanRel.LOGICAL.ANY([]).[](table=[dfs, ivan,
drill/p2008],groupscan=ParquetGroupScan [entries=[ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_4.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_5.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_6.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_0_7.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_4.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_5.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_1_6.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_1.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_2.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_3.parquet], ReadEntryWithPath
[path=hdfs://localhost:9000/ivan/drill/p2008/1_2_4.parquet]],
selectionRoot=hdfs://localhost:9000/ivan/drill/p2008, numFiles=17,
usedMetadataFile=false, columns=[`*`]])]
at org.apache.calcite.util.Util.newInternal(Util.java:792)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.ExplainHandler.getPlan(ExplainHandler.java:61)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
... 3 common frames omitted
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
range: -6
at java.lang.String.substring(String.java:1875) ~[na:1.7.0_79]
at
org.apache.drill.exec.planner.DFSPartitionLocation.<init>(DFSPartitionLocation.java:31)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.ParquetPartitionDescriptor.createPartitionSublists(ParquetPartitionDescriptor.java:126)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.AbstractPartitionDescriptor.iterator(AbstractPartitionDescriptor.java:53)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:190)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
... 13 common frames omitted
{noformat}
> Internal error: Error while applying rule PruneScanRule:Filter_On_Scan_Parquet
> ------------------------------------------------------------------------------
>
> Key: DRILL-3639
> URL: https://issues.apache.org/jira/browse/DRILL-3639
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.1.0
> Reporter: 徐波
> Assignee: Jinfeng Ni
> Fix For: 1.2.0
>
>
> exception:
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: Internal error: Error while applying rule
> PruneScanRule:Filter_On_Scan_Parquet
> Caused By (java.lang.AssertionError) null
>
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.isBitOn():490
>
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanAnd():434
>
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator():332
>
> org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator():147
> org.apache.drill.common.expression.BooleanOperator.accept():36
> my sql:
> select col1 from hdfs.root.`/dir/table1` where col2<cast('222222222.0' as
> double) and col3=101 limit 10
> col2 is not a partition column and col3 is a partition column, so the
> FindPartitionConditions get a wrong condition like
> 'AND(CAST('222222222.0'):DOUBLE NOT NULL, =($1, 101))'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
