Peter Rozsa created IMPALA-14569:
------------------------------------
Summary: Partition pruning fails with IllegalStateException on
mismatching types
Key: IMPALA-14569
URL: https://issues.apache.org/jira/browse/IMPALA-14569
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.5.0
Reporter: Peter Rozsa
Assignee: Peter Rozsa
HDFS Partition pruning fails with
{code:java}
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:496)
at
org.apache.impala.analysis.LiteralExpr.compareTo(LiteralExpr.java:304)
at
org.apache.impala.analysis.DateLiteral.compareTo(DateLiteral.java:120)
at org.apache.impala.analysis.DateLiteral.compareTo(DateLiteral.java:34)
at java.util.TreeMap.getEntry(TreeMap.java:352)
at java.util.TreeMap.get(TreeMap.java:278)
at
org.apache.impala.planner.HdfsPartitionPruner.evalInPredicate(HdfsPartitionPruner.java:449)
at
org.apache.impala.planner.HdfsPartitionPruner.evalSlotBindingFilter(HdfsPartitionPruner.java:510)
at
org.apache.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:155)
at
org.apache.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1614)
at
org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1907)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2239)
at
org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:938)
at
org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:753)
at
org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:285)
at
org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:177)
at
org.apache.impala.planner.Planner.createPlanFragments(Planner.java:137)
at org.apache.impala.planner.Planner.createPlans(Planner.java:314)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2041)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3363)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3137)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2545)
at
org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2414)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2103)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:179){code}
or older versions just skip matching partitions, as the comparison between
different types in 'HdfsPartitionPruner.java:evalInPredicate' returns a false
result. The problem comes from 'canEvalUsingPartitionMd' as it fails to detect
the case when the bound slot and the literals in the IN clause have different
types.
Steps to reproduce:
{code:java}
create table a(a int) partitioned by (date_stored_as_string string);create
table b(a int) partitioned by (date_stored_as_date date);insert into a(a,
date_stored_as_string) values(1, '2025-12-12');insert into b(a,
date_stored_as_date) values(1, '2025-12-12');select * from b left outer join a
on date_stored_as_date = date_stored_as_string where date_stored_as_date in (
'2025-12-12'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
