[
https://issues.apache.org/jira/browse/IMPALA-14569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18042061#comment-18042061
]
ASF subversion and git services commented on IMPALA-14569:
----------------------------------------------------------
Commit d67ab6f11f7a4dd6c246d1cba002e4aaffa26674 in impala's branch
refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d67ab6f11 ]
IMPALA-14569: (addendum) Fix 'partitions' row matching
IMPALA-14569 introduced a test that asserts for a profile row like
'HDFS partitions' and it's possible for test environments to run on a
different storage system. This change omits the storage type from the
row_regex.
Change-Id: If9b223f2be2dfe7be8724423fefdfb56ffeeba6e
Reviewed-on: http://gerrit.cloudera.org:8080/23727
Reviewed-by: Riza Suminto <[email protected]>
Tested-by: Riza Suminto <[email protected]>
> Partition pruning fails with IllegalStateException on mismatching types
> -----------------------------------------------------------------------
>
> Key: IMPALA-14569
> URL: https://issues.apache.org/jira/browse/IMPALA-14569
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.5.0
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
>
> HDFS Partition pruning fails with
> {code:java}
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.analysis.LiteralExpr.compareTo(LiteralExpr.java:304)
> at
> org.apache.impala.analysis.DateLiteral.compareTo(DateLiteral.java:120)
> at
> org.apache.impala.analysis.DateLiteral.compareTo(DateLiteral.java:34)
> at java.util.TreeMap.getEntry(TreeMap.java:352)
> at java.util.TreeMap.get(TreeMap.java:278)
> at
> org.apache.impala.planner.HdfsPartitionPruner.evalInPredicate(HdfsPartitionPruner.java:449)
> at
> org.apache.impala.planner.HdfsPartitionPruner.evalSlotBindingFilter(HdfsPartitionPruner.java:510)
> at
> org.apache.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:155)
> at
> org.apache.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1614)
> at
> org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1907)
> at
> org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2239)
> at
> org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:938)
> at
> org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:753)
> at
> org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:285)
> at
> org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:177)
> at
> org.apache.impala.planner.Planner.createPlanFragments(Planner.java:137)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:314)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2041)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3363)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3137)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2545)
> at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2414)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2103)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:179){code}
> or older versions just skip matching partitions, as the comparison between
> different types in 'HdfsPartitionPruner.java:evalInPredicate' returns a false
> result. The problem comes from 'canEvalUsingPartitionMd' as it fails to
> detect the case when the bound slot and the literals in the IN clause have
> different types.
> Steps to reproduce:
> {code:java}
> create table a(a int) partitioned by (date_stored_as_string string);create
> table b(a int) partitioned by (date_stored_as_date date);insert into a(a,
> date_stored_as_string) values(1, '2025-12-12');insert into b(a,
> date_stored_as_date) values(1, '2025-12-12');select * from b left outer join
> a on date_stored_as_date = date_stored_as_string where date_stored_as_date in
> ( '2025-12-12'); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
