Fang-Yu Rao created IMPALA-15545:
------------------------------------
Summary: estimatedMissingStats_ is not initialized in CalciteTable
when the underlying table is a Kudu table
Key: IMPALA-15545
URL: https://issues.apache.org/jira/browse/IMPALA-15545
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.5.3
Reporter: Fang-Yu Rao
Assignee: Fang-Yu Rao
We found that
[estimatedMissingStats_|https://github.com/apache/impala/blob/0b78b90/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java#L94-L98]
is not initialized to a non-null value when the underlying table is a Kudu
table. This could be a problem when we try to join a Kudu table with other
tables since in this case,
[getRowCount()|https://github.com/apache/impala/blob/0b78b90/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java#L222-L229]
would be called and we explicitly check that {{estimatedMissingStats_}} should
not be null.
This could be reproduced by the following.
{code:sql}
set planner=calcite;
set fallback_planner=calcite;
create database test_db_02;
create table test_db_02.kudu_tbl_01 (id int primary key, string_col string)
stored as kudu;
# The following hits a NullPointerException.
select k.id, a.id
from test_db_02.kudu_tbl_01 k JOIN functional.alltypestiny a
ON (k.id=a.id);
{code}
The respective stack-trace is also provided.
{code:java}
I20261012 14:23:23.854925 213953 jni-util.cc:335]
3c4420d5d3dd1975:0ac1ebab00000000] java.lang.RuntimeException:
java.lang.NullPointerException
at
org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2540)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2199)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:200)
Caused by: java.lang.NullPointerException
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:903)
at
org.apache.impala.calcite.schema.CalciteTable.getRowCount(CalciteTable.java:227)
at
org.apache.calcite.rel.core.TableScan.estimateRowCount(TableScan.java:97)
at
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:240)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_RowCountHandler.getRowCount_$(Unknown
Source)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_RowCountHandler.getRowCount(Unknown
Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:275)
at
org.apache.impala.calcite.schema.ImpalaRelMdNonCumulativeCost.getScanCost(ImpalaRelMdNonCumulativeCost.java:110)
at
org.apache.impala.calcite.schema.ImpalaRelMdNonCumulativeCost.getNonCumulativeCost(ImpalaRelMdNonCumulativeCost.java:66)
at
org.apache.impala.calcite.schema.ImpalaRelMdCumulativeCost.getCumulativeCost(ImpalaRelMdCumulativeCost.java:58)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost_$(Unknown
Source)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost(Unknown
Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:350)
at
org.apache.impala.calcite.schema.ImpalaRelMdCumulativeCost.getCumulativeCost(ImpalaRelMdCumulativeCost.java:68)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost_$(Unknown
Source)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost(Unknown
Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:350)
at
org.apache.impala.calcite.schema.ImpalaRelMdCumulativeCost.getCumulativeCost(ImpalaRelMdCumulativeCost.java:68)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost_$(Unknown
Source)
at
org.apache.calcite.rel.metadata.janino.GeneratedMetadata_CumulativeCostHandler.getCumulativeCost(Unknown
Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getCumulativeCost(RelMetadataQuery.java:350)
at
org.apache.calcite.rel.rules.LoptSemiJoinOptimizer$FactorCostComparator.compare(LoptSemiJoinOptimizer.java:817)
at
org.apache.calcite.rel.rules.LoptSemiJoinOptimizer$FactorCostComparator.compare(LoptSemiJoinOptimizer.java:811)
at
com.google.common.collect.ComparatorOrdering.compare(ComparatorOrdering.java:40)
at
java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
at java.base/java.util.TimSort.sort(TimSort.java:220)
at java.base/java.util.Arrays.sort(Arrays.java:1233)
at
com.google.common.collect.ImmutableList.sortedCopyOf(ImmutableList.java:347)
at
com.google.common.collect.Ordering.immutableSortedCopy(Ordering.java:889)
at
org.apache.calcite.rel.rules.LoptSemiJoinOptimizer.chooseBestSemiJoin(LoptSemiJoinOptimizer.java:540)
at
org.apache.impala.calcite.rules.ImpalaLoptOptimizeJoinRule.onMatch(ImpalaLoptOptimizeJoinRule.java:158)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:380)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:682)
at
org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:539)
at
org.apache.calcite.plan.hep.HepPlanner.executeRuleInstance(HepPlanner.java:355)
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance$State.execute(HepInstruction.java:178)
at
org.apache.calcite.plan.hep.HepPlanner.lambda$executeProgram$0(HepPlanner.java:323)
at
com.google.common.collect.ImmutableList.forEach(ImmutableList.java:422)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:322)
at
org.apache.calcite.plan.hep.HepProgram$State.execute(HepProgram.java:118)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:317)
at
org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:288)
at
org.apache.impala.calcite.service.CalciteOptimizer.runProgram(CalciteOptimizer.java:369)
at
org.apache.impala.calcite.service.CalciteOptimizer.runJoinProgram(CalciteOptimizer.java:310)
at
org.apache.impala.calcite.service.CalciteOptimizer.createPreImpalaConvertPlan(CalciteOptimizer.java:181)
at
org.apache.impala.calcite.service.CalciteOptimizer.optimize(CalciteOptimizer.java:128)
at
org.apache.impala.calcite.service.CalciteSingleNodePlanner.createSingleNodePlan(CalciteSingleNodePlanner.java:77)
at
org.apache.impala.planner.Planner.createPlanFragments(Planner.java:147)
at org.apache.impala.planner.Planner.createPlans(Planner.java:554)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2132)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3616)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3283)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2671)
at
org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2517)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
