Aleksandr Efimov has posted comments on this change. ( http://gerrit.cloudera.org:8080/24605 )
Change subject: IMPALA-15195: Calcite Planner performance improvements for tpcds ...................................................................... Patch Set 6: (3 comments) http://gerrit.cloudera.org:8080/#/c/24605/6/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java: http://gerrit.cloudera.org:8080/#/c/24605/6/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java@205 PS6, Line 205: RelColumnOrigin originCol = mq_.getColumnOrigin(childRel_, inputRef.getIndex()); > This needs to use relNode, not childRel_. After recursing into the right si Using `relNode` fixes the lookup. One issue with the regression test: `testLeftJoinTwoJoins()` uses `id` columns whose null counts are both zero, so it would also pass with the old `childRel_` lookup. Could we use a right-side column with different null stats so the test fails before this fix? http://gerrit.cloudera.org:8080/#/c/24605/6/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java@274 PS6, Line 274: Double innerRowCount = info.getRowCount(); > This throws for a valid non-equi outer join. With l LEFT JOIN r ON l.x < r. The guard avoids the exception, but this test uses `b.id IS NOT NULL`. The requested `IS NULL` case still falls through to the base-table stats, so `b.id` gets 0% nulls even though the left join can generate nulls. Could we test `b.id IS NULL` and use the default estimate when the outer-join null rate cannot be derived? http://gerrit.cloudera.org:8080/#/c/24605/6/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/FilterSelectivityEstimator.java@282 PS6, Line 282: Double percentage = Math.min(innerRowCount/nonOuterJoinRowCount, 1.0); > This is the matched-row ratio, not the null percentage. With a LEFT JOIN an The one-to-one cases work now, but the one-to-many case is still off. For `alltypes a LEFT JOIN alltypestiny b ON a.bigint_col = b.bigint_col`, the existing stats give 5840 inner rows and a 7300-row outer estimate, so this returns 20%. There are 5840 unmatched left rows and 11680 output rows, so the right column is NULL for 50%. Could we add this case and account for unmatched rows separately from matched pairs? -- To view, visit http://gerrit.cloudera.org:8080/24605 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ee6662dc6500ea6ebd9f859de4df1740c51a058 Gerrit-Change-Number: 24605 Gerrit-PatchSet: 6 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Thu, 06 Aug 2026 05:00:22 +0000 Gerrit-HasComments: Yes
