This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bb3156d30e6 [fix](topn) Guard TopN filters across Repeat and
AssertNumRows (#70194)
bb3156d30e6 is described below
commit bb3156d30e678d05a770aa741016b18a85f218e3
Author: minghong <[email protected]>
AuthorDate: Fri Nov 6 11:53:53 2026 +0800
[fix](topn) Guard TopN filters across Repeat and AssertNumRows (#70194)
### What problem does this PR solve?
Issue Number: N/A
Related PR: #70100
Problem Summary:
A TopN runtime filter above `UNION ALL` can reach scans below operators
that synthesize NULLs. Another UNION branch can publish a bound before
those scans finish, causing incorrect results:
- **Grouping sets:** rows rejected on their original grouping key can
still contribute to a grouping set that replaces that key with NULL.
Filtering them before Repeat produces incomplete aggregate counts or
loses the grand-total row. For one million input rows, `ORDER BY k NULLS
FIRST LIMIT 1` can return `(NULL, 1024)` instead of `(NULL, 1000000)`.
- **Scalar subqueries:** filtering before AssertNumRows can remove the
scalar's only row, making it synthesize a NULL row instead. A scalar
result of `20` unioned with `10` can consequently become NULL and win
`ASC NULLS FIRST`; `COALESCE(..., -1)` also fails with `NULLS LAST`.
Filtering can additionally remove multiple rows and suppress a required
cardinality error.
Only push a TopN filter through Repeat when every probe input slot is
common to all grouping sets, using the existing
common-grouping-expression helper. Stop pushdown at AssertNumRows to
preserve its input cardinality and empty-input behavior. Keep filtering
on common grouping keys and unaffected UNION branches.
### Release note
Fix incorrect TopN results for grouping sets and scalar subqueries, and
preserve scalar-subquery cardinality errors when TopN runtime filters
are enabled.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test
- [ ] No need to test or manual test
Validation:
- `./build.sh --fe -j16` — successful; zero Checkstyle violations.
- `./run-fe-ut.sh --run
org.apache.doris.nereids.postprocess.TopNRuntimeFilterTest` — 12 tests
passed. The new Repeat test covers 36
grouping-set/expression/sort/NULL-order combinations. The Assert test
covers eight combinations and directly visits the assertion, with a
positive control on its child, so it does not depend on CBO's
scalar-join orientation.
- `./run-regression-test.sh --run -d nereids_p0 -s
topn_filter_grouping_sets,topn_filter_assert_num_rows` — two suites
passed, with 35 result checks and eight expected-cardinality-error
checks. Covers GROUPING SETS/ROLLUP/CUBE, common grouping keys, scalar
one-row/empty/multi-row inputs, COALESCE, and both NULL orders. Expected
output was generated by the regression runner.
- The new Assert regression plan assertion fails on the unpatched FE:
the filter has two scan targets instead of only the unaffected UNION
scan.
- Manual reproduction confirmed incomplete grouping-set counts,
synthetic scalar NULLs, and suppressed scalar-cardinality errors before
the corresponding fixes.
Validation used a single-BE native-OLAP cluster. FE was built from this
branch; BE reused an existing ASAN master build (`7acfd9bfc27`).
- Behavior changed:
- [ ] No.
- [x] Yes. Prevent unsafe scan filtering across Repeat and AssertNumRows
while retaining safe targets.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
---
.../processor/post/TopnFilterPushDownVisitor.java | 20 +++++
.../nereids/postprocess/TopNRuntimeFilterTest.java | 89 ++++++++++++++++++++++
.../nereids_p0/topn_filter_assert_num_rows.out | 49 ++++++++++++
.../data/nereids_p0/topn_filter_grouping_sets.out | 58 ++++++++++++++
.../nereids_p0/topn_filter_assert_num_rows.groovy | 79 +++++++++++++++++++
.../nereids_p0/topn_filter_grouping_sets.groovy | 74 ++++++++++++++++++
6 files changed, 369 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/TopnFilterPushDownVisitor.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/TopnFilterPushDownVisitor.java
index 0efae9d7df8..197f63456c1 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/TopnFilterPushDownVisitor.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/TopnFilterPushDownVisitor.java
@@ -27,6 +27,7 @@ import
org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitors;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.algebra.TopN;
import org.apache.doris.nereids.trees.plans.algebra.Union;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalAssertNumRows;
import org.apache.doris.nereids.trees.plans.physical.PhysicalCTEAnchor;
import org.apache.doris.nereids.trees.plans.physical.PhysicalCTEProducer;
import org.apache.doris.nereids.trees.plans.physical.PhysicalFileScan;
@@ -37,6 +38,7 @@ import
org.apache.doris.nereids.trees.plans.physical.PhysicalOdbcScan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalOlapScan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalProject;
import org.apache.doris.nereids.trees.plans.physical.PhysicalRelation;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalRepeat;
import org.apache.doris.nereids.trees.plans.physical.PhysicalSetOperation;
import org.apache.doris.nereids.trees.plans.physical.PhysicalTopN;
import org.apache.doris.nereids.trees.plans.physical.PhysicalWindow;
@@ -158,6 +160,14 @@ public class TopnFilterPushDownVisitor extends
PlanVisitor<Boolean, PushDownCont
return false;
}
+ @Override
+ public Boolean visitPhysicalAssertNumRows(
+ PhysicalAssertNumRows<? extends Plan> assertNumRows,
PushDownContext ctx) {
+ // Filtering its input can hide a cardinality error or make a scalar
subquery
+ // synthesize a NULL row that did not exist before filtering.
+ return false;
+ }
+
@Override
public Boolean visitPhysicalTopN(PhysicalTopN<? extends Plan> topn,
PushDownContext ctx) {
if (topn.equals(ctx.topn)) {
@@ -166,6 +176,16 @@ public class TopnFilterPushDownVisitor extends
PlanVisitor<Boolean, PushDownCont
return false;
}
+ @Override
+ public Boolean visitPhysicalRepeat(PhysicalRepeat<? extends Plan> repeat,
PushDownContext ctx) {
+ // Repeat erases grouping keys absent from a grouping set to NULL.
Filtering those
+ // keys before Repeat can discard rows that still contribute to the
NULL group.
+ if
(!repeat.getCommonGroupingSetExpressions().containsAll(ctx.probeExpr.getInputSlots()))
{
+ return false;
+ }
+ return repeat.child().accept(this, ctx);
+ }
+
@Override
public Boolean visitPhysicalWindow(PhysicalWindow<? extends Plan> window,
PushDownContext ctx) {
return false;
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/postprocess/TopNRuntimeFilterTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/postprocess/TopNRuntimeFilterTest.java
index 63654f74851..837aa1b45eb 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/postprocess/TopNRuntimeFilterTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/postprocess/TopNRuntimeFilterTest.java
@@ -21,17 +21,22 @@ import org.apache.doris.nereids.datasets.ssb.SSBTestBase;
import org.apache.doris.nereids.glue.translator.PlanTranslatorContext;
import org.apache.doris.nereids.processor.post.PlanPostProcessors;
import org.apache.doris.nereids.processor.post.TopnFilterContext;
+import org.apache.doris.nereids.processor.post.TopnFilterPushDownVisitor;
+import
org.apache.doris.nereids.processor.post.TopnFilterPushDownVisitor.PushDownContext;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.expressions.functions.scalar.Coalesce;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Nullable;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Substring;
import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.SortPhase;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalAssertNumRows;
import
org.apache.doris.nereids.trees.plans.physical.PhysicalLazyMaterializeOlapScan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalPlan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalRelation;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalRepeat;
import org.apache.doris.nereids.trees.plans.physical.PhysicalTopN;
import org.apache.doris.nereids.trees.plans.physical.TopnFilter;
import org.apache.doris.nereids.util.MemoPatternMatchSupported;
@@ -133,6 +138,90 @@ public class TopNRuntimeFilterTest extends SSBTestBase
implements MemoPatternMat
}
}
+ @Test
+ public void testTopNFilterThroughRepeat() {
+ double previousRatio =
connectContext.getSessionVariable().topnFilterRatio;
+ boolean previousPushTopn =
connectContext.getSessionVariable().pushTopnToAgg;
+ String previousDisabledRules = String.join(",",
+
connectContext.getSessionVariable().getDisableNereidsRuleNames());
+ connectContext.getSessionVariable().topnFilterRatio = 1000000;
+ connectContext.getSessionVariable().pushTopnToAgg = false;
+
connectContext.getSessionVariable().setDisableNereidsRules(previousDisabledRules
+ + ",PRUNE_EMPTY_PARTITION,PUSH_DOWN_TOP_N_THROUGH_UNION");
+ try {
+ for (String groupingSets : List.of("((k), ())", "((k), (id))",
"((k, id), (k))")) {
+ for (String orderKey : List.of("k", "k + 1", "coalesce(k,
0)")) {
+ for (String direction : List.of("ASC", "DESC")) {
+ for (String nullOrder : List.of("FIRST", "LAST")) {
+ String sql = "SELECT k, c FROM (SELECT k, count(*)
AS c FROM topn_safety "
+ + "GROUP BY GROUPING SETS " + groupingSets
+ + " UNION ALL SELECT c_custkey, CAST(0 AS
BIGINT) FROM customer) u "
+ + "ORDER BY " + orderKey + " " + direction
+ " NULLS " + nullOrder + " LIMIT 1";
+ PlanChecker checker =
PlanChecker.from(connectContext).analyze(sql).rewrite().implement();
+ PhysicalPlan plan = new
PlanPostProcessors(checker.getCascadesContext())
+ .process(checker.getPhysicalPlan());
+
Assertions.assertTrue(plan.containsType(PhysicalRepeat.class),
plan.treeString());
+ List<TopnFilter> filters =
checker.getCascadesContext()
+ .getTopnFilterContext().getTopnFilters();
+ Assertions.assertFalse(filters.isEmpty(),
plan.treeString());
+ long targetCount = filters.stream().flatMap(filter
-> filter.targets.keySet().stream())
+ .distinct().count();
+ // Only the common grouping key keeps its value
through every Repeat expansion.
+ Assertions.assertEquals(groupingSets.equals("((k,
id), (k))") ? 2 : 1,
+ targetCount, sql + "\n" + filters);
+ }
+ }
+ }
+ }
+ } finally {
+ connectContext.getSessionVariable().topnFilterRatio =
previousRatio;
+ connectContext.getSessionVariable().pushTopnToAgg =
previousPushTopn;
+
connectContext.getSessionVariable().setDisableNereidsRules(previousDisabledRules);
+ }
+ }
+
+ @Test
+ public void testTopNFilterDoesNotCrossAssertNumRows() {
+ double previousRatio =
connectContext.getSessionVariable().topnFilterRatio;
+ String previousDisabledRules = String.join(",",
+
connectContext.getSessionVariable().getDisableNereidsRuleNames());
+ connectContext.getSessionVariable().topnFilterRatio = 1000000;
+
connectContext.getSessionVariable().setDisableNereidsRules(previousDisabledRules
+ + ",PRUNE_EMPTY_PARTITION,PUSH_DOWN_TOP_N_THROUGH_UNION");
+ try {
+ for (String direction : List.of("ASC", "DESC")) {
+ for (boolean nullsFirst : List.of(true, false)) {
+ String sql = "SELECT k FROM (SELECT (SELECT k FROM
topn_safety) AS k "
+ + "UNION ALL SELECT c_custkey FROM customer) u
ORDER BY k " + direction
+ + " NULLS " + (nullsFirst ? "FIRST" : "LAST") + "
LIMIT 1";
+ PlanChecker checker =
PlanChecker.from(connectContext).analyze(sql).rewrite().implement();
+ PhysicalPlan plan = checker.getPhysicalPlan();
+ List<PhysicalAssertNumRows<? extends Plan>> assertions =
plan.collectToList(
+ node -> node instanceof PhysicalAssertNumRows);
+ Assertions.assertEquals(1, assertions.size(),
plan.treeString());
+ PhysicalAssertNumRows<? extends Plan> assertion =
assertions.get(0);
+ List<PhysicalTopN<? extends Plan>> topns =
plan.collectToList(node -> node instanceof PhysicalTopN);
+ Assertions.assertFalse(topns.isEmpty(), plan.treeString());
+ Slot slot = assertion.getOutput().get(0);
+ for (Expression probe : List.of(slot, new Coalesce(slot,
new IntegerLiteral(-1)))) {
+ PushDownContext context = new
PushDownContext(topns.get(0), probe, nullsFirst);
+ TopnFilterContext filters = new TopnFilterContext();
+ TopnFilterPushDownVisitor visitor = new
TopnFilterPushDownVisitor(filters);
+ // Visit Assert directly so the test does not depend
on the scalar join's probe side.
+ Assertions.assertFalse(assertion.accept(visitor,
context), sql);
+
Assertions.assertTrue(filters.getTopnFilters().isEmpty(), sql);
+ // The child accepts this filter; Assert must protect
its row count and empty-row behavior.
+
Assertions.assertTrue(assertion.child().accept(visitor, context), sql);
+
Assertions.assertFalse(filters.getTopnFilters().isEmpty(), sql);
+ }
+ }
+ }
+ } finally {
+ connectContext.getSessionVariable().topnFilterRatio =
previousRatio;
+
connectContext.getSessionVariable().setDisableNereidsRules(previousDisabledRules);
+ }
+ }
+
@Test
public void testTopNFilterSupportsNestedOrderKeyWithSiblingPredicate() {
double previousRatio =
connectContext.getSessionVariable().topnFilterRatio;
diff --git a/regression-test/data/nereids_p0/topn_filter_assert_num_rows.out
b/regression-test/data/nereids_p0/topn_filter_assert_num_rows.out
new file mode 100644
index 00000000000..60fe60d09ba
--- /dev/null
+++ b/regression-test/data/nereids_p0/topn_filter_assert_num_rows.out
@@ -0,0 +1,49 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !enabled --
+10
+
+-- !disabled --
+10
+
+-- !enabled --
+10
+
+-- !disabled --
+10
+
+-- !enabled --
+10
+
+-- !disabled --
+10
+
+-- !enabled --
+10
+
+-- !disabled --
+10
+
+-- !enabled --
+\N
+
+-- !disabled --
+\N
+
+-- !enabled --
+10
+
+-- !disabled --
+10
+
+-- !enabled --
+-1
+
+-- !disabled --
+-1
+
+-- !enabled --
+-1
+
+-- !disabled --
+-1
+
diff --git a/regression-test/data/nereids_p0/topn_filter_grouping_sets.out
b/regression-test/data/nereids_p0/topn_filter_grouping_sets.out
new file mode 100644
index 00000000000..507bd24897a
--- /dev/null
+++ b/regression-test/data/nereids_p0/topn_filter_grouping_sets.out
@@ -0,0 +1,58 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !without_filter --
+\N 1000000
+
+-- !with_filter --
+\N 1000000
+
+-- !common_grouping_key --
+20 \N 1000000
+
diff --git
a/regression-test/suites/nereids_p0/topn_filter_assert_num_rows.groovy
b/regression-test/suites/nereids_p0/topn_filter_assert_num_rows.groovy
new file mode 100644
index 00000000000..e6bf784d077
--- /dev/null
+++ b/regression-test/suites/nereids_p0/topn_filter_assert_num_rows.groovy
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("topn_filter_assert_num_rows") {
+ sql "DROP TABLE IF EXISTS topn_assert_t"
+ sql """CREATE TABLE topn_assert_t (id BIGINT NOT NULL, k INT NULL)
+ DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES("replication_num" = "1")"""
+ sql "DROP TABLE IF EXISTS topn_assert_b"
+ sql """CREATE TABLE topn_assert_b (id BIGINT NOT NULL, k INT NULL)
+ DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES("replication_num" = "1")"""
+ sql "INSERT INTO topn_assert_t SELECT number, 20 FROM numbers('number' =
'1000000')"
+ sql "INSERT INTO topn_assert_b VALUES (0, 10)"
+ // Statistics allow the scalar branch to become the nested loop join's
probe side.
+ sql "ANALYZE TABLE topn_assert_t WITH SYNC"
+ sql "ANALYZE TABLE topn_assert_b WITH SYNC"
+
+ sql "SET enable_sql_cache = false"
+ sql "SET enable_query_cache = false"
+ sql "SET disable_nereids_rules = 'PUSH_DOWN_TOP_N_THROUGH_UNION'"
+ sql "SET force_sort_algorithm = 'heap'"
+ sql "SET batch_size = 1024"
+
+ for (int target : [999999, 1000000]) {
+ for (String expression : ["k", "coalesce(k, -1)"]) {
+ for (String nullOrder : ["FIRST", "LAST"]) {
+ // The scalar returns one row for 999999 and no rows for
1000000.
+ // Filtering its input must not manufacture or suppress its
NULL result.
+ String query = """SELECT ${expression} AS sort_key FROM (
+ SELECT (SELECT k FROM topn_assert_t WHERE abs(id -
${target}) = 0) AS k
+ UNION ALL SELECT k FROM topn_assert_b
+ ) u ORDER BY sort_key NULLS ${nullOrder} LIMIT 1"""
+ sql "SET topn_filter_ratio = 1000000"
+ explain {
+ sql query
+ contains "VASSERT NUMBER OF ROWS"
+ contains "assert number of rows: EQ 1"
+ // Keep the other UNION scan as a target, without reaching
the scalar's scan.
+ multiContains "TOPN OPT:", 1
+ }
+ qt_enabled query
+ sql "SET topn_filter_ratio = 0"
+ qt_disabled query
+ }
+ }
+ }
+
+ for (String expression : ["k", "coalesce(k, -1)"]) {
+ for (String nullOrder : ["FIRST", "LAST"]) {
+ for (int ratio : [0, 1000000]) {
+ sql "SET topn_filter_ratio = ${ratio}"
+ // Both rows must reach Assert: even with NULLS LAST, dropping
them
+ // could turn a required scalar-cardinality error into a
successful query.
+ test {
+ sql """SELECT ${expression} AS sort_key FROM (
+ SELECT (SELECT k FROM topn_assert_t WHERE abs(id -
999999) <= 1) AS k
+ UNION ALL SELECT k FROM topn_assert_b
+ ) u ORDER BY sort_key NULLS ${nullOrder} LIMIT 1"""
+ exception "Expected EQ 1"
+ }
+ }
+ }
+ }
+}
diff --git a/regression-test/suites/nereids_p0/topn_filter_grouping_sets.groovy
b/regression-test/suites/nereids_p0/topn_filter_grouping_sets.groovy
new file mode 100644
index 00000000000..423b16b5335
--- /dev/null
+++ b/regression-test/suites/nereids_p0/topn_filter_grouping_sets.groovy
@@ -0,0 +1,74 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("topn_filter_grouping_sets") {
+ sql "DROP TABLE IF EXISTS topn_repeat_t"
+ sql """CREATE TABLE topn_repeat_t (id BIGINT NOT NULL, k INT NULL)
+ DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES("replication_num" = "1")"""
+ sql "DROP TABLE IF EXISTS topn_repeat_b"
+ sql """CREATE TABLE topn_repeat_b (id BIGINT NOT NULL, k INT NULL)
+ DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES("replication_num" = "1")"""
+ sql "INSERT INTO topn_repeat_t SELECT number, 20 FROM numbers('number' =
'1000000')"
+ sql "INSERT INTO topn_repeat_b VALUES (0, 10)"
+
+ sql "SET enable_sql_cache = false"
+ sql "SET enable_query_cache = false"
+ // Keep a single parent TopN so the small UNION branch can publish a bound
+ // before the other branch has finished reading the aggregate's input.
+ sql "SET disable_nereids_rules = 'PUSH_DOWN_TOP_N_THROUGH_UNION'"
+ sql "SET push_topn_to_agg = false"
+ sql "SET force_sort_algorithm = 'heap'"
+ sql "SET batch_size = 1024"
+ sql "SET topn_filter_ratio = 1000000"
+
+ explain {
+ sql """SELECT k, COUNT(*) FROM topn_repeat_t
+ GROUP BY GROUPING SETS ((k), ()) ORDER BY k NULLS FIRST LIMIT
1"""
+ contains "VREPEAT"
+ notContains "TOPN OPT:"
+ }
+ // A key present in every grouping set can still be filtered at the scan.
+ explain {
+ sql """SELECT k, id, COUNT(*) FROM topn_repeat_t
+ GROUP BY GROUPING SETS ((k, id), (k)) ORDER BY k, id NULLS
FIRST LIMIT 1"""
+ contains "VREPEAT"
+ contains "TOPN OPT:"
+ }
+
+ for (String grouping : ["GROUPING SETS ((k), ())", "ROLLUP(k)",
"CUBE(k)"]) {
+ for (String ordering : ["k ASC NULLS FIRST", "k DESC NULLS FIRST",
"coalesce(k, 0)"]) {
+ String query = """SELECT k, c FROM (
+ SELECT k, COUNT(*) AS c FROM topn_repeat_t GROUP BY
${grouping}
+ UNION ALL
+ SELECT k, CAST(0 AS BIGINT) AS c FROM topn_repeat_b
+ ) u ORDER BY ${ordering} LIMIT 1"""
+ explain {
+ sql query
+ contains "VREPEAT"
+ contains "TOPN OPT:"
+ }
+ sql "SET topn_filter_ratio = 0"
+ qt_without_filter query
+ sql "SET topn_filter_ratio = 1000000"
+ qt_with_filter query
+ }
+ }
+ qt_common_grouping_key """SELECT k, id, COUNT(*) FROM topn_repeat_t
+ GROUP BY GROUPING SETS ((k, id), (k)) ORDER BY k, id NULLS FIRST
LIMIT 1"""
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
