This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 716f193dbd6 [test](regression) Fix p1 suites agg_sync_mv and
test_left_anti_join_batch_size (#70306)
716f193dbd6 is described below
commit 716f193dbd67148adca32bb56cc7ba761f4e1c50
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Nov 6 02:04:02 2026 +0000
[test](regression) Fix p1 suites agg_sync_mv and
test_left_anti_join_batch_size (#70306)
### Background
`nereids_syntax_p1/mv/aggregate/agg_sync_mv` creates one synchronous
materialized view per aggregate function on the same base table and
checks that queries are rewritten to it. An unaliased aggregate in a
sync MV gets a generated column name such as `__stddev_1`, and a new
MV's column names must not repeat any column of the table's full schema,
which includes the columns of its existing sync MVs.
`tpch_unique_sql_zstd_bucket1_p1/sql/test_left_anti_join_batch_size.sql`
runs a `NOT EXISTS` query over TPC-H `lineitem` with the hint
`SET_VAR(batch_size=3)`, so that the left anti join sees many small
blocks; the suite's `load.groovy` creates the tables in the suite
database `regression_test_tpch_unique_sql_zstd_bucket1_p1`.
### What problem does this PR solve?
Issue Number: None
Related PR: #65336 and #65338 (the branch-4.1 fixes by @shuke987, ported
here), #69446 (the branch-4.2 follow-up by @mrhhsg, ported here), #56986
(moved the TPC-H unique suite from p0 to p1), #65538 (fixed the suite's
second load on master)
PR Type: Test
Problem Summary:
Neither suite can pass on master. `agg_sync_mv` stops at `mv_sync41`
with `Duplicate column name '__stddev_1'`, because `stddev(kint)` and
`stddev_pop(kint)` are one function and get one generated name;
`variance` / `var_pop` and `variance_samp` / `var_samp` collide the same
way. Past that point its last check compares `map_agg` output with a
stale golden. `test_left_anti_join_batch_size` reads
`regression_test_tpch_unique_sql_zstd_bucket1_p0.lineitem`, a database
that no suite creates since #56986, and its hint is misspelled
`SEV_VAR`, which Doris ignores. This PR ports the branch-4.1 fixes
(aliases, the suite's own `lineitem`, `SET_VAR`) and branch-4.2's
order-independent rewrite of the last check.
### Details
- **Failure:** `agg_sync_mv` fails on every run at `mv_sync41` with
`errCode = 2, detailMessage = Duplicate column name '__stddev_1'`. With
the aliases alone it fails at its last check, `qt_test`: the golden
expects `{}` for the `kbint = NULL` group, master returns
`{null:"null"}`, and the `kbint = 1` group now holds two entries
(`{null:"string1", 0:"string1"}`). `test_left_anti_join_batch_size`
fails with `Database [regression_test_tpch_unique_sql_zstd_bucket1_p0]
does not exist` on any cluster that has not kept a database from the old
p0 suite.
- **Cause:** In `agg_sync_mv`, `stddev_pop`, `stddev`, and `std` bind to
the `Stddev` function, `var_pop` and `variance` to `Variance`, and
`var_samp` and `variance_samp` to `VarianceSamp`, so the unaliased
outputs of each pair get the same name, and `MaterializedViewHandler`
rejects a name already in the table's full schema. The `qt_test` golden
predates `map_agg` keeping NULL keys, and a map with two entries prints
them in no guaranteed order. In the anti-join case, the database name is
left over from when the suite lived in p0, and `SEV_VAR` is not a hint
name, so `batch_size` keeps its default (`SELECT
/*+SEV_VAR(batch_size=3)*/ @@batch_size` returns 8160; with `SET_VAR` it
returns 3).
- **Stability evidence:** The failures were deterministic, and the fixes
remove their causes: the aliases make the generated names irrelevant,
the rewritten `qt_test` checks the map size, NULL-key presence, and the
value for each key instead of the map's text, and the query reads the
database its own suite loads. Results of the local runs are under
Testing.
### Testing
All runs used `./run-regression-test.sh --run -d <dir> -s <suite>`
against a local master-based cluster.
| Suite | Version | Result |
|---|---|---|
| `agg_sync_mv` | master | fails at `mv_sync41`: `Duplicate column name
'__stddev_1'` |
| `agg_sync_mv` | with the aliases only | fails at `qt_test`: expected
`{}`, got `{null:"null"}` |
| `agg_sync_mv` | this PR | passes |
| `test_left_anti_join_batch_size` | master | fails: database
`regression_test_tpch_unique_sql_zstd_bucket1_p0` does not exist |
| `test_left_anti_join_batch_size` | this PR | passes against the
existing `.out` |
The new `qt_test` block in `agg_sync_mv.out` was generated by the
framework (`-forceGenOut`) on master and equals branch-4.2's block. The
rest of the regenerated file differs from the committed one only in how
doubles print (`0` instead of `0.0`), which the framework compares by
value, so those blocks are left unchanged, as #69446 did. For the
anti-join suite, the full TPC-H fixture download was too slow locally,
so `lineitem` held only the rows with `l_orderkey < 10000`, which are
all the query reads, taken from the start of the sf0.1 fixture file
(sorted by `l_orderkey`).
### Notes for Reviewers
- The second commit carries branch-4.2's #69446; without it
`agg_sync_mv` still fails on master at its last check. Apart from that
block, the `.out` stays as on master, including
`select_group_array_intersect` for the NULL group (`\N [1]`, from
#66284), where branch-4.2 has `\N []`.
- `mv_sync42` (`stddev_samp`) has no synonym MV in the suite; #65336
aliased it too, and this PR keeps that for consistency.
Co-authored-by: shuke <[email protected]>
Co-authored-by: Jerry Hu <[email protected]>
---
.../nereids_syntax_p1/mv/aggregate/agg_sync_mv.out | 26 +++++++++++-----------
.../mv/aggregate/agg_sync_mv.groovy | 19 +++++++++++-----
.../sql/test_left_anti_join_batch_size.sql | 6 ++---
3 files changed, 30 insertions(+), 21 deletions(-)
diff --git
a/regression-test/data/nereids_syntax_p1/mv/aggregate/agg_sync_mv.out
b/regression-test/data/nereids_syntax_p1/mv/aggregate/agg_sync_mv.out
index 33058f7d084..29b32d8f738 100644
--- a/regression-test/data/nereids_syntax_p1/mv/aggregate/agg_sync_mv.out
+++ b/regression-test/data/nereids_syntax_p1/mv/aggregate/agg_sync_mv.out
@@ -1500,17 +1500,17 @@
11 [0, 0]
-- !test --
-\N {}
-1 {0:"string1"}
-2 {1:"string2"}
-3 {2:"string3"}
-4 {3:"string1"}
-5 {4:"string2"}
-6 {5:"string3"}
-7 {6:"string1"}
-8 {7:"string2"}
-9 {8:"string3"}
-10 {9:"string1"}
-11 {10:"string2"}
-12 {11:"string3"}
+\N 1 true null null
+1 2 true string1 string1
+2 1 false \N string2
+3 1 false \N string3
+4 1 false \N string1
+5 1 false \N string2
+6 1 false \N string3
+7 1 false \N string1
+8 1 false \N string2
+9 1 false \N string3
+10 1 false \N string1
+11 1 false \N string2
+12 1 false \N string3
diff --git
a/regression-test/suites/nereids_syntax_p1/mv/aggregate/agg_sync_mv.groovy
b/regression-test/suites/nereids_syntax_p1/mv/aggregate/agg_sync_mv.groovy
index e2b0010e749..12c5842dd19 100644
--- a/regression-test/suites/nereids_syntax_p1/mv/aggregate/agg_sync_mv.groovy
+++ b/regression-test/suites/nereids_syntax_p1/mv/aggregate/agg_sync_mv.groovy
@@ -381,7 +381,7 @@ suite("agg_sync_mv") {
qt_select_stddev """select id, stddev(kint) from agg_mv_test group by id
order by id;"""
sql """drop materialized view if exists mv_sync40 on agg_mv_test;"""
- createMV("""create materialized view mv_sync40 as select id as g2,
stddev(kint) from agg_mv_test group by id order by id;""")
+ createMV("""create materialized view mv_sync40 as select id as g2,
stddev(kint) as g2_stddev from agg_mv_test group by id order by id;""")
mv_rewrite_success("select id, stddev(kint) from agg_mv_test group by id
order by id;", "mv_sync40")
qt_select_stddev_mv """select id, stddev(kint) from agg_mv_test group by
id order by id;"""
@@ -393,7 +393,7 @@ suite("agg_sync_mv") {
qt_select_stddev_samp """select id, stddev_samp(kint) from agg_mv_test
group by id order by id;"""
sql """drop materialized view if exists mv_sync42 on agg_mv_test;"""
- createMV("""create materialized view mv_sync42 as select id as g4,
stddev_samp(kint) from agg_mv_test group by id order by id;""")
+ createMV("""create materialized view mv_sync42 as select id as g4,
stddev_samp(kint) as g4_stddev_samp from agg_mv_test group by id order by
id;""")
mv_rewrite_success("select id, stddev_samp(kint) from agg_mv_test group by
id order by id;", "mv_sync42")
qt_select_stddev_samp_mv """select id, stddev_samp(kint) from agg_mv_test
group by id order by id;"""
@@ -423,7 +423,7 @@ suite("agg_sync_mv") {
qt_select_variance """select id, variance(kint) from agg_mv_test group by
id order by id;"""
sql """drop materialized view if exists mv_sync47 on agg_mv_test;"""
- createMV("""create materialized view mv_sync47 as select id as s3,
variance(kint) from agg_mv_test group by id order by id;""")
+ createMV("""create materialized view mv_sync47 as select id as s3,
variance(kint) as s3_variance from agg_mv_test group by id order by id;""")
mv_rewrite_any_success("select id, variance(kint) from agg_mv_test group
by id order by id;", ["mv_sync47", "mv_sync48"])
qt_select_variance_mv """select id, variance(kint) from agg_mv_test group
by id order by id;"""
@@ -435,7 +435,7 @@ suite("agg_sync_mv") {
qt_select_variance_samp """select id, variance_samp(kint) from agg_mv_test
group by id order by id;"""
sql """drop materialized view if exists mv_sync49 on agg_mv_test;"""
- createMV("""create materialized view mv_sync49 as select id as s5,
variance_samp(kint) from agg_mv_test group by id order by id;""")
+ createMV("""create materialized view mv_sync49 as select id as s5,
variance_samp(kint) as s5_variance_samp from agg_mv_test group by id order by
id;""")
mv_rewrite_success("select id, variance_samp(kint) from agg_mv_test group
by id order by id;", "mv_sync49")
qt_select_variance_samp_mv """select id, variance_samp(kint) from
agg_mv_test group by id order by id;"""
@@ -512,5 +512,14 @@ suite("agg_sync_mv") {
sql "insert into agg_mv_test select * from agg_mv_test;"
sql "set parallel_pipeline_task_num=1"
- qt_test "select kbint, map_agg(id, kstr) from agg_mv_test group by kbint
order by kbint;"
+ qt_test """
+ select kbint, map_size(m), map_contains_key(m, cast(null as int)),
+ m[cast(null as int)], m[cast(kbint - 1 as int)]
+ from (
+ select kbint, map_agg(id, kstr) as m
+ from agg_mv_test
+ group by kbint
+ ) t
+ order by kbint;
+ """
}
diff --git
a/regression-test/suites/tpch_unique_sql_zstd_bucket1_p1/sql/test_left_anti_join_batch_size.sql
b/regression-test/suites/tpch_unique_sql_zstd_bucket1_p1/sql/test_left_anti_join_batch_size.sql
index 6a8dc6edf5c..af67ce10af0 100644
---
a/regression-test/suites/tpch_unique_sql_zstd_bucket1_p1/sql/test_left_anti_join_batch_size.sql
+++
b/regression-test/suites/tpch_unique_sql_zstd_bucket1_p1/sql/test_left_anti_join_batch_size.sql
@@ -1,16 +1,16 @@
-- tables: supplier,lineitem,orders,nation
-SELECT /*+SEV_VAR(batch_size=3)*/
+SELECT /*+SET_VAR(batch_size=3)*/
l1.l_orderkey okey,
l1.l_suppkey skey
FROM
- regression_test_tpch_unique_sql_zstd_bucket1_p0.lineitem l1
+ lineitem l1
WHERE
l1.l_receiptdate > l1.l_commitdate
AND l1.L_ORDERKEY < 10000
AND NOT exists(
SELECT *
FROM
- regression_test_tpch_unique_sql_zstd_bucket1_p0.lineitem l3
+ lineitem l3
WHERE
l3.l_orderkey = l1.l_orderkey
AND l3.l_suppkey <> l1.l_suppkey
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
