This is an automated email from the ASF dual-hosted git repository.
morrySnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 6f4134cbe42 [fix](test) stabilize the flaky shuffle_left_join
regression test (#65769)
6f4134cbe42 is described below
commit 6f4134cbe426f2385cf2515c3e409b3c2706c8d7
Author: 924060929 <[email protected]>
AuthorDate: Tue Jul 21 14:03:35 2026 +0800
[fix](test) stabilize the flaky shuffle_left_join regression test (#65769)
The `nereids_syntax_p0/distribute/shuffle_left_join` regression test is
flaky.
With the nereids distribute planner enabled, the join in this suite can
be
planned either as `INNER JOIN(PARTITIONED)` or as a left-to-right
`INNER JOIN(BUCKET_SHUFFLE)` — the aggregated left side shuffled onto
the right
table's storage buckets — and the two candidates are chosen by cost.
That cost
is not stable during the test:
- the scan row count is reported asynchronously after the insert, so an
unreported count clamps to 1 (which favors a partitioned shuffle), while
the
real count favors a bucket shuffle;
- the bucket-shuffle downgrade gate (`isBucketShuffleDownGrade`) turns
bucket
shuffle back into a partitioned shuffle when the bucket count is small
relative to the instance count, so the plan also depends on the number
of
backends.
As a result the asserted plan flips between `INNER JOIN(PARTITIONED)`
and
`INNER JOIN(BUCKET_SHUFFLE)` depending on timing and cluster topology.
The suite is meant to assert the left-to-right bucket shuffle, which
saves one
exchange versus a plain partitioned shuffle. This PR restores that
assertion and
makes it deterministic:
- `analyze table ... with sync` fixes the row count instead of racing
the async
tablet report (the count is then read from the collected column stats);
- `set bucket_shuffle_downgrade_ratio=0` removes the dependency on the
number of
backends.
Verified on a 4-backend cluster: the suite now stably produces
`INNER JOIN(PARTITIONED)` for the legacy-planner leg and
`INNER JOIN(BUCKET_SHUFFLE)` for the distribute-planner leg across
parallelism
1/4/8 and both before/after stats reporting; the suite was run 3 times,
all
green.
---
.../distribute/shuffle_left_join.groovy | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git
a/regression-test/suites/nereids_syntax_p0/distribute/shuffle_left_join.groovy
b/regression-test/suites/nereids_syntax_p0/distribute/shuffle_left_join.groovy
index 4ee14eba481..356bf9e5fd4 100644
---
a/regression-test/suites/nereids_syntax_p0/distribute/shuffle_left_join.groovy
+++
b/regression-test/suites/nereids_syntax_p0/distribute/shuffle_left_join.groovy
@@ -18,6 +18,17 @@ import java.util.stream.Collectors
// under the License.
suite("shuffle_left_join") {
+ // The point of this suite is the left-to-right bucket shuffle: with the
nereids distribute
+ // planner on, the aggregated left side is shuffled onto the right table's
storage buckets, so
+ // the join becomes a bucket shuffle join that saves one exchange versus a
plain partitioned
+ // shuffle. The planner chooses between the two candidates by cost, which
depends on the scan
+ // row count and on the bucket-shuffle downgrade gate. Both are pinned
below so the asserted
+ // plan is stable:
+ // - `analyze ... with sync` fixes the row count (otherwise it is
reported asynchronously
+ // after the insert, and the plan flips depending on whether the
report has landed yet);
+ // - `bucket_shuffle_downgrade_ratio=0` disables the downgrade that
turns bucket shuffle back
+ // into a partitioned shuffle when the bucket count is small relative
to the instance count,
+ // which otherwise makes the plan depend on the number of backends.
multi_sql """
drop table if exists test_shuffle_left;
@@ -35,11 +46,14 @@ suite("shuffle_left_join") {
sync;
+ analyze table test_shuffle_left with sync;
+
set enable_nereids_distribute_planner=false;
set enable_pipeline_x_engine=true;
set disable_join_reorder=true;
set enable_local_shuffle=false;
set force_to_local_shuffle=false;
+ set bucket_shuffle_downgrade_ratio=0;
"""
def extractFragment = { String sqlStr, String containsString,
Closure<Integer> checkExchangeNum ->
@@ -95,8 +109,8 @@ suite("shuffle_left_join") {
.collect(Collectors.joining("\n"))
logger.info("Variables:\n${variableString}")
- extractFragment(sqlStr, "INNER JOIN(PARTITIONED)") { exchangeNum ->
- assertTrue(exchangeNum == 2)
+ extractFragment(sqlStr, "INNER JOIN(BUCKET_SHUFFLE)") { exchangeNum ->
+ assertTrue(exchangeNum == 1)
}
explain {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
