This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3bacce9b1ae [test](compaction) Fix test_time_series_compaction_level2
rowset counts with three replicas (#70304)
3bacce9b1ae is described below
commit 3bacce9b1aed1fa04057e29615b2cf9ea9f8f0fb
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Nov 6 01:44:58 2026 +0000
[test](compaction) Fix test_time_series_compaction_level2 rowset counts
with three replicas (#70304)
### Background
With `compaction_policy = time_series` and
`time_series_compaction_level_threshold = 2`, cumulative compaction
first merges loaded rowsets into level-1 rowsets, then merges enough
level-1 rowsets into one level-2 rowset.
`compaction/test_time_series_compaction_level2` (group `nonConcurrent`)
loads 10 rounds of 10 one-row inserts into a one-bucket table, compacts
every replica after each round, and checks the number of rowsets summed
over all replicas that `SHOW TABLETS` lists. It derives `replicaNum`
from `SHOW TABLETS` and accepts 1 or 3; an FE with
`force_olap_table_replication_num = 3` overrides the table's
`replication_num = 1`.
### What problem does this PR solve?
Issue Number: None
Related PR: #65342 (the branch-4.1 fix by @shuke987, ported here),
#45412 (added the suite and these assertions)
PR Type: Test
Problem Summary:
The suite's three rowset-count assertions multiply only the 10 newly
loaded rowsets by `replicaNum` and count the base rowset and the level-1
and level-2 rowsets once, although each replica has its own. With one
replica, as in the PR NonConcurrent pipeline, the numbers happen to be
right; with three replicas the first assertion expects 31 while every
run sees 33. This PR multiplies each expected per-replica count by
`replicaNum`, the same change branch-4.1 made in #65342.
### Details
- **Failure:** With `force_olap_table_replication_num = 3`, the suite
fails on every run at its first check: `assert (rowsetCount == 10 *
replicaNum + i + 1)` with `rowsetCount` 33 and the right side 31. #65342
reported it from a branch-4.1 NonConcurrent build, and it reproduces on
master locally. The PR NonConcurrent pipeline runs the suite with one
replica and passes.
- **Cause:** A wrong expectation in the test. Every replica keeps its
own base rowset `[0-1]`, level-1 rowsets, and level-2 rowset, because
the suite compacts and waits on every replica, and `get_rowset_count`
adds them all up. Per replica the count is `11 + i` before round `i`'s
compaction, 11 after the loop, and 2 after the level-2 merge, so the
totals are those numbers times `replicaNum`. The sibling suite
`test_time_series_compaction_policy` already writes its expectations
this way (`26 * replicaNum`).
- **Stability evidence:** The expectations stay exact counts, so the
result is deterministic. With `replicaNum = 1` the new expressions equal
the old ones term by term. With three replicas the master version fails
and this version passes, and the new final check still catches a missing
level-2 merge (see Testing).
### Testing
All runs used `./run-regression-test.sh --run -d compaction -s
test_time_series_compaction_level2` against a local master-based
cluster.
| Replicas | Suite version | Result |
|---|---|---|
| 1 | this PR | passes; `rowsetCount` 11 to 20 |
| 3 (three BEs on one host, `force_olap_table_replication_num = 3`) |
master | fails at the first check: `rowsetCount` 33, expected 31 |
| 3 | this PR | passes; `rowsetCount` 33 to 60 in steps of 3, then 33
and 6 |
| 3 | this PR with the level-2 debug point not enabled | fails as it
should: `expected 6 rowsets, got 33` |
### Notes for Reviewers
The same branch-4.1 batch also fixed a Groovy scope bug in
`fault_injection_p0/test_full_compaciton_run_status` (#65360); master
already fixed it in #58123, so this PR leaves that suite alone.
### Release note
None
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready
### AI Disclosure
- Tool: Claude Code (Claude Opus 5.5)
- Generated: the port of #65342's assertions, the local verification
runs, and this description
- Human review: not stated
- Prompts: The author asked to continue moving branch-4.2's changes back
to master, where branch-4.1 #65360 and #65342 were recorded as one task,
and chose to verify the three-replica path on a local three-BE cluster
before opening the PR.
Co-authored-by: shuke <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../suites/compaction/test_time_series_compaction_level2.groovy | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/regression-test/suites/compaction/test_time_series_compaction_level2.groovy
b/regression-test/suites/compaction/test_time_series_compaction_level2.groovy
index 83f583ba408..6b165d30003 100644
---
a/regression-test/suites/compaction/test_time_series_compaction_level2.groovy
+++
b/regression-test/suites/compaction/test_time_series_compaction_level2.groovy
@@ -120,20 +120,20 @@ suite("test_time_series_compaction_level2",
"nonConcurrent") {
int rowsetCount = get_rowset_count.call(tablets);
println "rowsetCount: ${rowsetCount}"
- assert (rowsetCount == 10 * replicaNum + i + 1)
+ assert (rowsetCount == (10 + i + 1) * replicaNum)
trigger_cumulative_compaction_on_tablets.call(tablets)
wait_cumulative_compaction_done.call(tablets)
}
int rowsetCount = get_rowset_count.call(tablets);
- assert (rowsetCount == 10 * replicaNum + 1)
+ assert (rowsetCount == (10 + 1) * replicaNum)
trigger_cumulative_compaction_on_tablets.call(tablets)
wait_cumulative_compaction_done.call(tablets)
rowsetCount = get_rowset_count.call(tablets);
- assert (rowsetCount == 1 + 1) : "10 level-1 rowsets should be level-2
compacted into 1; expected 2 rowsets, got ${rowsetCount}"
+ assert (rowsetCount == (1 + 1) * replicaNum) : "10 level-1 rowsets
should be level-2 compacted into 1; expected ${2 * replicaNum} rowsets, got
${rowsetCount}"
} finally {
GetDebugPoint().disableDebugPointForAllBEs("time_series_level2_file_count")
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
