matanper opened a new issue, #70771: URL: https://github.com/apache/doris/issues/70771
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.4. master and branch-4.1 have the same code in `be/src/storage/compaction/cumulative_compaction_time_series_policy.cpp`. ### What's Wrong? A time-series table can stop cumulative compaction permanently on tablets that had already compacted before `time_series_compaction_level_threshold` was raised from 1 to 2. Each affected tablet keeps one of the highest cumulative scores (hundreds of rowsets above its cumulative point) but is never compacted: - `/api/compaction/show` reports `last cumulative status: [E-2000]` (CUMULATIVE_NO_SUITABLE_VERSION) and no successful cumulative compaction. - The cloud tablet manager logs every minute: `highest-scoring tablet not picked for compaction, type=2 tablet_id=... score=209 reason=skipped (recent failure, no suitable version or frozen)`. On one such tablet (cumulative point 6315), the meta-service rowset metas above the cumulative point are: - 5 merged rowsets (`[6315-6342]` … `[6427-6450]`, 1 segment each) with `compaction_level = 0`, written by cumulative compactions that ran before the threshold change; - 203 merged rowsets with `compaction_level = 1`, older than 24 h; - 1 empty single-version rowset from a late load. ### Why In `TimeSeriesCumulativeCompactionPolicy::pick_input_rowsets` (the overload with `last_cumu`): 1. **Level-1 pick:** `first_rowset_iter` skips every leading rowset that is neither a single version nor empty, so the main loop never sees the 208 merged rowsets. It sees only the trailing load. 2. **Level-2 scan:** the `compaction_level >= 2` block stops at the first rowset with `compaction_level() == 0`. The first candidate is a merged rowset that was compacted while the threshold was 1, and `get_compaction_level` returns 0 for every output below threshold 2, so the scan collects nothing. 3. **Lone rowset:** once the trailing load is older than `time_series_compaction_time_threshold_seconds`, condition 3 returns that single non-overlapping rowset. `CloudCumulativeCompaction::pick_rowsets_to_compact` rejects a lone non-overlapping pick with CUMULATIVE_NO_SUITABLE_VERSION, and the level-2 block is after condition 3, so it is never reached. Condition 1 already skips a lone non-overlapping rowset; conditions 2 and 3 do not. Point 2 alone stalls the tablet from the moment the threshold changes. Point 3 also stalls any tablet whose oldest load waiting above level-1 rowsets is a lone non-overlapping rowset past the time threshold. `calc_cumulative_compaction_score` has the same level-0 `break` in its level-2 scan, and `get_compaction_level` would give a level-2 merge that starts at such a rowset the output level 1. ### What You Expected? Raising `time_series_compaction_level_threshold` on an existing table lets the level-2 compaction merge the rowsets that are already merged, and a single trailing load does not block compaction. ### How to Reproduce? As a BE unit test (`cumulative_compaction_time_series_policy_test.cpp`): a tablet with `time_series_compaction_level_threshold = 2`, cumulative point 2, and rowsets above it: - `[2-5]`, `[6-9]`: merged, 1 segment, `compaction_level = 0`, created 2 days ago; - `[10-13]`, `[14-17]`, `[18-21]`: merged, `compaction_level = 1`, created 2 days ago; - optionally `[22-22]`: empty, created 2 hours ago (time threshold 3600 s). With the trailing load, `pick_input_rowsets` returns only `[22-22]`. Without it, it returns nothing. Neither picks the level-2 run. On a cluster: create a time-series table with the default level threshold 1, load and let cumulative compaction merge rowsets, run `ALTER TABLE ... SET ("time_series_compaction_level_threshold" = "2")`, keep loading until level-1 rowsets accumulate, then stop loading except for an occasional single load. ### Anything Else? Proposed fix: 1. Count a merged, non-empty rowset with `compaction_level == 0` (start version != end version, segments > 0) as level 1 in the level-2 scan of both `pick_input_rowsets` and `calc_cumulative_compaction_score`, and in `get_compaction_level`. Below threshold 2 no level is recorded, so such a rowset is already a level-1 result. The level-2 merge then includes it and writes level 2, which moves the cumulative point. 2. In conditions 2 and 3, skip a lone non-overlapping rowset as condition 1 does, and fall through to the level-2 and empty-rowset conditions. We have a patch with unit tests and can open a PR. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
