aCoder2013 opened a new issue, #68807: URL: https://github.com/apache/doris/issues/68807
### Summary Suspected correctness issue in time-series level-2 compaction: active rowset version ranges overlap after compaction, and affected single-replica tablets become unavailable for Stream Load. This report deliberately uses generic descriptions and synthetic identifiers/ranges. No raw logs, deployment details, or data files are attached. ### Version and relevant settings - Doris 4.x; the exact build is omitted from this public report. - `compaction_policy = time_series` - `time_series_compaction_level_threshold = 2` - Affected tables use a single replica. The exact source corresponding to the running build has not yet been checked. The suspected selection behavior below was identified in an earlier source snapshot; its applicability to the running build remains unconfirmed. ### Observed symptoms - Multiple affected tablets have overlapping version ranges in active `rs_metas`, rather than overlaps only between active and stale rowsets. - A level-2 output rowset spans a wide version interval, while several level-1 rowsets inside that interval remain active. - The level-2 output's row count and size are unexpectedly small relative to the rowsets inside the claimed interval. This suggests, but does not prove, that some intermediate rowsets were not included in the merge. - Affected replicas are marked `isBad=true`; tablet health reports them as unrecoverable. - Stream Load is rejected because the number of eligible replicas is below the required replica count, even though the owning backend is alive. - Failures were observed in older partitions, around the period when level-1 rowsets can become eligible for level-2 compaction. The precise path from the overlapping metadata to the bad-replica state has not yet been established. ### Suspected mechanism Please check whether level-2 selection can skip a level-1 rowset that has not passed the age threshold and then select later eligible rowsets, producing non-contiguous actual compaction inputs. If the output version is constructed from the first input's start version and the last input's end version, it can claim an interval containing rowsets that were never merged. Replacing only the selected inputs could then leave overlapping active ranges. It would be useful to check continuity validation **after policy filtering**, and again before committing the output, rather than only validating the original candidate list. ### Synthetic illustration (not a production reproducer) Assume these consecutive level-1 candidates: | Version range | Age eligibility | | --- | --- | | [2-30] | Eligible | | [31-100] | Not yet eligible | | [101-120] | Eligible | A loop that uses `continue` for the middle candidate can select `[2-30]` and `[101-120]`. If this input passes the remaining guards and compaction is triggered, a first/last-based output range would be `[2-120]`, incorrectly spanning the unmerged `[31-100]`. A small standalone simulation reproduces this selection/range mismatch. This is **not** an end-to-end reproduction against a Doris binary, and these ranges and ages are synthetic. We do not claim that these were the original inputs of an affected tablet. ### Expected behavior - Actual compaction inputs must form a continuous version chain. - Output metadata must represent precisely the versions actually merged. - A non-contiguous input selection must be rejected without publishing or committing the output. - Unselected active rowsets must remain readable without being covered by a newly committed overlapping output. ### Mitigation under consideration Setting `time_series_compaction_level_threshold = 1` to bypass the suspected level-2 path is being considered. It has not been validated as a fix and does not repair already affected tablets. Clearing the replica bad flag alone would not repair an inconsistent version chain. ### Questions for maintainers 1. Is there a known issue or fix for non-contiguous level-2 selection around the rowset age threshold? 2. Which guards ensure continuity of the final filtered inputs and prevent committing overlapping active rowsets? 3. Could empty rowsets, non-monotonic rowset creation times, or concurrent compaction make this path reachable? 4. What supported recovery procedure should be used for an affected single-replica tablet while preserving recoverable data? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
