RoanHeNaN opened a new pull request, #70217:
URL: https://github.com/apache/doris/pull/70217
## Proposed changes
Issue Number: related #70206
### Problem
For a **MOW unique-key table with a sequence column**, a delete at sequence
`S`
must suppress every later out-of-order write of the same key whose sequence
`< S`.
Base/full compaction, however, physically prunes the delete-sign tombstone
(`enable_prune_delete_sign_when_base_compaction`, default `true`) **without
checking whether the table has a sequence column**. Once the tombstone — and
the
sequence value `S` it carries — is gone, `lookup_row_key` returns
`KEY_NOT_FOUND`
for a later lower-sequence write and that row becomes visible,
**resurrecting the
deleted key**.
- Correctness bug on a single copy (including cloud mode with no replicas).
- In local mode, a **permanent divergence** between replicas that compact at
different times, because the delete bitmap is computed against physical
data at
write time.
Reproduction (sequence column):
1. delete `k=1` with a high sequence (`seq=1000`) via delete sign
2. base/full compaction prunes the tombstone
3. a late write for `k=1` arrives with a lower sequence (`seq=900`)
Expected: `k=1` stays deleted. Actual: `k=1` is resurrected.
### Fix: grace watermark
Do not change any resolution logic — only **defer physical reclamation** of
the
tombstone until we can be confident no earlier (lower-sequence) write can
still
arrive (same idea as Cassandra `gc_grace_seconds` / Elasticsearch
`gc_deletes`).
In `TabletReader::_init_delete_condition`, for tables **with a sequence
column**,
only allow delete-sign pruning once the newest input rowset of the
compaction is
older than the grace window:
```
new BE config: mow_sequence_delete_sign_grace_seconds (mutable, default 0)
>0 : keep tombstones at least this many seconds
0 : prune immediately (default, legacy behavior, no change)
<0 : never prune delete-sign tombstones for sequence-column tables
```
Why the change is small: while the tombstone is physically alive, the
existing
`lookup_row_key` + sequence comparison already suppress the late write, so
the
read path and delete-bitmap calculation are untouched. Non-sequence MOW
tables
are unaffected (version order already makes immediate pruning safe). Both
local
and cloud compaction funnel through `Merger::vmerge_rowsets` →
`_init_delete_condition`, so one gate covers both.
It is **fail-safe**: an aggressive value only keeps tombstones longer (space
cost), it never changes a correctness decision.
### Verification
Added
`regression-test/suites/compaction/test_mow_seq_delete_sign_grace.groovy`,
run on a local single-node cluster built from this branch:
```
grace=0 -> k=1 visible (resurrected -> reproduces the bug)
grace=3600 -> k=1 deleted (suppressed -> the fix)
grace=-1 -> k=1 deleted (never prune -> fallback)
Test 1 suites, failed 0 suites
```
### Assumptions / limitations
- The sequence column must be roughly correlated with wall-clock time (e.g.
a CDC
event time); the grace window is a time window. For sequences unrelated to
time,
use `grace < 0` (never prune).
### Follow-ups (not in this PR)
- Expose as a table property instead of a BE-global config.
- Cover `enable_delete_when_cumu_compaction`.
- Default value discussion (keep `0` vs. a conservative positive default).
## Further comments
This is scoped as a minimal, opt-in (default-off) BE change. Marked as
**draft**
for maintainer review of the direction before productization. Feedback on
whether
a grace watermark vs. persisting a per-key sequence watermark is preferred
is very
welcome (see #70206).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
