numinnex commented on code in PR #3828:
URL: https://github.com/apache/iggy/pull/3828#discussion_r3730089250
##########
core/partitions/src/iggy_partition.rs:
##########
@@ -3153,6 +3227,33 @@ where
self.stats.zero_out_all();
self.stats.increment_segments_count(1);
+ // Fence the resident journal instead of clearing it: entries are
+ // consensus history (backup commit walks, repair, retransmission), so
+ // they stay, but every journal-apply path no-ops ops at or below this
+ // floor (see `purge_floor_op`). The write lock held here is the same
+ // one appends take, and the pump is single-threaded, so no op can be
+ // assigned between reading the sequence and installing the floor.
+ self.purge_floor_op = self.consensus.sequencer().current_sequence();
Review Comment:
**Critical.** The fence is derived from the *local* sequencer at *local
apply* time, which is strictly above the purge's own position in the partition
op order. The purge is asynchronous: `PurgeTopicRequest::apply` bumps the
generation and replies Ok (`core/metadata/src/stm/stream.rs:1785`), the
reconciler stages on a later pass (`partition_reconciler.rs:1096`), and the
pump applies here (`shard/src/router.rs:664`). Every op the partition sequencer
assigns in that window lands `<= purge_floor_op`.
So a send the client issued *after* `purge_topic()` returned Ok is either
already flushed and unlinked by the loop at `:3151-3166`, or resident and
fenced at `:2032`, or fenced at `:2364` and answered as success. An
acknowledged write is destroyed with no crash, on a single node. Legacy has no
equivalent window: it purges inline before replying
(`core/server/src/shard/execution.rs:348-354`).
Second symptom from the same line: because each replica reads its own
sequencer at its own reconciler timing, replicas fence *different* op sets. Ops
in `(min_floor, max_floor]` flush on some and not others, so
`offset`/`dirty_offset` diverge and the next live batch is stamped at a
different `base_offset` per replica — the invariant documented at `:2463-2474`
says that cannot happen. The `TODO` at `partition_reconciler.rs:1075` names the
skew but not the acked-write loss.
A replicated fence value fixes the divergence but not the wipe, since the
unlink loop still runs at reconciler timing. Suggest routing the purge through
the *partition* consensus log as a `PurgePartition` prepare, so its own op is
the fence: ops below it deleted, ops above kept, identical on every replica,
and ordered against repair by the same walk. Note a fence carried in the
`PurgeTopic` body cannot work — that is one metadata-group op, while each
partition has its own `VsrConsensus`, sequencer and primary
(`bootstrap.rs:2156-2185`), so one number is wrong for N-1 partitions.
##########
core/partitions/src/iggy_partition.rs:
##########
@@ -3153,6 +3227,33 @@ where
self.stats.zero_out_all();
self.stats.increment_segments_count(1);
+ // Fence the resident journal instead of clearing it: entries are
+ // consensus history (backup commit walks, repair, retransmission), so
+ // they stay, but every journal-apply path no-ops ops at or below this
+ // floor (see `purge_floor_op`). The write lock held here is the same
+ // one appends take, and the pump is single-threaded, so no op can be
+ // assigned between reading the sequence and installing the floor.
+ self.purge_floor_op = self.consensus.sequencer().current_sequence();
+ // The journal's flush accounting and resident poll indexes describe
+ // pre-purge bytes; reset them so the flush threshold counts only
+ // post-purge appends and polls fall back to the (fresh, empty)
+ // segments instead of resolving purged resident entries.
+ self.log.journal_mut().info = JournalInfo::default();
+ self.log
+ .journal()
+ .inner
+ .clear_poll_index(self.purge_floor_op);
+
+ // Last durable step: record the applied generation before the
+ // in-memory marker advances. On a write failure the marker stays old,
+ // the error propagates, and the reconciler retries the whole purge
+ // (idempotent). The reverse order would ack a purge that a crash
+ // then silently undoes: restart would hydrate the old generation,
+ // yet the reconciler believes the purge applied.
+ if let Some(dir) = self.partition_dir() {
+ let path = format!("{dir}/{PURGE_GENERATION_FILE}");
+ persist_purge_generation(&path, generation).await?;
Review Comment:
**Critical.** The comment above claims the retry is idempotent. It is not
idempotent with respect to messages: on `Err` the segments are already wiped,
the floor is installed, `write_lock` is released and `applied_purge_generation`
is unchanged, so the partition goes on accepting and acking sends at offset 0
until the next reconciler pass re-purges and deletes them.
Worse, the loss ratchets. `reconcile_partition_purges` has no backoff gate
(compare `reconcile_partition_teardowns` at `partition_reconciler.rs:800-803`)
and `request_purge_partition` is a bare `try_send`
(`shard/src/lib.rs:1537-1541`), so every ~1s pass re-runs the whole purge and
re-reads `current_sequence()` at `:3236`. The fenced set grows one step per
pass for as long as `persist_purge_generation` keeps failing, destroying
whatever was acked since the previous attempt.
Suggest refusing appends until the generation is durable, and routing
retries through the existing `failure_state` backoff with an attempt cap and a
`warn!`.
Separately on durability ordering: `persist_purge_generation` only
`sync_data()`s the file, and `purge()` fsyncs no directory. The dirent for a
first `purge.gen` is therefore not durable, and the segment unlinks are not
ordered before the applied mark. A power loss can leave `purge.gen = N` durable
with the deletions lost, in which case boot hydrates `applied == committed`,
skips the re-purge, and `load_persisted_segments` recovers the purged segments
permanently. Write-then-mark needs the deletion durable first — suggest
fsyncing the partition dir after the unlink loop and after
`install_empty_segment`, before writing the generation.
##########
core/shard/src/lib.rs:
##########
@@ -3311,8 +3312,47 @@ where
}
let cluster = partition.consensus().cluster();
let self_id = partition.consensus().replica();
+ // Purge convergence gate: while a committed purge is not yet locally
+ // applied, this journal still holds pre-purge entries with NO floor
+ // to fence them (the floor is installed by the purge itself), so
+ // serving now would hand a rejoiner batches the cluster purged.
+ // Defer instead: no RepairDone is sent, the rejoiner's stall retry
+ // re-asks, and the local purge (one reconciler wake away) installs
+ // the floor the fence below serves behind.
+ let namespace = IggyNamespace::from_raw(header.namespace);
+ let committed_purge = self
+ .plane
+ .metadata()
+ .mux_stm
+ .streams()
+ .partition_purge_generation(
+ namespace.stream_id(),
+ namespace.topic_id(),
+ namespace.partition_id(),
+ );
+ if committed_purge > partition.applied_purge_generation() {
+ tracing::debug!(
+ shard = self.id,
+ namespace_raw = header.namespace,
+ committed_purge,
+ applied_purge = partition.applied_purge_generation(),
+ "deferring repair serve until the committed purge applies
locally"
+ );
+ return;
+ }
let to_op = header.to_op.min(partition.consensus().commit_max());
- let retained_from =
partition.log.journal().inner.repair_retained_from();
+ // Purge fence: never serve entries at or below this replica's purge
+ // floor. The journal keeps them (own commit walk), but a rejoiner's
+ // floor died with its process, so served pre-purge batches would
+ // flush right back into its freshly reset segments. Reporting the
+ // floor as the retention start rides the normal `RangeEvicted` path:
+ // the rejoiner moves its commit floor to the purge point instead.
+ let purge_floor = partition.purge_floor_op();
+ let retained_from = match
partition.log.journal().inner.repair_retained_from() {
+ Some(op) => Some(op.max(purge_floor + 1)),
Review Comment:
**Critical — permanent rejoin wedge.** When a purge is the newest partition
op, `retained_from` becomes `purge_floor + 1 > to_op` and the served window is
empty.
Full chain on a quiescent purged partition, where the serving peer's
`purge_floor == sequencer == commit_max == to_op`: `handle_start_view` arms
repair with `from_op=1, to_op=commit_max` (`:3040-3055`, `to_op` frozen there)
→ this line raises `retained_from` → `RangeEvicted(purge_floor+1)` sets
`repair.floor = Some(purge_floor)` (`:3613`, `saturating_sub(1)`) → the serve
loop runs zero iterations so `RepairDone` follows → `complete_repair` clamps
`floor.min(commit_max) == to_op` and `repaired_window_is_offsets_only` returns
false on `floor >= to_op` (`core/partitions/src/iggy_partition.rs:3430`) →
refused. `commit_min` never leaves 0.
There is no exit. `complete_repair` returns at `:3390` with `self.repair`
still armed; the stall sweep reuses the frozen `session.to_op`, never a
refreshed `commit_max` (`:4811-4818`), so the identical window is re-asked
every `repair_retry_ticks` forever; the session clears only on `commit_min >=
session.to_op` (`:3406-3409`); and the partition `RangeEvicted` arm has no
`arm_metadata_transfer` equivalent — that call is metadata-only (`:3571-3582`).
Live traffic does not rescue it either, since the rejoiner's backup gap check
drops op `to_op+1`. The cost is a group running one replica short indefinitely,
so every commit's latency becomes the surviving backup's p99 with no hedging.
The comment at `:3348-3350` says "the rejoiner moves its commit floor to the
purge point instead" — the code refuses to.
On the fix: please do not simply exempt the purge floor from the
empty-window rule, since that deletes the guard at
`iggy_partition.rs:3368-3375` and lets a rejoiner still holding pre-purge
segments claim them as committed state and then serve them. Gate acceptance on
the rejoiner having applied the purge itself (`applied_purge_generation >=
committed`), which sets `recovered_durable_offset = None` and wipes the
segments, making the empty window true rather than assumed. Also worth
mirroring the metadata branch's explicit `from_op > to_op` reply (`:3234-3259`)
— the partition branch reaching `RangeEvicted` + `RepairDone` here is
incidental, not intended.
##########
core/partitions/src/iggy_partition.rs:
##########
@@ -2302,6 +2356,14 @@ where
if entry.header.operation != Operation::SendMessages {
return None;
}
+ // Purge floor: a pre-purge send committing after the purge
+ // reports no visible offsets ("send without confirmation", the
+ // established degradation), which also keeps
+ // `commit_partition_entry` from re-advancing the reset offset
+ // and stats with pre-purge values.
+ if entry.header.op <= self.purge_floor_op {
Review Comment:
**Critical.** A fenced `SendMessages` returns the empty body at `:3592`
(`Bytes::from_static(&[0,0,0,0])`), which `SendMessagesResponse::decode` turns
into `confirmations: vec![]`
(`core/binary_protocol/src/responses/messages/send_messages.rs:121-136`) —
indistinguishable from success at the SDK. The client is told Ok for a write
the server discarded.
The root problem is a semantic overload: before this change an empty
confirmation list meant "committed, offsets not resolvable" (`:2235-2239`), and
the wire doc already flags the ambiguity (`send_messages.rs:94-96`). This adds
"silently discarded" to the same shape.
Suggest replying a nonzero reply-header `status =
IggyError::TransientNotCommitted` (57) instead. It is compat-safe (status is
pre-existing wire, not a body change), all three transports already replay it
on the same connection with the same request id
(`core/sdk/src/tcp/tcp_client.rs:976-990`, `quic_client.rs:776-790`,
`websocket_client.rs:777`), and it converges because the replayed send gets a
fresh op above the floor. There is no double-apply risk here: the fenced batch
was never stored, so the replay is not a duplicate. If the floor is climbing
(see the retry comment on `:3255`), the replay exhausts its budget into a
visible timeout rather than a silent success, which is still the better failure.
Note this closes the fenced-at-commit window only. The variant where the
send committed and was answered with a real offset-bearing
`SendMessagesConfirmationResponse` *before* the purge applied, whose bytes the
unlink loop then deletes, cannot be reached by any reply-status change — that
one needs the ordering fix on `:3236`.
##########
core/partitions/src/iggy_partition.rs:
##########
@@ -1984,6 +2025,13 @@ where
}
continue;
}
+ // Purge floor: a pre-purge batch committing after the
+ // purge must not flush its (purged) bytes into the fresh
+ // segment. It still counts into `chunk_len`, so it joins
+ // the evictable prefix and commit_min advances normally.
+ if peek_op(&entry) <= self.purge_floor_op {
Review Comment:
**Critical (receiving side, distinct from the fence itself).** Pre-existing
mechanism, newly reachable, and it is the state a rejoiner escapes into once a
fresh view change lifts `to_op` above the purge floor.
A replica that has not yet applied the purge boots with
`recovered_durable_offset = Some(D)` for a large D
(`core/server-ng/src/bootstrap.rs:2242-2249`). A peer that *has* purged serves
only post-purge sends, whose `base_offset` restarts at 0, so `complete_repair`
takes the `(Some(0), Some(D))` arm, `0 <= D+1` holds, and the floor is accepted
(`:3366`). The flush loop just below then drops every one of those batches
unwritten at `:2062-2066` (`batch_end <= recovered_durable_offset`), and
`commit_partition_entry` skips the offset/stats advance at `:2422-2424`.
Because `commit_min` has advanced past them, nothing ever refetches; when that
replica finally purges, `recovered_durable_offset` becomes `None` and the new
floor covers those ops, fencing them permanently.
Net effect: committed post-purge messages never land on that replica, and
until it purges it serves pre-purge data — including as primary. Suggest
requiring local purge-apply before accepting a repair floor while a committed
purge is outstanding, and naming this mechanism in the `TODO(hubcio)` at
`partition_reconciler.rs:1075`, which currently covers only the ordering, not
this loss path.
##########
core/partitions/src/iggy_partition.rs:
##########
@@ -143,6 +146,15 @@ where
/// generation against this and resets only when it advances, so a
redundant
/// reconcile pass never re-wipes a partition already at this generation.
applied_purge_generation: u64,
+ /// Highest consensus op assigned when the last purge ran. INVARIANT: every
Review Comment:
The invariant stated here ("every journal-apply path must no-op entries with
`op <= purge_floor_op`") does not hold: `append_repaired_send_messages`
(`:3451`) is a journal-apply path with no guard. A repaired op at or below the
floor still writes `dirty_offset`, `journal.info.current_offset` and
`segment.current_position` from its *pre-purge* stamps, so post-purge appends
resume at a stale high offset while `segment.start_offset == 0`.
It is currently masked only externally — the `header_by_op` early return at
`:3288` plus the peer-side fence at `shard/src/lib.rs:3352` — so it breaks the
moment either changes. Suggest `if header.op <= self.purge_floor_op { return
Ok(None); }` to match the four sites that did get the guard.
This is also the reachable route into the resident-poll gap (see the comment
on `journal.rs:693`): the unguarded append lands out of order,
`append_with_meta`'s `op > poll_floor` check keeps it out of the index, but
`resident_entries()` still hands it to `select_resident`. Fixing this line
alone does not close that one, and vice versa.
##########
core/partitions/src/journal.rs:
##########
@@ -662,6 +679,25 @@ where
offset_to_op.keys().next().copied()
}
+ /// Seal the resident poll tier: clear the offset and timestamp poll
+ /// indexes ONLY, so `oldest_resident_offset` reads `None` and every poll
+ /// falls back to the on-disk segments. Called by a partition purge, which
+ /// wipes the segments but must KEEP the journal entries themselves:
+ /// headers, storage, `op_to_storage_offset` and the evicted ring are
+ /// consensus history that backups, repair and retransmission still walk.
+ /// Clearing those would wedge `commit_min` until a view change.
+ ///
+ /// `floor` (the purge's fence op) makes the seal survive eviction:
+ /// `evict_prefix` re-appends the retained tail, and without the floor
+ /// that re-append would re-index the pre-purge entries just cleared.
+ pub fn clear_poll_index(&self, floor: u64) {
Review Comment:
The seal reaches `offset_to_op`/`timestamp_to_op` and the `append` path, but
not the resident tier itself. `resident_entries()` (`:619`) returns fenced
pre-purge entries, `select_resident` (`:969`) filters on the query only, and
the `DiskReadOutcome::Empty` retention-recovery branch at
`core/partitions/src/poll_plan.rs:283` runs with no `oldest_resident` gate at
all. So a poll can serve purged bytes; today that is masked by tier selection
rather than by the invariant, which is a thin thing to rest on.
Suggest filtering in `resident_entries()` rather than in `select_resident`:
the positionally-paired `headers` vec (length-lock invariant at `:531-537`,
already relied on by `committed_prefix` at `:441`) lets the filter read
`header.op` out of a dense `Vec<PrepareHeader>` with no payload touch, whereas
filtering in `select_resident` costs a header cast plus a cold cache line into
each ~1 MiB buffer per entry per poll. A `poll_floor == 0` short-circuit keeps
the never-purged path byte-identical.
Related, on this function's own contract: zeroing `journal.info` in
`purge()` (`iggy_partition.rs:3241`) makes the `messages_count == 0` gate at
`:1925` short-circuit before `evict_committed_prefix`, and the shutdown
force-flush hits that same gate and only logs. The fenced entries are bounded
in bytes but have no eviction driver in time, so an idle purged partition pins
them until the process exits — on the operation users run to reclaim space. If
you evict inside `purge()` to fix that, it must be
`committed_prefix(commit_min).len()`, not `commit_max`: evicting ops in
`(commit_min, commit_max]` clears their `headers` (`:477`) and holes the
backup's `collect_committable_from_journal` walk
(`iggy_partition.rs:1778-1788`), wedging `commit_min`. `header_by_op` is also
what `RetransmitPrepares` resolves (`iggy_partition.rs:1822-1828`) and the
evicted ring is invisible to it, so uncommitted fenced ops must stay resident.
##########
core/metadata/src/impls/metadata.rs:
##########
@@ -3394,30 +3381,10 @@ where
&body,
))
}
- Operation::UpdateTopic => {
- let mut request = WireUpdateTopicRequest::decode_from(body)
- .map_err(|_| IggyError::InvalidCommand)?;
- // Same `ServerDefault` resolution as `CreateTopic` above;
rebuild
- // the prepare only if a sentinel actually needs stamping, else
- // project the untouched buffer zero-copy.
- let needs_rewrite = request.max_topic_size == 0 ||
request.message_expiry == 0;
- if request.max_topic_size == 0 {
- request.max_topic_size = self.default_max_topic_size.get();
- }
- if request.message_expiry == 0 {
- request.message_expiry = self.default_message_expiry.get();
- }
- if needs_rewrite {
- let body = request.to_bytes();
- return Ok(build_prepare_message(
- consensus,
- &header,
- Operation::UpdateTopic,
- &body,
- ));
- }
- Ok(message.project(consensus))
- }
+ // `UpdateTopic` deliberately takes the default arm: unlike create,
Review Comment:
Dropping this arm is right for the wire echo but it silently disables
retention enforcement. With the sentinel now stored,
`update_topic(ServerDefault)` leaves `core/server-ng/src/segment_cleaner.rs:88`
mapping it to no size cap, while `CreateTopic` still stamps the node default at
`:3320-3325`. The same topic therefore gets enforced retention if the value
came from create and unbounded growth if it came from an update, and the config
validator proves these are real values, not sentinels
(`server_ng_config/validators.rs:108-128`).
For the record on legacy, since the commit message frames this as parity:
legacy does not resolve at update either — `get_topic_config` returns the raw
stored pair (`core/server/src/metadata/reader.rs:722-734`), resolution is
create-only (`core/server/src/shard/system/topics.rs:49-50`). But its cleaner
then gates on `!matches!(max_topic_size, Unlimited)`
(`core/server/src/shard/system/segments.rs:65`), so a stored `ServerDefault`
enters the sized branch with `max_bytes = 0`
(`core/common/src/utils/topic_size.rs:52`), `threshold = 0`, and `current_size
< 0` is never true — it trims *every* sealed segment. So the wire echo is
genuinely legacy-identical, but enforcement now diverges in the opposite
direction from legacy and neither side honors `system.topic.max_size`. The
expiry half is a real parity win (`core/common/src/types/segment.rs:74` and
`segment_cleaner.rs:76` agree).
Suggest resolving `ServerDefault` against node config in
`topic_retention_config`/the cleaner and keeping the verbatim wire echo —
read-back shape and enforcement are separate concerns. Please also split the
"matching the legacy cleaner" comment at `segment_cleaner.rs:71-73`, which is
true for expiry and false for size. One caveat to accept explicitly: a resolved
sentinel makes local GC config-dependent where a stored `Custom` is not, so
replicas would need matching `system.topic.max_size`.
Also, this behavior change lives in metadata but its only guard is a TCP
integration test. There is a create-side sibling unit test at `:4379`; a mirror
asserting the `UpdateTopic` prepare now carries the sentinel verbatim would be
worth adding.
##########
core/server-ng/src/bootstrap.rs:
##########
@@ -3465,6 +3475,9 @@ const fn operation_triggers_partition_reconcile(op:
Operation) -> bool {
| Operation::DeleteTopic
| Operation::DeleteStream
| Operation::DeletePartitions
+ | Operation::PurgeStream
+ | Operation::PurgeTopic
+ | Operation::TruncatePartition
Review Comment:
Adding `TruncatePartition` here shortens the fuse on a pre-existing
data-loss path. `PurgeTopic`/`PurgeStream` bump `purge_generation` but never
reset `deleted_up_to_offset` (`core/metadata/src/stm/stream.rs:1778-1781`), and
`reconcile_segment_truncations` re-stages `request_truncate_partition(ns,
watermark)` on every pass whenever `watermark != 0`, with no applied-guard
(`partition_reconciler.rs:1044-1065`) — unlike purge, which at least has a
generation guard.
So after a `delete_segments` followed by a `purge`, post-purge offsets
restart at 0 and the first post-purge segment to seal below the stale pre-purge
watermark is removed by `remove_sealed_segments_up_to`. Its only brake is
`min_committed_offset()` (`iggy_partition.rs:2944`, `:2951-2956`), which the
purge just cleared. Waking the reconciler on the commit tick plus
`purges_pending` keeping passes running both make this land sooner.
Fix looks small and self-contained: zero `deleted_up_to_offset` in both
purge applies. It is a deterministic STM mutation on every replica, cannot
un-delete anything, and is semantically forced — a watermark in the pre-purge
offset space is meaningless after the reset.
##########
core/server-ng/src/partition_reconciler.rs:
##########
@@ -1077,6 +1094,7 @@ fn reconcile_partition_purges(ctx: &ReconcilerCtx) {
.map_or(0, partitions::IggyPartition::applied_purge_generation);
if committed > applied {
ctx.shard.request_purge_partition(namespace, committed);
+ counters.purges_pending += 1;
Review Comment:
`purges_pending` correctly disarms the fast-skip, but there is no backoff
behind it, unlike `reconcile_partition_teardowns` at `:800-803`. With a
persistently failing purge (ENOSPC/EROFS/EIO on `persist_purge_generation`)
this restages a frame per stuck partition on every ~1s pass, and each retry
re-runs the entire `purge()` on the pump: unlink every segment file, two file
creates in `install_empty_segment`, fsync. On a 1000-partition topic that is a
data-plane stall on a single-threaded pump.
It also re-reads `current_sequence()` at `iggy_partition.rs:3236` on every
attempt, so `purge_floor_op` climbs monotonically and the fenced set ratchets
over sends acked since the previous attempt. That turns the cross-replica
divergence from a one-shot skew bounded by reconciler timing into a window that
grows for the duration of the fault, since a failing replica's floor climbs
while healthy replicas' floors stay put.
Please keep the counter — it is the only self-heal for a stranded purge, and
before this PR a failed purge armed the fast-skip and stayed stranded until an
unrelated commit. The ask is `failure_state` backoff plus an attempt cap and a
`warn!`.
Two related gaps while here: `purges_pending` and the pre-existing
`trims_pending` feed `total()` but are absent from the pass-complete `debug!`
(`:473-486`), so an operator cannot see why passes stopped fast-skipping. And a
failing purge makes the repair-serve defer at `shard/src/lib.rs:3333` drop
every repair request silently and forever — it is `debug!`-only with no attempt
counter or metric, so the cluster loses redundancy invisibly.
##########
core/server-ng/src/partition_helpers.rs:
##########
@@ -536,6 +536,11 @@ pub async fn build_partition_fresh(
topic_id,
partition_id,
));
+ // Fresh dirs read generation 0; a dir surviving from a crashed process
+ // (this "fresh" build races repair re-materialization) reads the last
+ // durably-applied purge so the reconciler does not re-wipe messages
+ // appended after it.
+ partition.hydrate_applied_purge_generation().await?;
Review Comment:
Two problems with hydrating here.
First, this sets `applied_purge_generation = N` while `purge_floor_op` stays
0, so the replica claims the purge is applied yet carries no fence. Its
repair-serve then skips the defer at `shard/src/lib.rs:3333` and raises no
floor, which is exactly the state the fence was added to prevent.
Second, a `purge.gen` that survives a failed teardown
(`partition_reconciler.rs:817-836`, with the delete-then-recreate race already
named at `:782`) gets hydrated by a partition recreated on the reused slab ids,
whose `purge_generation` restarts at 0 (`core/metadata/src/stm/stream.rs:1615`,
`:1853`). Then `committed > applied` is false at
`partition_reconciler.rs:1095`, the wipe never runs, and the client still gets
Ok for the purge.
On the fix: unlinking `purge.gen` on a "fresh" build is not safe, because
this function cannot distinguish a true fresh create from a restart
materialization — its only signal is `restarted = dir exists && replica_count >
1` (`:470-476`), which is unconditionally false single-node, so unlinking would
re-arm the re-purge on a genuine restart and destroy post-purge data. Comparing
`committed != applied` is the minimal safe change, but it must be applied at
*both* sites: the router's own gate is `partition.applied_purge_generation() <
generation` (`core/shard/src/router.rs:662`), and `N < 1` is false, so changing
only the reconciler leaves the purge still swallowed. Keying the file on
`Partition::created_revision` (`stm/stream.rs:1849`) is the durable version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
