hubcio commented on code in PR #4146:
URL: https://github.com/apache/iggy/pull/4146#discussion_r3996538430
##########
core/journal/src/partition_journal.rs:
##########
@@ -1018,21 +1265,41 @@ impl<S: DurableStorage> PartitionPrepareJournal<S> {
Ok((buffer, frame_length, reference))
}
- async fn publish(&self, state: JournalState) -> io::Result<()> {
+ /// Publish the frontier by overwriting the older of two fixed slots.
+ ///
+ /// Both slots exist and hold a complete record from the moment the journal
+ /// opens ([`Self::install_frontier`]), so a publication is one 4096-byte
+ /// overwrite plus `fdatasync`: no create, no truncate, no size change, no
+ /// rename and no directory barrier. On a journaling filesystem that is the
+ /// difference between zero metadata transactions per acknowledgment and
+ /// roughly two, which at thousands of acknowledgments per second per node
+ /// costs more than the prepare bytes the batch carries.
+ ///
+ /// A torn slot fails its checksum and its partner still holds the previous
+ /// publication, the guarantee the temporary-file rename used to provide.
+ /// Unlike the rename, an unreadable NEWEST slot leaves the published
prefix
+ /// one publication behind; [`Self::recover_unpublished_tail`] recovers it
+ /// from the records themselves.
+ ///
+ /// The slot follows the sequence's parity, so a publication never
overwrites
+ /// the copy it would have to fall back on.
+ async fn publish(&mut self, state: JournalState) -> io::Result<()> {
if state
.segment_storage
.is_some_and(|segments| !segments.valid())
{
return Err(invalid("invalid durable segment boundaries"));
}
- let temporary = self.directory.join("frontier.tmp");
- let mut file = self.storage.open(&temporary, OpenMode::Create).await?;
- file.write(0, state.encode()).await?;
- file.sync().await?;
- self.storage
- .rename(&temporary, &self.directory.join("frontier"))
+ let sequence = self
+ .frontier_sequence
+ .checked_add(1)
+ .ok_or_else(|| invalid("WAL frontier sequence exhausted"))?;
+ self.frontier
+ .write_aligned(frontier_offset(sequence), state.encode(sequence))
Review Comment:
critical: after an acknowledged append lands in slot 1, a crash and binary
rollback can make the old reader accept stale slot 0 and truncate acknowledged
data. version the two-slot format so old readers refuse it, retaining one-slot
migration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
