Hi, On Fri, 13 Nov 2026 at 06:34, Ritesh Kumar <[email protected]> wrote: > > Hi Ayush, > > Thanks for working on these performance enhancements. > > Denis Smirnov (added to cc-list) and I did an initial code-review of > this patch together. We kept our focus on 0001 since its scope is > simpler and it should be able to go in independently of 0002. We’ll put > up some feedback on 0002 separately, since there’s some design aspects > to deliberate on.
Thanks to both of you for reviewing this! > [ Rationale within v4-0001 commit message ] > > Keep the existing LOG-level failure policy. If the batch directory > > fsync fails, retain the archive-status files rather than cleaning > > them up or taking down the server. > > v4-0001 intentionally kept the `LOG' and it matches master’s per-file > `durable_unlink(..., LOG)'. The removals themselves are harmless if the > fsync fails, since a resurrected old segment is just removed again. > > However, in the case of a fsync error, it (the error) would belong to > the whole directory and not just to the `unlink'. The concern is that > the directory fsync can report an error belonging to a concurrent > segment installation and if the checkpointer would log it at `LOG', then > the installing backend’s own fsync may succeed and commits written into > that segment could be lost on power failure. > > Whether this can happen depends on the filesystem (journaling > filesystems usually go read-only on a metadata I/O error), so it’s more > of a reason for caution instead of a demonstrated bug. Master has the > same exposure to this risk via `durable_unlink(..., LOG)', but since > 0001 rewrites this call, routing it through `data_sync_elevel(ERROR)' > seems to be the safer choice (i.e. calling `fsync_fname(XLOGDIR, true)' > and dropping the early-return). A `PANIC' by default does take the > server down, but after fsyncgate that’s the accepted trade-off. Yeah, fair point. I had changed it to LOG in v4 to match durable_unlink(), without thinking that an fsync error on pg_wal could belong to another caller's rename. Switched back to fsync_fname(XLOGDIR, true) in v5, like v3 had, and dropped the early return. > Denis pointed out a pattern where `durable_rename()' / > `durable_unlink()' use one elevel both for the `rename' / `unlink' > failing and for the fsync inside them failing. Callers chose `LOG' / > `DEBUG1' / `WARNING' for the former (e.g. 7f2a10fecd8 made > recycle/removal failures non-fatal because of Windows file locks), and > fsync failures silently inherited it, since 9ccdd7f66e3 didn’t revisit > some of these call-sites. > > There’s a case to be made about splitting the two levels — will defer > the discussion on this topic to its own thread. Denis has an initial > list of 6 call-sites that follow this pattern. The `pg_wal' ones matter > since they depend on durability. +1, a separate thread for that makes sense. > B. The `RemoveNonParentXlogFiles' path > ═══════════════════════════════════════ > > Was leaving `RemoveNonParentXlogFiles' out a scope choice, or is there a > safety reason? > > One difference I can see: that path removes old-timeline segments, some > of which may have no archive status file. So, a lost unlink could bring > back a file that the next checkpoint could mark `.ready', and the > `.done' ordering that makes 0001 safe wouldn’t cover it. > > If it’s just scope, maybe a sentence in the commit message saying so > would help. Initially I left it out mostly to keep 0001 scoped to checkpoint cleanup, but the missing status file case you mentioned looks like a real issue too. Also, at promotion it runs after the signal files are unlinked and the end-of-recovery record is written, so crash recovery wouldn't run it again if we crashed before the fsync. I updated the commit message to mention that. Thanks. > C. Nits > ═══════ > > C1: docstring should include `cleanup_names' > ──────────────────────────────────────────── > > `RemoveXlogFile()' ’s header comment should also document > `cleanup_names'. It should mention that `NULL' means durable-now and > non-`NULL' means the caller fsyncs and then cleans up. > > > C2: `NIL' and `NULL' > ──────────────────── > > `RemoveXlogFile()' has 2 callers that differ by their level of > indirection. `RemoveOldXlogFiles' passes `&cleanup_names' (a pointer to > a list that starts as `NIL'), and `RemoveNonParentXlogFiles()' passes > `NULL'. Since `List **' doubles as a mode-switch, consider documenting > that to make it more explicit or bringing 0002’s `bool batch' into 0001. Added the comment on cleanup_names. I kept it as List ** in 0001 rather than pulling in bool batch, hopefully the comment makes the NULL vs non-NULL split clear enough. Attached v5. 0002 is just rebased on top of 0001, still WIP (I need to spend more time on that).. Regards, Ayush
nocfbot-v5-0002-WIP-Batch-fsyncs-when-recycling-WAL-segments.patch
Description: Binary data
v5-0001-Batch-fsyncs-when-removing-old-WAL-segments.patch
Description: Binary data
