On Wednesday, September 23rd, 2026 at 8:39 AM, Andrey Borodin <[email protected]> wrote:
> Hi Ashutosh, > > On 23 Sep 2026, Ashutosh Sharma wrote: > > Please take a look and let me know. Hey Ashutosh! Thanks for the excellent email kicking off this thread, I couldn't agree more about the issue. I started in a different place, asking myself if I could resurrect UNDO from ZHEAP without modifying HEAP at all (because I tried and it didn't help, other different table AMs might find benefit but HEAP is rather solid as it is) and if I could how would I demonstrate and justify it without adding a new table AM or modifying HEAP? I used to work on Berkeley DB and one of the features was its ability to WAL log UNDO/REDO records for "filesystem operations" and then during recovery tidy up and make things consistent. That struck me as a solid first application of UNDO in Postgres, so I created that and called it FILEOPS (because I'm not creative at all). > At the design level, one manifest per XID still means a pg_fsync() for > each appended record, including during redo. Have you considered WAL > plus delayed manifest synchronization, along the lines Andres > suggested [0]? It would be useful to compare small-DDL and replay > costs before settling on synchronous per-record writes. > > From reading v2, I am concerned about mapped catalog rewrites. > write_relmap_file() flushes XLOG_RELMAP_UPDATE before calling > RelationPreserveStorage(), where the patch now records PRESERVE. > A crash between those steps leaves the new mapping durable, but the > creating transaction uncommitted and its manifest without PRESERVE. > relmap_redo() does not preserve the storage either. Wouldn't the new > end-of-recovery cleanup then remove files needed by the mapped catalog? > > Could preservation be part of the relmap update's recovery semantics? > A crash test in that window during VACUUM FULL of a mapped catalog > seems particularly important. I haven't run that reproducer yet. > > The truncated-manifest test expects startup to fail. Can a crash during > a normal append leave that state without replay repairing it? If so, > could we retain the uncertain files rather than refuse startup? > > Also, Greg recently mentioned renewed UNDO/FILEOPS work [1]. It may > be worth coordinating the scope with him. Preventing new orphans and > handling existing ones, as needed for online checksums, are separate > parts of the problem. I'm spending a lot of time this week cleaning up the UNDO and FILEOPS patches in hopes of posting them soon as either an RFC or a proposed patch set. I do have new table AMs that use it, but I don't think they are ready for prime time yet. There are other use cases for UNDO also like BLOB/CLOBs etc. that I think might be interesting and if I can get the integration with nbtree and hash correct there are also benefits for indexes. That said, it's a large change and one that has philosophical and technical challenges before the community could even consider merging it in. I have hope, but it'll be a long road. Your approach has less overhead/history to deal with. I'll need to dig into it more to appreciate the direction you've taken but I do agree that it needs to happen somehow and so I don't see this as a competing idea at all. best. -greg > Thank you! > > > Best regards, Andrey Borodin. > > [0] https://postgr.es/m/[email protected] > [1] https://postgr.es/m/[email protected]
