Hi hackers, This is a follow-up to my previous Umbra RFC thread:
https://www.postgresql.org/message-id/flat/a8d4e49b-7f20-435f-8555-7907554fedaf%40nayishan.top It has been about two months since my last message. During that time, I did two main things. First, I refactored Umbra into a three-slot physical placement model that is better suited to PostgreSQL. Second, I optimized the implementation around that model. With full_page_writes=on, Umbra's performance is now close to that of md with full_page_writes=off. These results were worth sharing promptly. I have completed a separate audit of P1-P7, so I am sending that review unit now. My audit of the later series is still in progress. Those patches already exist, but they are deliberately not part of this review request. The attached `umbra-p1-p7-20260808.tar.gz` contains P1-P7 only. Each patch has its own commit message describing its capability, ownership boundary, invariant, dependency, and non-goals. SHA256 (umbra-p1-p7-20260808.tar.gz): cc1ca7345b3d2503626643a825f57247414d6e12a5e84a15330c907c0b52931f The review plan is: 1. P1-P7: smgr selection, the md-compatible physical umfile layer, private MAP metadata bootstrap and residency, compatibility limits, and the first working mapped layout. 2. Selector-shift WAL/redo/checkpoint publication and correctness. 3. Performance and operational optimization. The first round introduces the infrastructure and layout state required for later selector shifts: smgr selection, umfile, private MAP metadata, the initial mapped layout, and selector reading and caching. It does not yet introduce selector-shift WAL, checkpoint-overlap handling, compaction, or other later mechanisms. Umbra keeps logical-to-physical placement below smgr. MAP is private, relation-bound physical-placement metadata owned by Umbra. Table and index AMs continue to operate on logical block numbers and do not interpret MAP pages, physical slots, or selectors. The model costs three physical slots per logical page. They retain the old-page version used as the redo baseline and rotate through the three slots in order. Although this increases physical page storage, the WAL reduction lowers overall storage amplification. Here are recent TPC-C results. The workload used 800 warehouses for 20 minutes with checkpoint_timeout=4min and checkpoint_completion_target=0.9. terminals md FPW=on md FPW=off Umbra FPW=on ------------------------------------------------------- 200 385616 614035 604807 400 279927 540942 568584 800 192126 420854 426757 WAL per New-Order was: terminals md FPW=on md FPW=off Umbra FPW=on ------------------------------------------------------- 200 26.05 KiB 7.53 KiB 7.54 KiB 400 36.25 KiB 7.83 KiB 7.81 KiB 800 37.71 KiB 8.40 KiB 8.24 KiB The test host was a Xeon Platinum 8173M at 2.00 GHz with 112 logical CPUs, 125 GiB RAM, a SATA DC500R SSD, and XFS. This is a single-host, single-workload result. md with full_page_writes=off is an upper-bound and sensitivity reference, not a correctness-equivalent configuration. I am also attaching the detailed benchmark summary and hardware report. For testing or for reading the complete design beyond this first review scope, the currently relatively stable branch is: https://github.com/nayishan/postgre_umbra/tree/chunksplit-restack That branch is intended for running tests and understanding the complete design. The submitted series will remain split into the review rounds described above. The complete work is still substantial. If anything in P1-P7 or its background is unclear, please contact me and I will provide more context. Regards, Mingwei Jia
umbra-p1-p7-20260808.tar.gz
Description: Binary data
tpcc_checkpoint_4min_concurrency_200_400_800.md
Description: Binary data
hardware_disk_report.txt
Description: Binary data
