From: "David Hildenbrand (Arm)" <[email protected]> Compile-time folded page tables are not necessarily easy to understand, and even people the were once familiar with the concept might need to refresh their memory.
Add proper documentation, including a nice diagram, for the current design. Mention details about dummy functions, including the recently changed pXdp_get() helpers. Signed-off-by: David Hildenbrand (Arm) <[email protected]> --- Documentation/mm/page_tables.rst | 84 +++++++++++++++++++++++++++++++++++----- 1 file changed, 75 insertions(+), 9 deletions(-) diff --git a/Documentation/mm/page_tables.rst b/Documentation/mm/page_tables.rst index 126c87628250..84f2715c7de0 100644 --- a/Documentation/mm/page_tables.rst +++ b/Documentation/mm/page_tables.rst @@ -143,15 +143,81 @@ pointers on each level is architecture-defined.:: Page Table Folding ================== -If the architecture does not use all the page table levels, they can be *folded* -which means skipped, and all operations performed on page tables will be -compile-time augmented to just skip a level when accessing the next lower -level. - -Page table handling code that wishes to be architecture-neutral, such as the -virtual memory manager, will need to be written so that it traverses all of the -currently five levels. This style should also be preferred for -architecture-specific code, so as to be robust to future changes. +Not all architectures support 5-level page tables; while for some of them +the exact number of supported page table levels is known at compile time, +others can determine the number of page table levels at runtime based on +hardware support and address space sizes. + +Generic page table walking code always assumes that 5 levels of page table +exist. To make page table walking code not have to worry about that, +`compile-time folding` and `runtime folding` of page tables are used. +Compile-time folding is mostly handled in common code, whereas runtime folding +is exclusively handled in architecture code. + +This description focuses on generic compile-time folded page tables; for +architecture-specific variants, some details can vary, however, without +affecting common page table walkers. + +When walking folded page tables, all upper page table levels up to the supported +level are skipped in page table walkers: this is achieved by (a) treating +entries in upper page table levels as present and pointing at a page table; and +(b) having page table walkers cast the entry pointer to the next-level entry +instead of dereferencing that table. From the perspective of a page table +walker, the entry points at itself. + +Assuming compile-time folded 4-level page tables, to achieve (a), pgd_present() +and pgd_leaf() are hard-coded to indicate a present page table entry that +points at a page table, and to achieve (b) p4d_offset() and +p4d_offset_lockless() simply cast the page table entry pointer to the next +lower level. + +In the current design, this is further modeled by having the P4D have a +single page table entry:: + + PGD + --> +------+ NOP4D + | ptr0 |-------> +------+ PUD + | ptr1 |- | ptr0 |-------> +-----+ + | ptr2 | \ +------+ | ptr |-------> ... + | ptr3 | \ | ptr | + ... \ .. + \ NOP4D + +----> +------+ PUD + | ptr1 |-------> +-----+ + +------+ | ptr |-------> ... + | ptr | + ... + +Note that the arrows from PGD to NOP4D represent page-table-walker +transitions, not pointers stored in the pgd entries. + +Using p4d as an example, `nop4d`/`p4d folded` translates to the following: + +- p4d is considered folded into pgd; both are operating on the same page + table. + +- Most pgd_* helpers are hard-coded dummy functions that ignore the passed + pgd_t values entirely. Exceptions are pgd_val() and low-level helpers + set_pgd() + pgd_page_vaddr(), which effectively translate to set_p4d()/ + p4d_pgtable() to keep existing arch code working. + + Architectures must provide p4d_* helpers (unless further common + compile-time folding applies). + +- PTRS_PER_P4D is hard-coded to 1. Architectures must define PTRS_PER_PGD. + +To avoid reading a value that will never be used but cannot be entirely +optimized out, compile-time folded page table code also makes pXdp_get() +return a constant dummy value. + +In common code, this only affects pXd_val() when used for printing page +table entries for debugging purposes. As we don't want architecture code +that uses set_pXd(), pgd_page_vaddr() or pXd_pgtable() to accidentally +operate on dummy values, the compiler will error out if it detects that the +helpers are used with dummy values. For a folded level, pXd_page() must not +be used and unconditionally triggers a compiler error. Architecture code must +instead call the helpers on the proper first page table level: e.g., set_p4d() +instead of set_pgd(). MMU, TLB, and Page Faults -- 2.43.0
