Hi all, Are there any further comments on this series? If not, I’ll post the series without the RFC tag sometime next week :)
Thanks! > Using ptep_get() and its counterparts in common code is suboptimal on > kernel configurations with generic compile-time folded page tables. > By default, ptep_get() and its friends expands to READ_ONCE(), > forcing the compiler to emit a load even when the value is not used > afterwards. > > This issue was recently reported by Christophe Leroy [1] for ppc32 > preventing futher code conversion to ptep_get()/pmdp_get()/... helper > and the same behavior can also be observed on arm64 when built with > 2- or 3-level page tables > > e.g) perf_get_page_size() in arm64 with CONFIG_PGTABLE_LEVEL=3: > > 00000000000052a0 <perf_get_page_size>: > ... > 52dc: d53b4234 mrs x20, DAIF > 52e0: d50343df msr DAIFSet, #0x3 > ... > 52fc: d35e9a69 ubfx x9, x19, #30, #9 /* > pud_offset_lockless() */ > 5300: f9403508 ldr x8, [x8, #0x68] > 5304: f869790a ldr x10, [x8, x9, lsl #3] /* pudp_get() */ > 5308: f90007ea str x10, [sp, #0x8] > 530c: f8697908 ldr x8, [x8, x9, lsl #3] /* pudp_get() */ > ... > 5360: 90000009 adrp x9, 0x5000 <perf_prepare_sample+0x548> > 5364: 92746908 and x8, x8, #0x7ffffff000 > 5368: d3557675 ubfx x21, x19, #21, #9 /* > pmd_offset_lockless() */ > ... > 5394: f8757ac8 ldr x8, [x22, x21, lsl #3] /* pmdp_get() */ > > Though PGTABLE_LEVEL=3, since the pudp_get() still remain with > READ_ONCE(), there's redundant load for the pud which is folded. > > To prevent generating suboptimal code, make pXdp_get() return a dummy > entry for compile-time folded page tables, make the helpers such as > pXd_offset()/pXd_offset_lockless(), set_pXd() validate dummy entries > at compile time to catch the wrong usage and prohibit calls to > pXd_page() in pgtable-nopXd.h. > > This series does not change the behaviour of existing code that directly > manipulates folded page-table levels using set_pgd(), pgd_page_vaddr(), and > related helpers. Those helpers continue to behave as before. > > The new restrictions only apply to code that adopts the pXdp_get()-based > access model for compile-time folded page tables. > > As the pXdp_get() can return *dummy* entry, some of code using > the stack value where saves the pXdp_get() could be a problematic: > > 1. Passing address of stack value where saves the pXdp_get() result > to pXd_offset() for example: > > pud_t *pudp, pud; > pmd_t *pmdp; > > pud = pudp_get(pudp, address); > pmdp = pmd_offset(&pud, pud, address); > > (e.g. host_pfn_mapping_level() in loongarch). > > 2. Using the pXdp_get() result to use as argument of pXd_val() and > to check prot without checking pgtable is folded. > for example, x86's effective_prot(). > > 3. Using set_pXd() with pXdp_get() will set problematic dummy entry > in folded page table like: > > set_pXd(pxdp, pXdp_get(pxdp_k)); > > 4. Using pgd_page_vaddr() to get the first-level pgtable. > passing dummy pxdp_get() for pgd_page_vaddr() will return wrong > address. Therefore, make pgd_page_vaddr() and pXd_pgtable() to > trigger the error for improper usage with folded dummy entry in the > generic compile-time folded pgtable. > > Thanksfully, above cases are rare since (1) most of usage using > pXd_offset() with result of upper pXd_offset(), (2) it's extreamely > rare to use pXd_val() for non-leaf entry in the kernel, > (3) is to handle the vmalloc_fault or set the first level of page table > and (4) to setup early page table and etc. > > Therefore, properly handle this uncommon and problematic pattern, and > document the current design of compile-time folded page tables. > > This patch is based on mm-unstable. > > Future work > =========== > - print_bad_page_map() and show_pte() still prints dummy values > instead of printing the same content for all generic compile-time > folded page tables. We might want to skip printing dummy values later. > > - We currently catch abuse of dummy values on the stack at compile-time by > relying on constant propagation by the compiler. Usama's work [3] on using > distinct types for sw vs. hw PTEs could help here as well." > > - Clean up vmalloc fault handling by synchronizing the vmalloc entry on > 32-bit architectures. This code is almost identical across architectures. > > - Unfortunately, the current design of compile-time folded page tables > appears > to be internally consistent but confusing. For example, > when CONFIG_PGTABLE_LEVELS is 2, p4d, pud, and pmd are expected to > be folded into pgd. However, the architecture code uses set_pmd() > to update the top-level page-table entry, even though it includes > pgtable-nopmd.h. > > In the future, it would be good to eliminate this source of confusion, > possibly by treating all folded upper levels consistently as dummy wrappers > around the highest real page-table level: > > NOPGD > --> +------+ P4D > | ptr0 |-------> +------+ PUD > +------+ | ptr0 |-------> +-----+ > | ptr1 |- | ptr | -------> ... > | ptr2 | \ | ptr | > | ptr3 | \ ... > ... \ > \ PUD > +----> +-----+ > | ptr | -------> ... > | ptr | > ... > > Patch History > ============= > from v2 to v3: > - repasre commit message > - move ptdump_pgtable_first_level() into pgalloc.h > - skip the huge pud operation when CONFIG_X86_DIRECT_GBPAGES is disabled > - docuemtns compile-time folded page table > - drop the applied patch. > - > https://lore.kernel.org/all/[email protected]/ > > from v1 to v2: > - Restore slient fallback to next pXd in set_pXd() and pXd_pgtable() > and add check whether they're called with dummy entry. > - Add some comment for returning first entry of pgd in arm with > 2 pgtable-level > - https://lore.kernel.org/all/[email protected]/ > > Link: [1] > https://lore.kernel.org/all/[email protected]/ > Link: [2] > https://lore.kernel.org/all/[email protected]/ > Link: [3] > https://lore.kernel.org/r/[email protected] > --- > David Hildenbrand (Arm) (13): > ARM: mm: make nommu pgd_t a scalar > ARM: mm: make 2-level pgd_t a scalar > ARM: mm: remove custom pgdp_get() > LoongArch: mm: define pud_leaf() only when PUD exists > MIPS: mm: define pud_leaf() only when PUD exists > mm/pgtable: define (pgd|p4d|pud)_leaf() for folded page tables > mm/pgtable: define (pgd|p4d|pud)_offset_lockless() for folded page > tables > openrisc/pgtable: drop __pmd_offset() > mm/pgtable: optimize pmdp_get() and friends for folded pagetable levels > mm/pgtable: catch abuse of folded dummy pgd_t/p4d_t/pud_t > mm/pgtable: disallow calling (pgd|p4d|pud)_page, pgd_page_vaddr() and > (p4d|pud)_pgtable with dummy > mm/pgtable: disallow calling folded set_pgd/set_p4d/set_pud with dummy > Documentation: mm: clarify behaviour of compile-time folded page tables > > Yeoreum Yun (8): > mm: vmscan: remove stack copy address of pud/pmd pass in > walk_pud/pmd_range() > loongarch: kvm: remove stack copy address of pXd in pXd_offset() > riscv: kvm: remove stack copy address of pXd in pXd_offset() > riscv: mm: use proper set_pXd() for generic compile-time folded patable > in vmalloc_fault() > mm/pgtable: redefine PGTABLE_LEVEL enum with ascend order from PGD > x86: mm: use pgtable_level enum in effective_prot_pXd() > x86: mm: carve out the generic compile-time folded pgtable case in > effective_prot() > x86: mm: skip collapse_pud_page() when CONFIG_X86_DIRECT_GBPAGES > disabled > > Documentation/mm/page_tables.rst | 84 > +++++++++++++++++++++++++---- > arch/arm/include/asm/page-nommu.h | 4 +- > arch/arm/include/asm/pgtable-2level-types.h | 20 +++++-- > arch/arm/include/asm/pgtable.h | 2 - > arch/arm64/include/asm/pgtable.h | 23 -------- > arch/loongarch/include/asm/pgtable.h | 2 + > arch/loongarch/kvm/mmu.c | 20 ++++--- > arch/mips/include/asm/pgtable.h | 2 + > arch/openrisc/include/asm/pgtable.h | 3 -- > arch/riscv/kvm/mmu.c | 20 ++++--- > arch/riscv/mm/fault.c | 52 +++++++++++------- > arch/x86/mm/dump_pagetables.c | 18 ++++--- > arch/x86/mm/pat/set_memory.c | 2 +- > include/asm-generic/pgtable-nop4d.h | 54 +++++++++++++++---- > include/asm-generic/pgtable-nopmd.h | 54 +++++++++++++++---- > include/asm-generic/pgtable-nopud.h | 56 +++++++++++++++---- > include/linux/pgtable.h | 69 +++++++++++++++++------- > mm/vmscan.c | 4 +- > 18 files changed, 352 insertions(+), 137 deletions(-) > --- > base-commit: e3b5239afe1b8f0194db7436b17c33e94c1988c4 > change-id: 20260722-dummy_ptxp3-3741d78cc70f > > Best regards, > -- > Sincerely, > Yeoreum Yun > -- Sincerely, Yeoreum Yun
