Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

On Thu, Aug 17, 2017 at 01:10:05PM +0200, Ingo Molnar wrote:
> 
> * Kirill A. Shutemov  wrote:
> 
> > On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> > > 
> > > * Kirill A. Shutemov  wrote:
> > > 
> > > > For boot-time switching between 4- and 5-level paging we need to be able
> > > > to fold p4d page table level at runtime. It requires variable
> > > > PGDIR_SHIFT and PTRS_PER_P4D.
> > > > 
> > > > Signed-off-by: Kirill A. Shutemov 
> > > > ---
> > > >  arch/x86/boot/compressed/kaslr.c|  5 +
> > > >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> > > >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> > > >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> > > >  arch/x86/kernel/head64.c|  9 -
> > > >  arch/x86/mm/dump_pagetables.c   | 12 +---
> > > >  arch/x86/mm/init_64.c   |  2 +-
> > > >  arch/x86/mm/kasan_init_64.c |  2 +-
> > > >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> > > >  include/asm-generic/5level-fixup.h  |  1 +
> > > >  include/asm-generic/pgtable-nop4d.h |  1 +
> > > >  include/linux/kasan.h   |  2 +-
> > > >  mm/kasan/kasan_init.c   |  2 +-
> > > >  13 files changed, 43 insertions(+), 16 deletions(-)
> > > 
> > > So I'm wondering what the code generation effect of this is - what's the 
> > > before/after vmlinux size?
> > > 
> > > My guess is that the effect should be very small, as these constants are 
> > > not 
> > > widely used - but I'm only guessing and could be wrong.
> > 
> > This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
> > 0.01%.
> 
> Please add this info to one of the changelogs.
> 
> BTW., some of that might be won back via the later optimizations, right?

Sorry, but no. Just chacked. Introducing alternatives increases .text
segment size by about 4k.

-- 
 Kirill A. Shutemov

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

On Thu, Aug 17, 2017 at 01:10:05PM +0200, Ingo Molnar wrote:
> 
> * Kirill A. Shutemov  wrote:
> 
> > On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> > > 
> > > * Kirill A. Shutemov  wrote:
> > > 
> > > > For boot-time switching between 4- and 5-level paging we need to be able
> > > > to fold p4d page table level at runtime. It requires variable
> > > > PGDIR_SHIFT and PTRS_PER_P4D.
> > > > 
> > > > Signed-off-by: Kirill A. Shutemov 
> > > > ---
> > > >  arch/x86/boot/compressed/kaslr.c|  5 +
> > > >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> > > >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> > > >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> > > >  arch/x86/kernel/head64.c|  9 -
> > > >  arch/x86/mm/dump_pagetables.c   | 12 +---
> > > >  arch/x86/mm/init_64.c   |  2 +-
> > > >  arch/x86/mm/kasan_init_64.c |  2 +-
> > > >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> > > >  include/asm-generic/5level-fixup.h  |  1 +
> > > >  include/asm-generic/pgtable-nop4d.h |  1 +
> > > >  include/linux/kasan.h   |  2 +-
> > > >  mm/kasan/kasan_init.c   |  2 +-
> > > >  13 files changed, 43 insertions(+), 16 deletions(-)
> > > 
> > > So I'm wondering what the code generation effect of this is - what's the 
> > > before/after vmlinux size?
> > > 
> > > My guess is that the effect should be very small, as these constants are 
> > > not 
> > > widely used - but I'm only guessing and could be wrong.
> > 
> > This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
> > 0.01%.
> 
> Please add this info to one of the changelogs.
> 
> BTW., some of that might be won back via the later optimizations, right?

Sorry, but no. Just chacked. Introducing alternatives increases .text
segment size by about 4k.

-- 
 Kirill A. Shutemov

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Ingo Molnar]

* Kirill A. Shutemov  wrote:

> On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> > 
> > * Kirill A. Shutemov  wrote:
> > 
> > > For boot-time switching between 4- and 5-level paging we need to be able
> > > to fold p4d page table level at runtime. It requires variable
> > > PGDIR_SHIFT and PTRS_PER_P4D.
> > > 
> > > Signed-off-by: Kirill A. Shutemov 
> > > ---
> > >  arch/x86/boot/compressed/kaslr.c|  5 +
> > >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> > >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> > >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> > >  arch/x86/kernel/head64.c|  9 -
> > >  arch/x86/mm/dump_pagetables.c   | 12 +---
> > >  arch/x86/mm/init_64.c   |  2 +-
> > >  arch/x86/mm/kasan_init_64.c |  2 +-
> > >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> > >  include/asm-generic/5level-fixup.h  |  1 +
> > >  include/asm-generic/pgtable-nop4d.h |  1 +
> > >  include/linux/kasan.h   |  2 +-
> > >  mm/kasan/kasan_init.c   |  2 +-
> > >  13 files changed, 43 insertions(+), 16 deletions(-)
> > 
> > So I'm wondering what the code generation effect of this is - what's the 
> > before/after vmlinux size?
> > 
> > My guess is that the effect should be very small, as these constants are 
> > not 
> > widely used - but I'm only guessing and could be wrong.
> 
> This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
> 0.01%.

Please add this info to one of the changelogs.

BTW., some of that might be won back via the later optimizations, right?

BTW, I think what matters most is the .text section size, while 'size' would 
also 
add exceptions/out-of-line sections too.

Here's a way to display the .text section only:

 triton:~/tip> printf "# vmlinux .text size: %d\n" 0x$(echo $(readelf -W -e 
vmlinux | grep ' \[ 1\] .text ') | cut -d' ' -f7)
 # vmlinux .text size: 9797610

 triton:~/tip> size vmlinux
   textdata bss dec hex filename
 107483064875500  876544 16500350 fbc67e vmlinux

Thanks,

Ingo

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Ingo Molnar]

* Kirill A. Shutemov  wrote:

> On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> > 
> > * Kirill A. Shutemov  wrote:
> > 
> > > For boot-time switching between 4- and 5-level paging we need to be able
> > > to fold p4d page table level at runtime. It requires variable
> > > PGDIR_SHIFT and PTRS_PER_P4D.
> > > 
> > > Signed-off-by: Kirill A. Shutemov 
> > > ---
> > >  arch/x86/boot/compressed/kaslr.c|  5 +
> > >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> > >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> > >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> > >  arch/x86/kernel/head64.c|  9 -
> > >  arch/x86/mm/dump_pagetables.c   | 12 +---
> > >  arch/x86/mm/init_64.c   |  2 +-
> > >  arch/x86/mm/kasan_init_64.c |  2 +-
> > >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> > >  include/asm-generic/5level-fixup.h  |  1 +
> > >  include/asm-generic/pgtable-nop4d.h |  1 +
> > >  include/linux/kasan.h   |  2 +-
> > >  mm/kasan/kasan_init.c   |  2 +-
> > >  13 files changed, 43 insertions(+), 16 deletions(-)
> > 
> > So I'm wondering what the code generation effect of this is - what's the 
> > before/after vmlinux size?
> > 
> > My guess is that the effect should be very small, as these constants are 
> > not 
> > widely used - but I'm only guessing and could be wrong.
> 
> This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
> 0.01%.

Please add this info to one of the changelogs.

BTW., some of that might be won back via the later optimizations, right?

BTW, I think what matters most is the .text section size, while 'size' would 
also 
add exceptions/out-of-line sections too.

Here's a way to display the .text section only:

 triton:~/tip> printf "# vmlinux .text size: %d\n" 0x$(echo $(readelf -W -e 
vmlinux | grep ' \[ 1\] .text ') | cut -d' ' -f7)
 # vmlinux .text size: 9797610

 triton:~/tip> size vmlinux
   textdata bss dec hex filename
 107483064875500  876544 16500350 fbc67e vmlinux

Thanks,

Ingo

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> 
> * Kirill A. Shutemov  wrote:
> 
> > For boot-time switching between 4- and 5-level paging we need to be able
> > to fold p4d page table level at runtime. It requires variable
> > PGDIR_SHIFT and PTRS_PER_P4D.
> > 
> > Signed-off-by: Kirill A. Shutemov 
> > ---
> >  arch/x86/boot/compressed/kaslr.c|  5 +
> >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> >  arch/x86/kernel/head64.c|  9 -
> >  arch/x86/mm/dump_pagetables.c   | 12 +---
> >  arch/x86/mm/init_64.c   |  2 +-
> >  arch/x86/mm/kasan_init_64.c |  2 +-
> >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> >  include/asm-generic/5level-fixup.h  |  1 +
> >  include/asm-generic/pgtable-nop4d.h |  1 +
> >  include/linux/kasan.h   |  2 +-
> >  mm/kasan/kasan_init.c   |  2 +-
> >  13 files changed, 43 insertions(+), 16 deletions(-)
> 
> So I'm wondering what the code generation effect of this is - what's the 
> before/after vmlinux size?
> 
> My guess is that the effect should be very small, as these constants are not 
> widely used - but I'm only guessing and could be wrong.

This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
0.01%.

-- 
 Kirill A. Shutemov

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

On Thu, Aug 17, 2017 at 11:00:38AM +0200, Ingo Molnar wrote:
> 
> * Kirill A. Shutemov  wrote:
> 
> > For boot-time switching between 4- and 5-level paging we need to be able
> > to fold p4d page table level at runtime. It requires variable
> > PGDIR_SHIFT and PTRS_PER_P4D.
> > 
> > Signed-off-by: Kirill A. Shutemov 
> > ---
> >  arch/x86/boot/compressed/kaslr.c|  5 +
> >  arch/x86/include/asm/pgtable_32.h   |  2 ++
> >  arch/x86/include/asm/pgtable_32_types.h |  2 ++
> >  arch/x86/include/asm/pgtable_64_types.h | 15 +--
> >  arch/x86/kernel/head64.c|  9 -
> >  arch/x86/mm/dump_pagetables.c   | 12 +---
> >  arch/x86/mm/init_64.c   |  2 +-
> >  arch/x86/mm/kasan_init_64.c |  2 +-
> >  arch/x86/platform/efi/efi_64.c  |  4 ++--
> >  include/asm-generic/5level-fixup.h  |  1 +
> >  include/asm-generic/pgtable-nop4d.h |  1 +
> >  include/linux/kasan.h   |  2 +-
> >  mm/kasan/kasan_init.c   |  2 +-
> >  13 files changed, 43 insertions(+), 16 deletions(-)
> 
> So I'm wondering what the code generation effect of this is - what's the 
> before/after vmlinux size?
> 
> My guess is that the effect should be very small, as these constants are not 
> widely used - but I'm only guessing and could be wrong.

This change increase vmlinux size for defconfig + X86_5LEVEL=y by ~4k or
0.01%.

-- 
 Kirill A. Shutemov

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Ingo Molnar]

* Kirill A. Shutemov  wrote:

> For boot-time switching between 4- and 5-level paging we need to be able
> to fold p4d page table level at runtime. It requires variable
> PGDIR_SHIFT and PTRS_PER_P4D.
> 
> Signed-off-by: Kirill A. Shutemov 
> ---
>  arch/x86/boot/compressed/kaslr.c|  5 +
>  arch/x86/include/asm/pgtable_32.h   |  2 ++
>  arch/x86/include/asm/pgtable_32_types.h |  2 ++
>  arch/x86/include/asm/pgtable_64_types.h | 15 +--
>  arch/x86/kernel/head64.c|  9 -
>  arch/x86/mm/dump_pagetables.c   | 12 +---
>  arch/x86/mm/init_64.c   |  2 +-
>  arch/x86/mm/kasan_init_64.c |  2 +-
>  arch/x86/platform/efi/efi_64.c  |  4 ++--
>  include/asm-generic/5level-fixup.h  |  1 +
>  include/asm-generic/pgtable-nop4d.h |  1 +
>  include/linux/kasan.h   |  2 +-
>  mm/kasan/kasan_init.c   |  2 +-
>  13 files changed, 43 insertions(+), 16 deletions(-)

So I'm wondering what the code generation effect of this is - what's the 
before/after vmlinux size?

My guess is that the effect should be very small, as these constants are not 
widely used - but I'm only guessing and could be wrong.

Thanks,

Ingo

Re: [PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Ingo Molnar]

* Kirill A. Shutemov  wrote:

> For boot-time switching between 4- and 5-level paging we need to be able
> to fold p4d page table level at runtime. It requires variable
> PGDIR_SHIFT and PTRS_PER_P4D.
> 
> Signed-off-by: Kirill A. Shutemov 
> ---
>  arch/x86/boot/compressed/kaslr.c|  5 +
>  arch/x86/include/asm/pgtable_32.h   |  2 ++
>  arch/x86/include/asm/pgtable_32_types.h |  2 ++
>  arch/x86/include/asm/pgtable_64_types.h | 15 +--
>  arch/x86/kernel/head64.c|  9 -
>  arch/x86/mm/dump_pagetables.c   | 12 +---
>  arch/x86/mm/init_64.c   |  2 +-
>  arch/x86/mm/kasan_init_64.c |  2 +-
>  arch/x86/platform/efi/efi_64.c  |  4 ++--
>  include/asm-generic/5level-fixup.h  |  1 +
>  include/asm-generic/pgtable-nop4d.h |  1 +
>  include/linux/kasan.h   |  2 +-
>  mm/kasan/kasan_init.c   |  2 +-
>  13 files changed, 43 insertions(+), 16 deletions(-)

So I'm wondering what the code generation effect of this is - what's the 
before/after vmlinux size?

My guess is that the effect should be very small, as these constants are not 
widely used - but I'm only guessing and could be wrong.

Thanks,

Ingo

[PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

For boot-time switching between 4- and 5-level paging we need to be able
to fold p4d page table level at runtime. It requires variable
PGDIR_SHIFT and PTRS_PER_P4D.

Signed-off-by: Kirill A. Shutemov 
---
 arch/x86/boot/compressed/kaslr.c|  5 +
 arch/x86/include/asm/pgtable_32.h   |  2 ++
 arch/x86/include/asm/pgtable_32_types.h |  2 ++
 arch/x86/include/asm/pgtable_64_types.h | 15 +--
 arch/x86/kernel/head64.c|  9 -
 arch/x86/mm/dump_pagetables.c   | 12 +---
 arch/x86/mm/init_64.c   |  2 +-
 arch/x86/mm/kasan_init_64.c |  2 +-
 arch/x86/platform/efi/efi_64.c  |  4 ++--
 include/asm-generic/5level-fixup.h  |  1 +
 include/asm-generic/pgtable-nop4d.h |  1 +
 include/linux/kasan.h   |  2 +-
 mm/kasan/kasan_init.c   |  2 +-
 13 files changed, 43 insertions(+), 16 deletions(-)

diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 99c7194f7ea6..f0f53817a8d2 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -43,6 +43,11 @@
 #define STATIC
 #include 
 
+#ifdef CONFIG_X86_5LEVEL
+unsigned int pgdir_shift __read_mostly = 48;
+unsigned int ptrs_per_p4d __read_mostly = 512;
+#endif
+
 extern unsigned long get_cmd_line_ptr(void);
 
 /* Simplified build-specific string for starting entropy. */
diff --git a/arch/x86/include/asm/pgtable_32.h 
b/arch/x86/include/asm/pgtable_32.h
index bfab55675c16..9c3c811347b0 100644
--- a/arch/x86/include/asm/pgtable_32.h
+++ b/arch/x86/include/asm/pgtable_32.h
@@ -32,6 +32,8 @@ static inline void pgtable_cache_init(void) { }
 static inline void check_pgt_cache(void) { }
 void paging_init(void);
 
+static inline int pgd_large(pgd_t pgd) { return 0; }
+
 /*
  * Define this if things work differently on an i386 and an i486:
  * it will (on an i486) warn about kernel memory accesses that are
diff --git a/arch/x86/include/asm/pgtable_32_types.h 
b/arch/x86/include/asm/pgtable_32_types.h
index 9fb2f2bc8245..8928eac4ef08 100644
--- a/arch/x86/include/asm/pgtable_32_types.h
+++ b/arch/x86/include/asm/pgtable_32_types.h
@@ -14,6 +14,8 @@
 # include 
 #endif
 
+#define p4d_folded 1
+
 #define PGDIR_SIZE (1UL << PGDIR_SHIFT)
 #define PGDIR_MASK (~(PGDIR_SIZE - 1))
 
diff --git a/arch/x86/include/asm/pgtable_64_types.h 
b/arch/x86/include/asm/pgtable_64_types.h
index a9f77ead7088..a5338b0936ad 100644
--- a/arch/x86/include/asm/pgtable_64_types.h
+++ b/arch/x86/include/asm/pgtable_64_types.h
@@ -19,6 +19,15 @@ typedef unsigned longpgprotval_t;
 
 typedef struct { pteval_t pte; } pte_t;
 
+#ifdef CONFIG_X86_5LEVEL
+#define p4d_folded 0
+#else
+#define p4d_folded 1
+#endif
+
+extern unsigned int pgdir_shift;
+extern unsigned int ptrs_per_p4d;
+
 #endif /* !__ASSEMBLY__ */
 
 #define SHARED_KERNEL_PMD  0
@@ -28,14 +37,15 @@ typedef struct { pteval_t pte; } pte_t;
 /*
  * PGDIR_SHIFT determines what a top-level page table entry can map
  */
-#define PGDIR_SHIFT48
+#define PGDIR_SHIFTpgdir_shift
 #define PTRS_PER_PGD   512
 
 /*
  * 4th level page in 5-level paging case
  */
 #define P4D_SHIFT  39
-#define PTRS_PER_P4D   512
+#define __PTRS_PER_P4D 512
+#define PTRS_PER_P4D   ptrs_per_p4d
 #define P4D_SIZE   (_AC(1, UL) << P4D_SHIFT)
 #define P4D_MASK   (~(P4D_SIZE - 1))
 
@@ -46,6 +56,7 @@ typedef struct { pteval_t pte; } pte_t;
  */
 #define PGDIR_SHIFT39
 #define PTRS_PER_PGD   512
+#define __PTRS_PER_P4D 1
 
 #endif /* CONFIG_X86_5LEVEL */
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index aa163bda4a29..f05ea8d6b0fa 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -38,6 +38,13 @@ extern pmd_t 
early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
 
+#ifdef CONFIG_X86_5LEVEL
+unsigned int pgdir_shift __read_mostly = 48;
+EXPORT_SYMBOL(pgdir_shift);
+unsigned int ptrs_per_p4d __read_mostly = 512;
+EXPORT_SYMBOL(ptrs_per_p4d);
+#endif
+
 #if defined(CONFIG_RANDOMIZE_MEMORY) || defined(CONFIG_X86_5LEVEL)
 unsigned long page_offset_base __read_mostly = __PAGE_OFFSET_BASE;
 EXPORT_SYMBOL(page_offset_base);
@@ -329,7 +336,7 @@ asmlinkage __visible void __init x86_64_start_kernel(char * 
real_mode_data)
BUILD_BUG_ON((__START_KERNEL_map & ~PMD_MASK) != 0);
BUILD_BUG_ON((MODULES_VADDR & ~PMD_MASK) != 0);
BUILD_BUG_ON(!(MODULES_VADDR > __START_KERNEL));
-   BUILD_BUG_ON(!(((MODULES_END - 1) & PGDIR_MASK) ==
+   MAYBE_BUILD_BUG_ON(!(((MODULES_END - 1) & PGDIR_MASK) ==
(__START_KERNEL & PGDIR_MASK)));
BUILD_BUG_ON(__fix_to_virt(__end_of_fixed_addresses) <= MODULES_END);
 
diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c
index 5e3ac6fe6c9e..bad178deffba 

[PATCHv4 08/14] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable

[Kirill A. Shutemov]

For boot-time switching between 4- and 5-level paging we need to be able
to fold p4d page table level at runtime. It requires variable
PGDIR_SHIFT and PTRS_PER_P4D.

Signed-off-by: Kirill A. Shutemov 
---
 arch/x86/boot/compressed/kaslr.c|  5 +
 arch/x86/include/asm/pgtable_32.h   |  2 ++
 arch/x86/include/asm/pgtable_32_types.h |  2 ++
 arch/x86/include/asm/pgtable_64_types.h | 15 +--
 arch/x86/kernel/head64.c|  9 -
 arch/x86/mm/dump_pagetables.c   | 12 +---
 arch/x86/mm/init_64.c   |  2 +-
 arch/x86/mm/kasan_init_64.c |  2 +-
 arch/x86/platform/efi/efi_64.c  |  4 ++--
 include/asm-generic/5level-fixup.h  |  1 +
 include/asm-generic/pgtable-nop4d.h |  1 +
 include/linux/kasan.h   |  2 +-
 mm/kasan/kasan_init.c   |  2 +-
 13 files changed, 43 insertions(+), 16 deletions(-)

diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c
index 99c7194f7ea6..f0f53817a8d2 100644
--- a/arch/x86/boot/compressed/kaslr.c
+++ b/arch/x86/boot/compressed/kaslr.c
@@ -43,6 +43,11 @@
 #define STATIC
 #include 
 
+#ifdef CONFIG_X86_5LEVEL
+unsigned int pgdir_shift __read_mostly = 48;
+unsigned int ptrs_per_p4d __read_mostly = 512;
+#endif
+
 extern unsigned long get_cmd_line_ptr(void);
 
 /* Simplified build-specific string for starting entropy. */
diff --git a/arch/x86/include/asm/pgtable_32.h 
b/arch/x86/include/asm/pgtable_32.h
index bfab55675c16..9c3c811347b0 100644
--- a/arch/x86/include/asm/pgtable_32.h
+++ b/arch/x86/include/asm/pgtable_32.h
@@ -32,6 +32,8 @@ static inline void pgtable_cache_init(void) { }
 static inline void check_pgt_cache(void) { }
 void paging_init(void);
 
+static inline int pgd_large(pgd_t pgd) { return 0; }
+
 /*
  * Define this if things work differently on an i386 and an i486:
  * it will (on an i486) warn about kernel memory accesses that are
diff --git a/arch/x86/include/asm/pgtable_32_types.h 
b/arch/x86/include/asm/pgtable_32_types.h
index 9fb2f2bc8245..8928eac4ef08 100644
--- a/arch/x86/include/asm/pgtable_32_types.h
+++ b/arch/x86/include/asm/pgtable_32_types.h
@@ -14,6 +14,8 @@
 # include 
 #endif
 
+#define p4d_folded 1
+
 #define PGDIR_SIZE (1UL << PGDIR_SHIFT)
 #define PGDIR_MASK (~(PGDIR_SIZE - 1))
 
diff --git a/arch/x86/include/asm/pgtable_64_types.h 
b/arch/x86/include/asm/pgtable_64_types.h
index a9f77ead7088..a5338b0936ad 100644
--- a/arch/x86/include/asm/pgtable_64_types.h
+++ b/arch/x86/include/asm/pgtable_64_types.h
@@ -19,6 +19,15 @@ typedef unsigned longpgprotval_t;
 
 typedef struct { pteval_t pte; } pte_t;
 
+#ifdef CONFIG_X86_5LEVEL
+#define p4d_folded 0
+#else
+#define p4d_folded 1
+#endif
+
+extern unsigned int pgdir_shift;
+extern unsigned int ptrs_per_p4d;
+
 #endif /* !__ASSEMBLY__ */
 
 #define SHARED_KERNEL_PMD  0
@@ -28,14 +37,15 @@ typedef struct { pteval_t pte; } pte_t;
 /*
  * PGDIR_SHIFT determines what a top-level page table entry can map
  */
-#define PGDIR_SHIFT48
+#define PGDIR_SHIFTpgdir_shift
 #define PTRS_PER_PGD   512
 
 /*
  * 4th level page in 5-level paging case
  */
 #define P4D_SHIFT  39
-#define PTRS_PER_P4D   512
+#define __PTRS_PER_P4D 512
+#define PTRS_PER_P4D   ptrs_per_p4d
 #define P4D_SIZE   (_AC(1, UL) << P4D_SHIFT)
 #define P4D_MASK   (~(P4D_SIZE - 1))
 
@@ -46,6 +56,7 @@ typedef struct { pteval_t pte; } pte_t;
  */
 #define PGDIR_SHIFT39
 #define PTRS_PER_PGD   512
+#define __PTRS_PER_P4D 1
 
 #endif /* CONFIG_X86_5LEVEL */
 
diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index aa163bda4a29..f05ea8d6b0fa 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -38,6 +38,13 @@ extern pmd_t 
early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD];
 static unsigned int __initdata next_early_pgt;
 pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);
 
+#ifdef CONFIG_X86_5LEVEL
+unsigned int pgdir_shift __read_mostly = 48;
+EXPORT_SYMBOL(pgdir_shift);
+unsigned int ptrs_per_p4d __read_mostly = 512;
+EXPORT_SYMBOL(ptrs_per_p4d);
+#endif
+
 #if defined(CONFIG_RANDOMIZE_MEMORY) || defined(CONFIG_X86_5LEVEL)
 unsigned long page_offset_base __read_mostly = __PAGE_OFFSET_BASE;
 EXPORT_SYMBOL(page_offset_base);
@@ -329,7 +336,7 @@ asmlinkage __visible void __init x86_64_start_kernel(char * 
real_mode_data)
BUILD_BUG_ON((__START_KERNEL_map & ~PMD_MASK) != 0);
BUILD_BUG_ON((MODULES_VADDR & ~PMD_MASK) != 0);
BUILD_BUG_ON(!(MODULES_VADDR > __START_KERNEL));
-   BUILD_BUG_ON(!(((MODULES_END - 1) & PGDIR_MASK) ==
+   MAYBE_BUILD_BUG_ON(!(((MODULES_END - 1) & PGDIR_MASK) ==
(__START_KERNEL & PGDIR_MASK)));
BUILD_BUG_ON(__fix_to_virt(__end_of_fixed_addresses) <= MODULES_END);
 
diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c
index 5e3ac6fe6c9e..bad178deffba 100644
---