Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 04:17 PM, Andy Lutomirski wrote:
> On Tue, Nov 21, 2017 at 3:42 PM, Dave Hansen
> unsigned long start = (unsigned long)get_cpu_entry_area(cpu);
> for (unsigned long addr = start; addr < start + sizeof(struct
> cpu_entry_area); addr += PAGE_SIZE) {
>   pte_t pte = *pte_offset_k(addr);  /* or however you do this */
>   kaiser_add_mapping(pte_pfn(pte), pte_prot(pte));
> }
> 
> modulo the huge pile of typos in there that surely exist.

That would work.  I just need to find a suitable pte_offset_k() in the
kernel and make sure it works for these purposes.  We probably have one.

> But I still prefer my approach of just sharing the cpu_entry_area pmd
> entries between the user and kernel tables.

That would be spiffy.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 04:17 PM, Andy Lutomirski wrote:
> On Tue, Nov 21, 2017 at 3:42 PM, Dave Hansen
> unsigned long start = (unsigned long)get_cpu_entry_area(cpu);
> for (unsigned long addr = start; addr < start + sizeof(struct
> cpu_entry_area); addr += PAGE_SIZE) {
>   pte_t pte = *pte_offset_k(addr);  /* or however you do this */
>   kaiser_add_mapping(pte_pfn(pte), pte_prot(pte));
> }
> 
> modulo the huge pile of typos in there that surely exist.

That would work.  I just need to find a suitable pte_offset_k() in the
kernel and make sure it works for these purposes.  We probably have one.

> But I still prefer my approach of just sharing the cpu_entry_area pmd
> entries between the user and kernel tables.

That would be spiffy.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Tue, Nov 21, 2017 at 3:42 PM, Dave Hansen
 wrote:
> On 11/21/2017 03:32 PM, Andy Lutomirski wrote:
>>> To do this, we need to special-case the kernel page table walker to deal
>>> with PTEs only since we can't just grab PMD or PUD flags and stick them
>>> in a PTE.  We would only be able to use this path when populating things
>>> that we know are 4k-mapped in the kernel.
>> I'm not sure I'm understanding the issue.  We'd promise to map the
>> cpu_entry_area without using large pages, but I'm not sure I know what
>> you're referring to.  The only issue I see is that we'd have to be
>> quite careful when tearing down the user tables to avoid freeing the
>> shared part.
>
> It's just that it currently handles large and small pages in the kernel
> mapping that it's copying.  If we want to have it just copy the PTE,
> we've got to refactor things a bit to separate out the PTE flags from
> the paddr being targeted, and also make sure we don't munge the flags
> conversion from the large-page entries to 4k PTEs.  The PAT and PSE bits
> cause a bit of trouble here.

I'm confused.  I mean something like:

unsigned long start = (unsigned long)get_cpu_entry_area(cpu);
for (unsigned long addr = start; addr < start + sizeof(struct
cpu_entry_area); addr += PAGE_SIZE) {
  pte_t pte = *pte_offset_k(addr);  /* or however you do this */
  kaiser_add_mapping(pte_pfn(pte), pte_prot(pte));
}

modulo the huge pile of typos in there that surely exist.

But I still prefer my approach of just sharing the cpu_entry_area pmd
entries between the user and kernel tables.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Tue, Nov 21, 2017 at 3:42 PM, Dave Hansen
 wrote:
> On 11/21/2017 03:32 PM, Andy Lutomirski wrote:
>>> To do this, we need to special-case the kernel page table walker to deal
>>> with PTEs only since we can't just grab PMD or PUD flags and stick them
>>> in a PTE.  We would only be able to use this path when populating things
>>> that we know are 4k-mapped in the kernel.
>> I'm not sure I'm understanding the issue.  We'd promise to map the
>> cpu_entry_area without using large pages, but I'm not sure I know what
>> you're referring to.  The only issue I see is that we'd have to be
>> quite careful when tearing down the user tables to avoid freeing the
>> shared part.
>
> It's just that it currently handles large and small pages in the kernel
> mapping that it's copying.  If we want to have it just copy the PTE,
> we've got to refactor things a bit to separate out the PTE flags from
> the paddr being targeted, and also make sure we don't munge the flags
> conversion from the large-page entries to 4k PTEs.  The PAT and PSE bits
> cause a bit of trouble here.

I'm confused.  I mean something like:

unsigned long start = (unsigned long)get_cpu_entry_area(cpu);
for (unsigned long addr = start; addr < start + sizeof(struct
cpu_entry_area); addr += PAGE_SIZE) {
  pte_t pte = *pte_offset_k(addr);  /* or however you do this */
  kaiser_add_mapping(pte_pfn(pte), pte_prot(pte));
}

modulo the huge pile of typos in there that surely exist.

But I still prefer my approach of just sharing the cpu_entry_area pmd
entries between the user and kernel tables.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 03:32 PM, Andy Lutomirski wrote:
>> To do this, we need to special-case the kernel page table walker to deal
>> with PTEs only since we can't just grab PMD or PUD flags and stick them
>> in a PTE.  We would only be able to use this path when populating things
>> that we know are 4k-mapped in the kernel.
> I'm not sure I'm understanding the issue.  We'd promise to map the
> cpu_entry_area without using large pages, but I'm not sure I know what
> you're referring to.  The only issue I see is that we'd have to be
> quite careful when tearing down the user tables to avoid freeing the
> shared part.

It's just that it currently handles large and small pages in the kernel
mapping that it's copying.  If we want to have it just copy the PTE,
we've got to refactor things a bit to separate out the PTE flags from
the paddr being targeted, and also make sure we don't munge the flags
conversion from the large-page entries to 4k PTEs.  The PAT and PSE bits
cause a bit of trouble here.

IOW, it would make the call-sites look cleaner, but it largely just
shifts the complexity elsewhere.  But, either way, it's all contained to
kaiser.c

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 03:32 PM, Andy Lutomirski wrote:
>> To do this, we need to special-case the kernel page table walker to deal
>> with PTEs only since we can't just grab PMD or PUD flags and stick them
>> in a PTE.  We would only be able to use this path when populating things
>> that we know are 4k-mapped in the kernel.
> I'm not sure I'm understanding the issue.  We'd promise to map the
> cpu_entry_area without using large pages, but I'm not sure I know what
> you're referring to.  The only issue I see is that we'd have to be
> quite careful when tearing down the user tables to avoid freeing the
> shared part.

It's just that it currently handles large and small pages in the kernel
mapping that it's copying.  If we want to have it just copy the PTE,
we've got to refactor things a bit to separate out the PTE flags from
the paddr being targeted, and also make sure we don't munge the flags
conversion from the large-page entries to 4k PTEs.  The PAT and PSE bits
cause a bit of trouble here.

IOW, it would make the call-sites look cleaner, but it largely just
shifts the complexity elsewhere.  But, either way, it's all contained to
kaiser.c

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Tue, Nov 21, 2017 at 3:17 PM, Dave Hansen
 wrote:
> On 11/21/2017 02:46 PM, Andy Lutomirski wrote:
>>> GDT: R/O TSS: R/W at least because of trampoline stack entry code:
>>> EXEC+R/O exception stacks: R/W
>> Can you avoid code duplication by adding some logic right after the
>> kernel cpu_entry_area is set up to iterate page by page over the PTEs
>> in the cpu_entry_area for that CPU and just install exactly the same
>> PTEs into the kaiser table?  E.g. just call kaiser_add_mapping once
>> per page but with the parameters read out from the fixmap PTEs
>> instead of hard coded?
>
> Yes, we could do that.  But, what's the gain?  We end up removing
> effectively three (long) lines of code from three kaiser_add_mapping()
> calls.

I'm hoping we can remove kaiser_add_mapping() entirely.  Maybe that's
silly optimism.

>
> To do this, we need to special-case the kernel page table walker to deal
> with PTEs only since we can't just grab PMD or PUD flags and stick them
> in a PTE.  We would only be able to use this path when populating things
> that we know are 4k-mapped in the kernel.

I'm not sure I'm understanding the issue.  We'd promise to map the
cpu_entry_area without using large pages, but I'm not sure I know what
you're referring to.  The only issue I see is that we'd have to be
quite careful when tearing down the user tables to avoid freeing the
shared part.

>
> I guess the upside is that we don't open-code the permissions in the
> KAISER code that *have* to match the permissions that the kernel itself
> established.
>
> It also means that theoretically you could not touch the KAISER code the
> next time we expand the cpu entry area.

I definitely like that part.

--Andy

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Tue, Nov 21, 2017 at 3:17 PM, Dave Hansen
 wrote:
> On 11/21/2017 02:46 PM, Andy Lutomirski wrote:
>>> GDT: R/O TSS: R/W at least because of trampoline stack entry code:
>>> EXEC+R/O exception stacks: R/W
>> Can you avoid code duplication by adding some logic right after the
>> kernel cpu_entry_area is set up to iterate page by page over the PTEs
>> in the cpu_entry_area for that CPU and just install exactly the same
>> PTEs into the kaiser table?  E.g. just call kaiser_add_mapping once
>> per page but with the parameters read out from the fixmap PTEs
>> instead of hard coded?
>
> Yes, we could do that.  But, what's the gain?  We end up removing
> effectively three (long) lines of code from three kaiser_add_mapping()
> calls.

I'm hoping we can remove kaiser_add_mapping() entirely.  Maybe that's
silly optimism.

>
> To do this, we need to special-case the kernel page table walker to deal
> with PTEs only since we can't just grab PMD or PUD flags and stick them
> in a PTE.  We would only be able to use this path when populating things
> that we know are 4k-mapped in the kernel.

I'm not sure I'm understanding the issue.  We'd promise to map the
cpu_entry_area without using large pages, but I'm not sure I know what
you're referring to.  The only issue I see is that we'd have to be
quite careful when tearing down the user tables to avoid freeing the
shared part.

>
> I guess the upside is that we don't open-code the permissions in the
> KAISER code that *have* to match the permissions that the kernel itself
> established.
>
> It also means that theoretically you could not touch the KAISER code the
> next time we expand the cpu entry area.

I definitely like that part.

--Andy

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 02:46 PM, Andy Lutomirski wrote:
>> GDT: R/O TSS: R/W at least because of trampoline stack entry code:
>> EXEC+R/O exception stacks: R/W
> Can you avoid code duplication by adding some logic right after the
> kernel cpu_entry_area is set up to iterate page by page over the PTEs
> in the cpu_entry_area for that CPU and just install exactly the same
> PTEs into the kaiser table?  E.g. just call kaiser_add_mapping once
> per page but with the parameters read out from the fixmap PTEs
> instead of hard coded?

Yes, we could do that.  But, what's the gain?  We end up removing
effectively three (long) lines of code from three kaiser_add_mapping()
calls.

To do this, we need to special-case the kernel page table walker to deal
with PTEs only since we can't just grab PMD or PUD flags and stick them
in a PTE.  We would only be able to use this path when populating things
that we know are 4k-mapped in the kernel.

I guess the upside is that we don't open-code the permissions in the
KAISER code that *have* to match the permissions that the kernel itself
established.

It also means that theoretically you could not touch the KAISER code the
next time we expand the cpu entry area.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/21/2017 02:46 PM, Andy Lutomirski wrote:
>> GDT: R/O TSS: R/W at least because of trampoline stack entry code:
>> EXEC+R/O exception stacks: R/W
> Can you avoid code duplication by adding some logic right after the
> kernel cpu_entry_area is set up to iterate page by page over the PTEs
> in the cpu_entry_area for that CPU and just install exactly the same
> PTEs into the kaiser table?  E.g. just call kaiser_add_mapping once
> per page but with the parameters read out from the fixmap PTEs
> instead of hard coded?

Yes, we could do that.  But, what's the gain?  We end up removing
effectively three (long) lines of code from three kaiser_add_mapping()
calls.

To do this, we need to special-case the kernel page table walker to deal
with PTEs only since we can't just grab PMD or PUD flags and stick them
in a PTE.  We would only be able to use this path when populating things
that we know are 4k-mapped in the kernel.

I guess the upside is that we don't open-code the permissions in the
KAISER code that *have* to match the permissions that the kernel itself
established.

It also means that theoretically you could not touch the KAISER code the
next time we expand the cpu entry area.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

> On Nov 21, 2017, at 2:19 PM, Dave Hansen  wrote:
> 
> On 11/20/2017 12:46 PM, Andy Lutomirski wrote:
 + /*
 +  * We could theoretically do this in setup_fixmap_gdt().
 +  * But, we would need to rewrite the above page table
 +  * allocation code to use the bootmem allocator.  The
 +  * buddy allocator is not available at the time that we
 +  * call setup_fixmap_gdt() for CPU 0.
 +  */
 + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
 +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>>> This one is needs to stay.
>> When you rebase on to my latest version, this should change to mapping
>> the entire cpu_entry_area.
> 
> I did this, but unfortunately it ends up having to individually map all
> four pieces of cpu_entry_area.  They all need different permissions and
> while theoretically we could do TSS+exception-stacks in the same call,
> they're not next to each other:
> 
> GDT: R/O
> TSS: R/W at least because of trampoline stack
> entry code: EXEC+R/O
> exception stacks: R/W

Can you avoid code duplication by adding some logic right after the kernel 
cpu_entry_area is set up to iterate page by page over the PTEs in the 
cpu_entry_area for that CPU and just install exactly the same PTEs into the 
kaiser table?  E.g. just call kaiser_add_mapping once per page but with the 
parameters read out from the fixmap PTEs instead of hard coded?

As a fancier but maybe better option, we could fiddle with the fixmap indices 
so that the whole cpu_entry_area range is aligned to a PMD boundary or higher.  
We'd preallocate all the page tables for this range before booting any APs.  
Then the kaiser tables could just reference the same page tables, and we don't 
need any AP kaiser setup at all.

This should be a wee bit faster, too, since we reduce the number of cache lines 
needed to refill the TLB when needed.

I'm really hoping we can get rid of kaiser_add_mapping entirely.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

> On Nov 21, 2017, at 2:19 PM, Dave Hansen  wrote:
> 
> On 11/20/2017 12:46 PM, Andy Lutomirski wrote:
 + /*
 +  * We could theoretically do this in setup_fixmap_gdt().
 +  * But, we would need to rewrite the above page table
 +  * allocation code to use the bootmem allocator.  The
 +  * buddy allocator is not available at the time that we
 +  * call setup_fixmap_gdt() for CPU 0.
 +  */
 + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
 +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>>> This one is needs to stay.
>> When you rebase on to my latest version, this should change to mapping
>> the entire cpu_entry_area.
> 
> I did this, but unfortunately it ends up having to individually map all
> four pieces of cpu_entry_area.  They all need different permissions and
> while theoretically we could do TSS+exception-stacks in the same call,
> they're not next to each other:
> 
> GDT: R/O
> TSS: R/W at least because of trampoline stack
> entry code: EXEC+R/O
> exception stacks: R/W

Can you avoid code duplication by adding some logic right after the kernel 
cpu_entry_area is set up to iterate page by page over the PTEs in the 
cpu_entry_area for that CPU and just install exactly the same PTEs into the 
kaiser table?  E.g. just call kaiser_add_mapping once per page but with the 
parameters read out from the fixmap PTEs instead of hard coded?

As a fancier but maybe better option, we could fiddle with the fixmap indices 
so that the whole cpu_entry_area range is aligned to a PMD boundary or higher.  
We'd preallocate all the page tables for this range before booting any APs.  
Then the kaiser tables could just reference the same page tables, and we don't 
need any AP kaiser setup at all.

This should be a wee bit faster, too, since we reduce the number of cache lines 
needed to refill the TLB when needed.

I'm really hoping we can get rid of kaiser_add_mapping entirely.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/20/2017 12:22 PM, Thomas Gleixner wrote:
> On Fri, 10 Nov 2017, Dave Hansen wrote:
>>  __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
>> +
>> +/* CPU 0's mapping is done in kaiser_init() */
>> +if (cpu) {
>> +int ret;
>> +
>> +ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
>> + PAGE_SIZE, __PAGE_KERNEL_RO);
>> +/*
>> + * We do not have a good way to fail CPU bringup.
>> + * Just WARN about it and hope we boot far enough
>> + * to get a good log out.
>> + */
> 
> The GDT fixmap can be set up before the CPU is started. There is no reason
> to do that in cpu_init().

Do you mean the __set_fixmap(), or my call to kaiser_add_mapping()?

Where would you suggest we move it?  Here seems kinda nice because it's
right next to where the get_cpu_gdt_ro() mapping is created.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/20/2017 12:22 PM, Thomas Gleixner wrote:
> On Fri, 10 Nov 2017, Dave Hansen wrote:
>>  __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
>> +
>> +/* CPU 0's mapping is done in kaiser_init() */
>> +if (cpu) {
>> +int ret;
>> +
>> +ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
>> + PAGE_SIZE, __PAGE_KERNEL_RO);
>> +/*
>> + * We do not have a good way to fail CPU bringup.
>> + * Just WARN about it and hope we boot far enough
>> + * to get a good log out.
>> + */
> 
> The GDT fixmap can be set up before the CPU is started. There is no reason
> to do that in cpu_init().

Do you mean the __set_fixmap(), or my call to kaiser_add_mapping()?

Where would you suggest we move it?  Here seems kinda nice because it's
right next to where the get_cpu_gdt_ro() mapping is created.

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/20/2017 12:46 PM, Andy Lutomirski wrote:
>>> + /*
>>> +  * We could theoretically do this in setup_fixmap_gdt().
>>> +  * But, we would need to rewrite the above page table
>>> +  * allocation code to use the bootmem allocator.  The
>>> +  * buddy allocator is not available at the time that we
>>> +  * call setup_fixmap_gdt() for CPU 0.
>>> +  */
>>> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
>>> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>> This one is needs to stay.
> When you rebase on to my latest version, this should change to mapping
> the entire cpu_entry_area.

I did this, but unfortunately it ends up having to individually map all
four pieces of cpu_entry_area.  They all need different permissions and
while theoretically we could do TSS+exception-stacks in the same call,
they're not next to each other:

 GDT: R/O
 TSS: R/W at least because of trampoline stack
 entry code: EXEC+R/O
 exception stacks: R/W

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Dave Hansen]

On 11/20/2017 12:46 PM, Andy Lutomirski wrote:
>>> + /*
>>> +  * We could theoretically do this in setup_fixmap_gdt().
>>> +  * But, we would need to rewrite the above page table
>>> +  * allocation code to use the bootmem allocator.  The
>>> +  * buddy allocator is not available at the time that we
>>> +  * call setup_fixmap_gdt() for CPU 0.
>>> +  */
>>> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
>>> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>> This one is needs to stay.
> When you rebase on to my latest version, this should change to mapping
> the entire cpu_entry_area.

I did this, but unfortunately it ends up having to individually map all
four pieces of cpu_entry_area.  They all need different permissions and
while theoretically we could do TSS+exception-stacks in the same call,
they're not next to each other:

 GDT: R/O
 TSS: R/W at least because of trampoline stack
 entry code: EXEC+R/O
 exception stacks: R/W

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Thomas Gleixner]

On Mon, 20 Nov 2017, Andy Lutomirski wrote:
> On Mon, Nov 20, 2017 at 12:22 PM, Thomas Gleixner  wrote:
> > On Fri, 10 Nov 2017, Dave Hansen wrote:
> >>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), 
> >> prot);
> >> +
> >> + /* CPU 0's mapping is done in kaiser_init() */
> >> + if (cpu) {
> >> + int ret;
> >> +
> >> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
> >> +  PAGE_SIZE, __PAGE_KERNEL_RO);
> >> + /*
> >> +  * We do not have a good way to fail CPU bringup.
> >> +  * Just WARN about it and hope we boot far enough
> >> +  * to get a good log out.
> >> +  */
> >
> > The GDT fixmap can be set up before the CPU is started. There is no reason
> > to do that in cpu_init().
> >
> >> +
> >> + /*
> >> +  * We could theoretically do this in setup_fixmap_gdt().
> >> +  * But, we would need to rewrite the above page table
> >> +  * allocation code to use the bootmem allocator.  The
> >> +  * buddy allocator is not available at the time that we
> >> +  * call setup_fixmap_gdt() for CPU 0.
> >> +  */
> >> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
> >> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
> >
> > This one is needs to stay.
> 
> When you rebase on to my latest version, this should change to mapping
> the entire cpu_entry_area.

Too much flux left and right :)

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Thomas Gleixner]

On Mon, 20 Nov 2017, Andy Lutomirski wrote:
> On Mon, Nov 20, 2017 at 12:22 PM, Thomas Gleixner  wrote:
> > On Fri, 10 Nov 2017, Dave Hansen wrote:
> >>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), 
> >> prot);
> >> +
> >> + /* CPU 0's mapping is done in kaiser_init() */
> >> + if (cpu) {
> >> + int ret;
> >> +
> >> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
> >> +  PAGE_SIZE, __PAGE_KERNEL_RO);
> >> + /*
> >> +  * We do not have a good way to fail CPU bringup.
> >> +  * Just WARN about it and hope we boot far enough
> >> +  * to get a good log out.
> >> +  */
> >
> > The GDT fixmap can be set up before the CPU is started. There is no reason
> > to do that in cpu_init().
> >
> >> +
> >> + /*
> >> +  * We could theoretically do this in setup_fixmap_gdt().
> >> +  * But, we would need to rewrite the above page table
> >> +  * allocation code to use the bootmem allocator.  The
> >> +  * buddy allocator is not available at the time that we
> >> +  * call setup_fixmap_gdt() for CPU 0.
> >> +  */
> >> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
> >> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
> >
> > This one is needs to stay.
> 
> When you rebase on to my latest version, this should change to mapping
> the entire cpu_entry_area.

Too much flux left and right :)

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Mon, Nov 20, 2017 at 12:22 PM, Thomas Gleixner  wrote:
> On Fri, 10 Nov 2017, Dave Hansen wrote:
>>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
>> +
>> + /* CPU 0's mapping is done in kaiser_init() */
>> + if (cpu) {
>> + int ret;
>> +
>> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
>> +  PAGE_SIZE, __PAGE_KERNEL_RO);
>> + /*
>> +  * We do not have a good way to fail CPU bringup.
>> +  * Just WARN about it and hope we boot far enough
>> +  * to get a good log out.
>> +  */
>
> The GDT fixmap can be set up before the CPU is started. There is no reason
> to do that in cpu_init().
>
>> +
>> + /*
>> +  * We could theoretically do this in setup_fixmap_gdt().
>> +  * But, we would need to rewrite the above page table
>> +  * allocation code to use the bootmem allocator.  The
>> +  * buddy allocator is not available at the time that we
>> +  * call setup_fixmap_gdt() for CPU 0.
>> +  */
>> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
>> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>
> This one is needs to stay.

When you rebase on to my latest version, this should change to mapping
the entire cpu_entry_area.

--Andy

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Andy Lutomirski]

On Mon, Nov 20, 2017 at 12:22 PM, Thomas Gleixner  wrote:
> On Fri, 10 Nov 2017, Dave Hansen wrote:
>>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
>> +
>> + /* CPU 0's mapping is done in kaiser_init() */
>> + if (cpu) {
>> + int ret;
>> +
>> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
>> +  PAGE_SIZE, __PAGE_KERNEL_RO);
>> + /*
>> +  * We do not have a good way to fail CPU bringup.
>> +  * Just WARN about it and hope we boot far enough
>> +  * to get a good log out.
>> +  */
>
> The GDT fixmap can be set up before the CPU is started. There is no reason
> to do that in cpu_init().
>
>> +
>> + /*
>> +  * We could theoretically do this in setup_fixmap_gdt().
>> +  * But, we would need to rewrite the above page table
>> +  * allocation code to use the bootmem allocator.  The
>> +  * buddy allocator is not available at the time that we
>> +  * call setup_fixmap_gdt() for CPU 0.
>> +  */
>> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
>> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);
>
> This one is needs to stay.

When you rebase on to my latest version, this should change to mapping
the entire cpu_entry_area.

--Andy

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Thomas Gleixner]

On Fri, 10 Nov 2017, Dave Hansen wrote:
>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
> +
> + /* CPU 0's mapping is done in kaiser_init() */
> + if (cpu) {
> + int ret;
> +
> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
> +  PAGE_SIZE, __PAGE_KERNEL_RO);
> + /*
> +  * We do not have a good way to fail CPU bringup.
> +  * Just WARN about it and hope we boot far enough
> +  * to get a good log out.
> +  */

The GDT fixmap can be set up before the CPU is started. There is no reason
to do that in cpu_init().

> +
> + /*
> +  * We could theoretically do this in setup_fixmap_gdt().
> +  * But, we would need to rewrite the above page table
> +  * allocation code to use the bootmem allocator.  The
> +  * buddy allocator is not available at the time that we
> +  * call setup_fixmap_gdt() for CPU 0.
> +  */
> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);

This one is needs to stay.

Thanks,

tglx

Re: [PATCH 12/30] x86, kaiser: map GDT into user page tables

[Thomas Gleixner]

On Fri, 10 Nov 2017, Dave Hansen wrote:
>   __set_fixmap(get_cpu_gdt_ro_index(cpu), get_cpu_gdt_paddr(cpu), prot);
> +
> + /* CPU 0's mapping is done in kaiser_init() */
> + if (cpu) {
> + int ret;
> +
> + ret = kaiser_add_mapping((unsigned long) get_cpu_gdt_ro(cpu),
> +  PAGE_SIZE, __PAGE_KERNEL_RO);
> + /*
> +  * We do not have a good way to fail CPU bringup.
> +  * Just WARN about it and hope we boot far enough
> +  * to get a good log out.
> +  */

The GDT fixmap can be set up before the CPU is started. There is no reason
to do that in cpu_init().

> +
> + /*
> +  * We could theoretically do this in setup_fixmap_gdt().
> +  * But, we would need to rewrite the above page table
> +  * allocation code to use the bootmem allocator.  The
> +  * buddy allocator is not available at the time that we
> +  * call setup_fixmap_gdt() for CPU 0.
> +  */
> + kaiser_add_user_map_early(get_cpu_gdt_ro(0), PAGE_SIZE,
> +   __PAGE_KERNEL_RO | _PAGE_GLOBAL);

This one is needs to stay.

Thanks,

tglx