Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Dou Liyang]

Hi Ingo, Kees, Baoquan and Chao

At 03/12/2018 06:57 PM, Ingo Molnar wrote:
[...]

So there's apparently a mis-design here:

  - KASLR needs to be done very early on during bootup: - it's not realistic to
expect KASLR to be done with a booted up kernel, because pointers to various
KASLR-ed objects are already widely spread out in memory.

  - But for some unfathomable reason the memory hotplug attribute of memory
regions is not part of the regular memory map but part of late-init ACPI 
data
structures.

The right solution would be _not_ to fudge the KASLR location, but to provide 
the
memory hotplug information to early code, preferably via the primary memory map.
KASLR can then make use of it and avoid those regions, just like it avoids other
memory regions already.

In addition to that hardware makers (including virtualized hardware) should also
fix their systems to provide memory hotplug information to early code.



Yes, but before this, can we fix this problem by the following patch 
which has been sent and reviewed by Kees before[1]. its solution is:


  Extend movable_node option to restrict kernel to be randomized in
  immovable nodes by adding a parameter. this parameter sets up
  the boundaries between the home nodes and other nodes.

My reason is here:

  - What we really want to solve is the KASLR breaks *physical Node
    hotplug*, Keep the decompressed kernel in an immovable node is
    enough.

  - AFAICS, there are not too many systems where physical Node hotplug
actually works in practice, and there mush be one node called *home
    node* which is immovable for storing basic information.

  - the node in modern systems could have double-digit gigabytes memory,
    It can completely satisfy the operation of KASLR.

So, Just restrict kernel to be randomized in the home node, and ignore
other nodes when kernel has the *movable_node* option in the command
line.

Thoughts? may I rebase and resend the patch?

[1] https://lkml.org/lkml/2017/8/3/401

Thanks,

dou

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Dou Liyang]

Hi Ingo, Kees, Baoquan and Chao

At 03/12/2018 06:57 PM, Ingo Molnar wrote:
[...]

So there's apparently a mis-design here:

  - KASLR needs to be done very early on during bootup: - it's not realistic to
expect KASLR to be done with a booted up kernel, because pointers to various
KASLR-ed objects are already widely spread out in memory.

  - But for some unfathomable reason the memory hotplug attribute of memory
regions is not part of the regular memory map but part of late-init ACPI 
data
structures.

The right solution would be _not_ to fudge the KASLR location, but to provide 
the
memory hotplug information to early code, preferably via the primary memory map.
KASLR can then make use of it and avoid those regions, just like it avoids other
memory regions already.

In addition to that hardware makers (including virtualized hardware) should also
fix their systems to provide memory hotplug information to early code.



Yes, but before this, can we fix this problem by the following patch 
which has been sent and reviewed by Kees before[1]. its solution is:


  Extend movable_node option to restrict kernel to be randomized in
  immovable nodes by adding a parameter. this parameter sets up
  the boundaries between the home nodes and other nodes.

My reason is here:

  - What we really want to solve is the KASLR breaks *physical Node
    hotplug*, Keep the decompressed kernel in an immovable node is
    enough.

  - AFAICS, there are not too many systems where physical Node hotplug
actually works in practice, and there mush be one node called *home
    node* which is immovable for storing basic information.

  - the node in modern systems could have double-digit gigabytes memory,
    It can completely satisfy the operation of KASLR.

So, Just restrict kernel to be randomized in the home node, and ignore
other nodes when kernel has the *movable_node* option in the command
line.

Thoughts? may I rebase and resend the patch?

[1] https://lkml.org/lkml/2017/8/3/401

Thanks,

dou

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Chao Fan]

On Mon, Mar 19, 2018 at 03:24:46PM +0800, Baoquan He wrote:
>On 03/12/18 at 08:04pm, Chao Fan wrote:
>> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
>> >
>> >* Baoquan He  wrote:
>> >
>> >> Hi Ingo,
>> >> 
>> >> On 03/12/18 at 10:35am, Ingo Molnar wrote:
>> >> > 
>> >> > * Chao Fan  wrote:
>> >> > 
>> >> > > Long time no reply, rebase the patchset, change the parameter name
>> >> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
>> >> > > 
>> >> > > ***Background:
>> >> > > People reported that kaslr may randomly chooses some positions
>> >> > > which are located in movable memory regions. This will break memory
>> >> > > hotplug feature.
>> >> > 
>> >> > [...]
>> >> > 
>> >> > > ***Solutions:
>> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users 
>> >> > > to
>> >> > > specify the memory regions where kernel can be allowed to randomize
>> >> > > safely.
>> >> > 
>> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
>> >> > 
>> >> > In what way does memory hotplug feature 'break'? Does it crash or 
>> >> > misbehave? Or 
>> >> > simply does it not allow the movement of the affected memory region, 
>> >> > while still 
>> >> > allowing the rest to be moved?
>> >> 
>> >> AFAIT, if kernel is randomized into the movable memory region, the
>> >> affected memory region can not be hot added/removed since it has kernel
>> >> data. Surely, the system can still work, the unaffected part still can
>> >> be moved. Still it will cause regression on memory hotplug.
>> >> 
>> >> Mainly we parse SRAT table to get the ranges of memory provided by
>> >> hot-added memory devices in initmem_init(), that's very late. During boot,
>> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
>> >> stage, the code is very complicated and not elegant, ACPI maintainer
>> >> NACKed that.
>
>Hi Chao,
>
>Seems Ingo prefers the handling in kaslr boot code. Maybe you can try
>to optimize and split your below patch and post anouther round?
>
>I will see how to sove the hugepage in boot/compressed/kaslr.c . 

Yes, seems that I need to pick up the old patch and try to optimize it.

Thanks,
Chao Fan

>
>Thanks
>Baoquan
>
>> 
>> Thanks for Ingo's suggestion and Baoquan's explaination.
>> 
>> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
>> https://lkml.org/lkml/2017/9/3/77
>> But the change is too huge so made this patchset to avoid this bug in a
>> small change, which will not make the code looks messy.
>> 
>> Thanks,
>> Chao Fan
>> 
>> >
>> >So there's apparently a mis-design here:
>> >
>> > - KASLR needs to be done very early on during bootup: - it's not realistic 
>> > to 
>> >   expect KASLR to be done with a booted up kernel, because pointers to 
>> > various 
>> >   KASLR-ed objects are already widely spread out in memory.
>> >
>> > - But for some unfathomable reason the memory hotplug attribute of memory
>> >   regions is not part of the regular memory map but part of late-init ACPI 
>> > data
>> >   structures.
>> >
>> >The right solution would be _not_ to fudge the KASLR location, but to 
>> >provide the 
>> >memory hotplug information to early code, preferably via the primary memory 
>> >map. 
>> >KASLR can then make use of it and avoid those regions, just like it avoids 
>> >other 
>> >memory regions already.
>> >
>> >In addition to that hardware makers (including virtualized hardware) should 
>> >also 
>> >fix their systems to provide memory hotplug information to early code.
>> >
>> >Thanks,
>> >
>> >Ingo
>> >
>> >
>> 
>> 
>
>

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Chao Fan]

On Mon, Mar 19, 2018 at 03:24:46PM +0800, Baoquan He wrote:
>On 03/12/18 at 08:04pm, Chao Fan wrote:
>> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
>> >
>> >* Baoquan He  wrote:
>> >
>> >> Hi Ingo,
>> >> 
>> >> On 03/12/18 at 10:35am, Ingo Molnar wrote:
>> >> > 
>> >> > * Chao Fan  wrote:
>> >> > 
>> >> > > Long time no reply, rebase the patchset, change the parameter name
>> >> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
>> >> > > 
>> >> > > ***Background:
>> >> > > People reported that kaslr may randomly chooses some positions
>> >> > > which are located in movable memory regions. This will break memory
>> >> > > hotplug feature.
>> >> > 
>> >> > [...]
>> >> > 
>> >> > > ***Solutions:
>> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users 
>> >> > > to
>> >> > > specify the memory regions where kernel can be allowed to randomize
>> >> > > safely.
>> >> > 
>> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
>> >> > 
>> >> > In what way does memory hotplug feature 'break'? Does it crash or 
>> >> > misbehave? Or 
>> >> > simply does it not allow the movement of the affected memory region, 
>> >> > while still 
>> >> > allowing the rest to be moved?
>> >> 
>> >> AFAIT, if kernel is randomized into the movable memory region, the
>> >> affected memory region can not be hot added/removed since it has kernel
>> >> data. Surely, the system can still work, the unaffected part still can
>> >> be moved. Still it will cause regression on memory hotplug.
>> >> 
>> >> Mainly we parse SRAT table to get the ranges of memory provided by
>> >> hot-added memory devices in initmem_init(), that's very late. During boot,
>> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
>> >> stage, the code is very complicated and not elegant, ACPI maintainer
>> >> NACKed that.
>
>Hi Chao,
>
>Seems Ingo prefers the handling in kaslr boot code. Maybe you can try
>to optimize and split your below patch and post anouther round?
>
>I will see how to sove the hugepage in boot/compressed/kaslr.c . 

Yes, seems that I need to pick up the old patch and try to optimize it.

Thanks,
Chao Fan

>
>Thanks
>Baoquan
>
>> 
>> Thanks for Ingo's suggestion and Baoquan's explaination.
>> 
>> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
>> https://lkml.org/lkml/2017/9/3/77
>> But the change is too huge so made this patchset to avoid this bug in a
>> small change, which will not make the code looks messy.
>> 
>> Thanks,
>> Chao Fan
>> 
>> >
>> >So there's apparently a mis-design here:
>> >
>> > - KASLR needs to be done very early on during bootup: - it's not realistic 
>> > to 
>> >   expect KASLR to be done with a booted up kernel, because pointers to 
>> > various 
>> >   KASLR-ed objects are already widely spread out in memory.
>> >
>> > - But for some unfathomable reason the memory hotplug attribute of memory
>> >   regions is not part of the regular memory map but part of late-init ACPI 
>> > data
>> >   structures.
>> >
>> >The right solution would be _not_ to fudge the KASLR location, but to 
>> >provide the 
>> >memory hotplug information to early code, preferably via the primary memory 
>> >map. 
>> >KASLR can then make use of it and avoid those regions, just like it avoids 
>> >other 
>> >memory regions already.
>> >
>> >In addition to that hardware makers (including virtualized hardware) should 
>> >also 
>> >fix their systems to provide memory hotplug information to early code.
>> >
>> >Thanks,
>> >
>> >Ingo
>> >
>> >
>> 
>> 
>
>

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

On 03/12/18 at 08:04pm, Chao Fan wrote:
> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
> >
> >* Baoquan He  wrote:
> >
> >> Hi Ingo,
> >> 
> >> On 03/12/18 at 10:35am, Ingo Molnar wrote:
> >> > 
> >> > * Chao Fan  wrote:
> >> > 
> >> > > Long time no reply, rebase the patchset, change the parameter name
> >> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> >> > > 
> >> > > ***Background:
> >> > > People reported that kaslr may randomly chooses some positions
> >> > > which are located in movable memory regions. This will break memory
> >> > > hotplug feature.
> >> > 
> >> > [...]
> >> > 
> >> > > ***Solutions:
> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> >> > > specify the memory regions where kernel can be allowed to randomize
> >> > > safely.
> >> > 
> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
> >> > 
> >> > In what way does memory hotplug feature 'break'? Does it crash or 
> >> > misbehave? Or 
> >> > simply does it not allow the movement of the affected memory region, 
> >> > while still 
> >> > allowing the rest to be moved?
> >> 
> >> AFAIT, if kernel is randomized into the movable memory region, the
> >> affected memory region can not be hot added/removed since it has kernel
> >> data. Surely, the system can still work, the unaffected part still can
> >> be moved. Still it will cause regression on memory hotplug.
> >> 
> >> Mainly we parse SRAT table to get the ranges of memory provided by
> >> hot-added memory devices in initmem_init(), that's very late. During boot,
> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> >> stage, the code is very complicated and not elegant, ACPI maintainer
> >> NACKed that.

Hi Chao,

Seems Ingo prefers the handling in kaslr boot code. Maybe you can try
to optimize and split your below patch and post anouther round?

I will see how to sove the hugepage in boot/compressed/kaslr.c . 

Thanks
Baoquan

> 
> Thanks for Ingo's suggestion and Baoquan's explaination.
> 
> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
> https://lkml.org/lkml/2017/9/3/77
> But the change is too huge so made this patchset to avoid this bug in a
> small change, which will not make the code looks messy.
> 
> Thanks,
> Chao Fan
> 
> >
> >So there's apparently a mis-design here:
> >
> > - KASLR needs to be done very early on during bootup: - it's not realistic 
> > to 
> >   expect KASLR to be done with a booted up kernel, because pointers to 
> > various 
> >   KASLR-ed objects are already widely spread out in memory.
> >
> > - But for some unfathomable reason the memory hotplug attribute of memory
> >   regions is not part of the regular memory map but part of late-init ACPI 
> > data
> >   structures.
> >
> >The right solution would be _not_ to fudge the KASLR location, but to 
> >provide the 
> >memory hotplug information to early code, preferably via the primary memory 
> >map. 
> >KASLR can then make use of it and avoid those regions, just like it avoids 
> >other 
> >memory regions already.
> >
> >In addition to that hardware makers (including virtualized hardware) should 
> >also 
> >fix their systems to provide memory hotplug information to early code.
> >
> >Thanks,
> >
> > Ingo
> >
> >
> 
> 

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

On 03/12/18 at 08:04pm, Chao Fan wrote:
> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
> >
> >* Baoquan He  wrote:
> >
> >> Hi Ingo,
> >> 
> >> On 03/12/18 at 10:35am, Ingo Molnar wrote:
> >> > 
> >> > * Chao Fan  wrote:
> >> > 
> >> > > Long time no reply, rebase the patchset, change the parameter name
> >> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> >> > > 
> >> > > ***Background:
> >> > > People reported that kaslr may randomly chooses some positions
> >> > > which are located in movable memory regions. This will break memory
> >> > > hotplug feature.
> >> > 
> >> > [...]
> >> > 
> >> > > ***Solutions:
> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> >> > > specify the memory regions where kernel can be allowed to randomize
> >> > > safely.
> >> > 
> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
> >> > 
> >> > In what way does memory hotplug feature 'break'? Does it crash or 
> >> > misbehave? Or 
> >> > simply does it not allow the movement of the affected memory region, 
> >> > while still 
> >> > allowing the rest to be moved?
> >> 
> >> AFAIT, if kernel is randomized into the movable memory region, the
> >> affected memory region can not be hot added/removed since it has kernel
> >> data. Surely, the system can still work, the unaffected part still can
> >> be moved. Still it will cause regression on memory hotplug.
> >> 
> >> Mainly we parse SRAT table to get the ranges of memory provided by
> >> hot-added memory devices in initmem_init(), that's very late. During boot,
> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> >> stage, the code is very complicated and not elegant, ACPI maintainer
> >> NACKed that.

Hi Chao,

Seems Ingo prefers the handling in kaslr boot code. Maybe you can try
to optimize and split your below patch and post anouther round?

I will see how to sove the hugepage in boot/compressed/kaslr.c . 

Thanks
Baoquan

> 
> Thanks for Ingo's suggestion and Baoquan's explaination.
> 
> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
> https://lkml.org/lkml/2017/9/3/77
> But the change is too huge so made this patchset to avoid this bug in a
> small change, which will not make the code looks messy.
> 
> Thanks,
> Chao Fan
> 
> >
> >So there's apparently a mis-design here:
> >
> > - KASLR needs to be done very early on during bootup: - it's not realistic 
> > to 
> >   expect KASLR to be done with a booted up kernel, because pointers to 
> > various 
> >   KASLR-ed objects are already widely spread out in memory.
> >
> > - But for some unfathomable reason the memory hotplug attribute of memory
> >   regions is not part of the regular memory map but part of late-init ACPI 
> > data
> >   structures.
> >
> >The right solution would be _not_ to fudge the KASLR location, but to 
> >provide the 
> >memory hotplug information to early code, preferably via the primary memory 
> >map. 
> >KASLR can then make use of it and avoid those regions, just like it avoids 
> >other 
> >memory regions already.
> >
> >In addition to that hardware makers (including virtualized hardware) should 
> >also 
> >fix their systems to provide memory hotplug information to early code.
> >
> >Thanks,
> >
> > Ingo
> >
> >
> 
> 

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

On 03/12/18 at 08:04pm, Chao Fan wrote:
> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
> >> > > ***Background:
> >> > > People reported that kaslr may randomly chooses some positions
> >> > > which are located in movable memory regions. This will break memory
> >> > > hotplug feature.
> >> > 
> >> > [...]
> >> > 
> >> > > ***Solutions:
> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> >> > > specify the memory regions where kernel can be allowed to randomize
> >> > > safely.
> >> > 
> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
> >> > 
> >> > In what way does memory hotplug feature 'break'? Does it crash or 
> >> > misbehave? Or 
> >> > simply does it not allow the movement of the affected memory region, 
> >> > while still 
> >> > allowing the rest to be moved?
> >> 
> >> AFAIT, if kernel is randomized into the movable memory region, the
> >> affected memory region can not be hot added/removed since it has kernel
> >> data. Surely, the system can still work, the unaffected part still can
> >> be moved. Still it will cause regression on memory hotplug.
> >> 
> >> Mainly we parse SRAT table to get the ranges of memory provided by
> >> hot-added memory devices in initmem_init(), that's very late. During boot,
> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> >> stage, the code is very complicated and not elegant, ACPI maintainer
> >> NACKed that.
> 
> Thanks for Ingo's suggestion and Baoquan's explaination.
> 
> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
> https://lkml.org/lkml/2017/9/3/77
> But the change is too huge so made this patchset to avoid this bug in a
> small change, which will not make the code looks messy.

ACPI tables are not independent, to parse SRAT to get information of
hotplug memory, we need get RSDP pointer, which points at RSDT or XSDT.
Then find SRAT from them. While RSDP is not in a fixed location, there
are several candidate positions, code can be checked in acpi_find_root_pointer()
of drivers/acpi/acpica/tbxfroot.c . And then iterate RSDT/XSDT to search
SRAT. These codes can not be reused between kaslr.c and drivers/acpi
because acpi code has special handling. So it will bloat kaslr boot
code. This is why both Rafael and I think it might be not good to grab
parse ACPI SRAT table in kaslr boot code.

> 
> >
> >So there's apparently a mis-design here:
> >
> > - KASLR needs to be done very early on during bootup: - it's not realistic 
> > to 
> >   expect KASLR to be done with a booted up kernel, because pointers to 
> > various 
> >   KASLR-ed objects are already widely spread out in memory.
> >
> > - But for some unfathomable reason the memory hotplug attribute of memory
> >   regions is not part of the regular memory map but part of late-init ACPI 
> > data
> >   structures.
> >
> >The right solution would be _not_ to fudge the KASLR location, but to 
> >provide the 
> >memory hotplug information to early code, preferably via the primary memory 
> >map. 
> >KASLR can then make use of it and avoid those regions, just like it avoids 
> >other 
> >memory regions already.
> >
> >In addition to that hardware makers (including virtualized hardware) should 
> >also 
> >fix their systems to provide memory hotplug information to early code.

The hugepage allocation on kvm guest is a different situation. If people
want to allocate n pages of 1G size, they will get one page less in
kaslr enabled kernel than kaslr disabled kernel, casually. Because
kernel might be randomized to those 1G aligned huge pages in kaslr
kernel. While in no kaslr case, kernel will be put at 16M.

default_hugepagesz=1G hugepagesz=1G hugepages='n'

For this issue, unless we use a algorithm to analyze kernel cmdline and
do a flexiable estimate to avoid those 1G aligned huge pages. Still we
can't avoid the case that memblock may break the good 1G page. I can't
think of a good way to fix this in kaslr boot code.

Thanks
Baoquan

> >
> >
> 
> 

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

On 03/12/18 at 08:04pm, Chao Fan wrote:
> On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
> >> > > ***Background:
> >> > > People reported that kaslr may randomly chooses some positions
> >> > > which are located in movable memory regions. This will break memory
> >> > > hotplug feature.
> >> > 
> >> > [...]
> >> > 
> >> > > ***Solutions:
> >> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> >> > > specify the memory regions where kernel can be allowed to randomize
> >> > > safely.
> >> > 
> >> > Manual solutions like that are pretty suboptimal to users, aren't they?
> >> > 
> >> > In what way does memory hotplug feature 'break'? Does it crash or 
> >> > misbehave? Or 
> >> > simply does it not allow the movement of the affected memory region, 
> >> > while still 
> >> > allowing the rest to be moved?
> >> 
> >> AFAIT, if kernel is randomized into the movable memory region, the
> >> affected memory region can not be hot added/removed since it has kernel
> >> data. Surely, the system can still work, the unaffected part still can
> >> be moved. Still it will cause regression on memory hotplug.
> >> 
> >> Mainly we parse SRAT table to get the ranges of memory provided by
> >> hot-added memory devices in initmem_init(), that's very late. During boot,
> >> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> >> stage, the code is very complicated and not elegant, ACPI maintainer
> >> NACKed that.
> 
> Thanks for Ingo's suggestion and Baoquan's explaination.
> 
> Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
> https://lkml.org/lkml/2017/9/3/77
> But the change is too huge so made this patchset to avoid this bug in a
> small change, which will not make the code looks messy.

ACPI tables are not independent, to parse SRAT to get information of
hotplug memory, we need get RSDP pointer, which points at RSDT or XSDT.
Then find SRAT from them. While RSDP is not in a fixed location, there
are several candidate positions, code can be checked in acpi_find_root_pointer()
of drivers/acpi/acpica/tbxfroot.c . And then iterate RSDT/XSDT to search
SRAT. These codes can not be reused between kaslr.c and drivers/acpi
because acpi code has special handling. So it will bloat kaslr boot
code. This is why both Rafael and I think it might be not good to grab
parse ACPI SRAT table in kaslr boot code.

> 
> >
> >So there's apparently a mis-design here:
> >
> > - KASLR needs to be done very early on during bootup: - it's not realistic 
> > to 
> >   expect KASLR to be done with a booted up kernel, because pointers to 
> > various 
> >   KASLR-ed objects are already widely spread out in memory.
> >
> > - But for some unfathomable reason the memory hotplug attribute of memory
> >   regions is not part of the regular memory map but part of late-init ACPI 
> > data
> >   structures.
> >
> >The right solution would be _not_ to fudge the KASLR location, but to 
> >provide the 
> >memory hotplug information to early code, preferably via the primary memory 
> >map. 
> >KASLR can then make use of it and avoid those regions, just like it avoids 
> >other 
> >memory regions already.
> >
> >In addition to that hardware makers (including virtualized hardware) should 
> >also 
> >fix their systems to provide memory hotplug information to early code.

The hugepage allocation on kvm guest is a different situation. If people
want to allocate n pages of 1G size, they will get one page less in
kaslr enabled kernel than kaslr disabled kernel, casually. Because
kernel might be randomized to those 1G aligned huge pages in kaslr
kernel. While in no kaslr case, kernel will be put at 16M.

default_hugepagesz=1G hugepagesz=1G hugepages='n'

For this issue, unless we use a algorithm to analyze kernel cmdline and
do a flexiable estimate to avoid those 1G aligned huge pages. Still we
can't avoid the case that memblock may break the good 1G page. I can't
think of a good way to fix this in kaslr boot code.

Thanks
Baoquan

> >
> >
> 
> 

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Chao Fan]

On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
>
>* Baoquan He  wrote:
>
>> Hi Ingo,
>> 
>> On 03/12/18 at 10:35am, Ingo Molnar wrote:
>> > 
>> > * Chao Fan  wrote:
>> > 
>> > > Long time no reply, rebase the patchset, change the parameter name
>> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
>> > > 
>> > > ***Background:
>> > > People reported that kaslr may randomly chooses some positions
>> > > which are located in movable memory regions. This will break memory
>> > > hotplug feature.
>> > 
>> > [...]
>> > 
>> > > ***Solutions:
>> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
>> > > specify the memory regions where kernel can be allowed to randomize
>> > > safely.
>> > 
>> > Manual solutions like that are pretty suboptimal to users, aren't they?
>> > 
>> > In what way does memory hotplug feature 'break'? Does it crash or 
>> > misbehave? Or 
>> > simply does it not allow the movement of the affected memory region, while 
>> > still 
>> > allowing the rest to be moved?
>> 
>> AFAIT, if kernel is randomized into the movable memory region, the
>> affected memory region can not be hot added/removed since it has kernel
>> data. Surely, the system can still work, the unaffected part still can
>> be moved. Still it will cause regression on memory hotplug.
>> 
>> Mainly we parse SRAT table to get the ranges of memory provided by
>> hot-added memory devices in initmem_init(), that's very late. During boot,
>> we don't know it. Chao ever posted patches to grab SRAT at decompressing
>> stage, the code is very complicated and not elegant, ACPI maintainer
>> NACKed that.

Thanks for Ingo's suggestion and Baoquan's explaination.

Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
https://lkml.org/lkml/2017/9/3/77
But the change is too huge so made this patchset to avoid this bug in a
small change, which will not make the code looks messy.

Thanks,
Chao Fan

>
>So there's apparently a mis-design here:
>
> - KASLR needs to be done very early on during bootup: - it's not realistic to 
>   expect KASLR to be done with a booted up kernel, because pointers to 
> various 
>   KASLR-ed objects are already widely spread out in memory.
>
> - But for some unfathomable reason the memory hotplug attribute of memory
>   regions is not part of the regular memory map but part of late-init ACPI 
> data
>   structures.
>
>The right solution would be _not_ to fudge the KASLR location, but to provide 
>the 
>memory hotplug information to early code, preferably via the primary memory 
>map. 
>KASLR can then make use of it and avoid those regions, just like it avoids 
>other 
>memory regions already.
>
>In addition to that hardware makers (including virtualized hardware) should 
>also 
>fix their systems to provide memory hotplug information to early code.
>
>Thanks,
>
>   Ingo
>
>

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Chao Fan]

On Mon, Mar 12, 2018 at 11:57:27AM +0100, Ingo Molnar wrote:
>
>* Baoquan He  wrote:
>
>> Hi Ingo,
>> 
>> On 03/12/18 at 10:35am, Ingo Molnar wrote:
>> > 
>> > * Chao Fan  wrote:
>> > 
>> > > Long time no reply, rebase the patchset, change the parameter name
>> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
>> > > 
>> > > ***Background:
>> > > People reported that kaslr may randomly chooses some positions
>> > > which are located in movable memory regions. This will break memory
>> > > hotplug feature.
>> > 
>> > [...]
>> > 
>> > > ***Solutions:
>> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
>> > > specify the memory regions where kernel can be allowed to randomize
>> > > safely.
>> > 
>> > Manual solutions like that are pretty suboptimal to users, aren't they?
>> > 
>> > In what way does memory hotplug feature 'break'? Does it crash or 
>> > misbehave? Or 
>> > simply does it not allow the movement of the affected memory region, while 
>> > still 
>> > allowing the rest to be moved?
>> 
>> AFAIT, if kernel is randomized into the movable memory region, the
>> affected memory region can not be hot added/removed since it has kernel
>> data. Surely, the system can still work, the unaffected part still can
>> be moved. Still it will cause regression on memory hotplug.
>> 
>> Mainly we parse SRAT table to get the ranges of memory provided by
>> hot-added memory devices in initmem_init(), that's very late. During boot,
>> we don't know it. Chao ever posted patches to grab SRAT at decompressing
>> stage, the code is very complicated and not elegant, ACPI maintainer
>> NACKed that.

Thanks for Ingo's suggestion and Baoquan's explaination.

Yes, I did ever try to dig SRAT table in boot period in early RFC PATCH:
https://lkml.org/lkml/2017/9/3/77
But the change is too huge so made this patchset to avoid this bug in a
small change, which will not make the code looks messy.

Thanks,
Chao Fan

>
>So there's apparently a mis-design here:
>
> - KASLR needs to be done very early on during bootup: - it's not realistic to 
>   expect KASLR to be done with a booted up kernel, because pointers to 
> various 
>   KASLR-ed objects are already widely spread out in memory.
>
> - But for some unfathomable reason the memory hotplug attribute of memory
>   regions is not part of the regular memory map but part of late-init ACPI 
> data
>   structures.
>
>The right solution would be _not_ to fudge the KASLR location, but to provide 
>the 
>memory hotplug information to early code, preferably via the primary memory 
>map. 
>KASLR can then make use of it and avoid those regions, just like it avoids 
>other 
>memory regions already.
>
>In addition to that hardware makers (including virtualized hardware) should 
>also 
>fix their systems to provide memory hotplug information to early code.
>
>Thanks,
>
>   Ingo
>
>

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Ingo Molnar]

* Baoquan He  wrote:

> Hi Ingo,
> 
> On 03/12/18 at 10:35am, Ingo Molnar wrote:
> > 
> > * Chao Fan  wrote:
> > 
> > > Long time no reply, rebase the patchset, change the parameter name
> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> > > 
> > > ***Background:
> > > People reported that kaslr may randomly chooses some positions
> > > which are located in movable memory regions. This will break memory
> > > hotplug feature.
> > 
> > [...]
> > 
> > > ***Solutions:
> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> > > specify the memory regions where kernel can be allowed to randomize
> > > safely.
> > 
> > Manual solutions like that are pretty suboptimal to users, aren't they?
> > 
> > In what way does memory hotplug feature 'break'? Does it crash or 
> > misbehave? Or 
> > simply does it not allow the movement of the affected memory region, while 
> > still 
> > allowing the rest to be moved?
> 
> AFAIT, if kernel is randomized into the movable memory region, the
> affected memory region can not be hot added/removed since it has kernel
> data. Surely, the system can still work, the unaffected part still can
> be moved. Still it will cause regression on memory hotplug.
> 
> Mainly we parse SRAT table to get the ranges of memory provided by
> hot-added memory devices in initmem_init(), that's very late. During boot,
> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> stage, the code is very complicated and not elegant, ACPI maintainer
> NACKed that.

So there's apparently a mis-design here:

 - KASLR needs to be done very early on during bootup: - it's not realistic to 
   expect KASLR to be done with a booted up kernel, because pointers to various 
   KASLR-ed objects are already widely spread out in memory.

 - But for some unfathomable reason the memory hotplug attribute of memory
   regions is not part of the regular memory map but part of late-init ACPI data
   structures.

The right solution would be _not_ to fudge the KASLR location, but to provide 
the 
memory hotplug information to early code, preferably via the primary memory 
map. 
KASLR can then make use of it and avoid those regions, just like it avoids 
other 
memory regions already.

In addition to that hardware makers (including virtualized hardware) should 
also 
fix their systems to provide memory hotplug information to early code.

Thanks,

Ingo

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Ingo Molnar]

* Baoquan He  wrote:

> Hi Ingo,
> 
> On 03/12/18 at 10:35am, Ingo Molnar wrote:
> > 
> > * Chao Fan  wrote:
> > 
> > > Long time no reply, rebase the patchset, change the parameter name
> > > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> > > 
> > > ***Background:
> > > People reported that kaslr may randomly chooses some positions
> > > which are located in movable memory regions. This will break memory
> > > hotplug feature.
> > 
> > [...]
> > 
> > > ***Solutions:
> > > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> > > specify the memory regions where kernel can be allowed to randomize
> > > safely.
> > 
> > Manual solutions like that are pretty suboptimal to users, aren't they?
> > 
> > In what way does memory hotplug feature 'break'? Does it crash or 
> > misbehave? Or 
> > simply does it not allow the movement of the affected memory region, while 
> > still 
> > allowing the rest to be moved?
> 
> AFAIT, if kernel is randomized into the movable memory region, the
> affected memory region can not be hot added/removed since it has kernel
> data. Surely, the system can still work, the unaffected part still can
> be moved. Still it will cause regression on memory hotplug.
> 
> Mainly we parse SRAT table to get the ranges of memory provided by
> hot-added memory devices in initmem_init(), that's very late. During boot,
> we don't know it. Chao ever posted patches to grab SRAT at decompressing
> stage, the code is very complicated and not elegant, ACPI maintainer
> NACKed that.

So there's apparently a mis-design here:

 - KASLR needs to be done very early on during bootup: - it's not realistic to 
   expect KASLR to be done with a booted up kernel, because pointers to various 
   KASLR-ed objects are already widely spread out in memory.

 - But for some unfathomable reason the memory hotplug attribute of memory
   regions is not part of the regular memory map but part of late-init ACPI data
   structures.

The right solution would be _not_ to fudge the KASLR location, but to provide 
the 
memory hotplug information to early code, preferably via the primary memory 
map. 
KASLR can then make use of it and avoid those regions, just like it avoids 
other 
memory regions already.

In addition to that hardware makers (including virtualized hardware) should 
also 
fix their systems to provide memory hotplug information to early code.

Thanks,

Ingo

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

Hi Ingo,

On 03/12/18 at 10:35am, Ingo Molnar wrote:
> 
> * Chao Fan  wrote:
> 
> > Long time no reply, rebase the patchset, change the parameter name
> > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> > 
> > ***Background:
> > People reported that kaslr may randomly chooses some positions
> > which are located in movable memory regions. This will break memory
> > hotplug feature.
> 
> [...]
> 
> > ***Solutions:
> > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> > specify the memory regions where kernel can be allowed to randomize
> > safely.
> 
> Manual solutions like that are pretty suboptimal to users, aren't they?
> 
> In what way does memory hotplug feature 'break'? Does it crash or misbehave? 
> Or 
> simply does it not allow the movement of the affected memory region, while 
> still 
> allowing the rest to be moved?

AFAIT, if kernel is randomized into the movable memory region, the
affected memory region can not be hot added/removed since it has kernel
data. Surely, the system can still work, the unaffected part still can
be moved. Still it will cause regression on memory hotplug.

Mainly we parse SRAT table to get the ranges of memory provided by
hot-added memory devices in initmem_init(), that's very late. During boot,
we don't know it. Chao ever posted patches to grab SRAT at decompressing
stage, the code is very complicated and not elegant, ACPI maintainer
NACKed that.

Thanks
Baoquan

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Baoquan He]

Hi Ingo,

On 03/12/18 at 10:35am, Ingo Molnar wrote:
> 
> * Chao Fan  wrote:
> 
> > Long time no reply, rebase the patchset, change the parameter name
> > from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> > 
> > ***Background:
> > People reported that kaslr may randomly chooses some positions
> > which are located in movable memory regions. This will break memory
> > hotplug feature.
> 
> [...]
> 
> > ***Solutions:
> > Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> > specify the memory regions where kernel can be allowed to randomize
> > safely.
> 
> Manual solutions like that are pretty suboptimal to users, aren't they?
> 
> In what way does memory hotplug feature 'break'? Does it crash or misbehave? 
> Or 
> simply does it not allow the movement of the affected memory region, while 
> still 
> allowing the rest to be moved?

AFAIT, if kernel is randomized into the movable memory region, the
affected memory region can not be hot added/removed since it has kernel
data. Surely, the system can still work, the unaffected part still can
be moved. Still it will cause regression on memory hotplug.

Mainly we parse SRAT table to get the ranges of memory provided by
hot-added memory devices in initmem_init(), that's very late. During boot,
we don't know it. Chao ever posted patches to grab SRAT at decompressing
stage, the code is very complicated and not elegant, ACPI maintainer
NACKed that.

Thanks
Baoquan

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Ingo Molnar]

* Chao Fan  wrote:

> Long time no reply, rebase the patchset, change the parameter name
> from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> 
> ***Background:
> People reported that kaslr may randomly chooses some positions
> which are located in movable memory regions. This will break memory
> hotplug feature.

[...]

> ***Solutions:
> Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> specify the memory regions where kernel can be allowed to randomize
> safely.

Manual solutions like that are pretty suboptimal to users, aren't they?

In what way does memory hotplug feature 'break'? Does it crash or misbehave? Or 
simply does it not allow the movement of the affected memory region, while 
still 
allowing the rest to be moved?

Thanks,

Ingo

Re: [PATCH v9 0/5] x86/KASLR: Add parameter kaslr_boot_mem=nn[KMG]@ss[KMG]

[Ingo Molnar]

* Chao Fan  wrote:

> Long time no reply, rebase the patchset, change the parameter name
> from 'kaslr_mem' to 'kaslr_boot_mem'. There's no more code change.
> 
> ***Background:
> People reported that kaslr may randomly chooses some positions
> which are located in movable memory regions. This will break memory
> hotplug feature.

[...]

> ***Solutions:
> Introduce a new kernel parameter 'kaslr_boot_mem=nn@ss' to let users to
> specify the memory regions where kernel can be allowed to randomize
> safely.

Manual solutions like that are pretty suboptimal to users, aren't they?

In what way does memory hotplug feature 'break'? Does it crash or misbehave? Or 
simply does it not allow the movement of the affected memory region, while 
still 
allowing the rest to be moved?

Thanks,

Ingo