Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/7 16:43, Scott Branden wrote:
> Hi Xishi,
>
> I followed you suggestions and found pfn_valid is always true. Answers to
> your questions inline.
>
> I could keep debugging this but hope Marcin sends out some code - I'm quite
> willing to test and help clean up the patchset.
>
> On 16-12-01 07:11 PM, Xishi Qiu wrote:
>> On 2016/12/2 10:38, Scott Branden wrote:
>>
>>> Hi Xishi,
>>>
>>> Thanks for the reply - please see comments below.
>>>
>>> On 16-12-01 05:49 PM, Xishi Qiu wrote:
On 2016/12/2 8:19, Scott Branden wrote:
> This patchset is sent for comment to add memory hotplug support for ARM64
> based platforms. It follows hotplug code added for other architectures
> in the linux kernel.
>
> I tried testing the memory hotplug feature following documentation from
> Documentation/memory-hotplug.txt. I don't think it is working as expected
> - see below:
>
> To add memory to the system I did the following:
> echo 0x4 > /sys/devices/system/memory/probe
>
> The memory is displayed as system ram:
> cat /proc/iomem:
> 7400-77ff : System RAM
> 7408-748d : Kernel code
> 7495-749d2fff : Kernel data
> 4-43fff : System RAM
>
> But does not seem to be added to the kernel memory.
> /proc/meminfo did not change.
>
> What else needs to be done so the memory is added to the kernel memory
> pool for normal allocation?
>
Hi Scott,
Do you mean it still don't support hod-add after apply this patchset?
>>>
>>> After applying the patch it appears to partially support hot-add. Please
>>> let me know if you think it is working as expected?
>>>
>>> The memory probe functions in that the memory is registered with the system
>>> and shows up in /proc/iomem. But, the memory is not available in
>>> /proc/meminfo. Do you think something else needs to be adjusted for ARM64
>>> to hotadd the memory
>>>
>>> I just found another clue:
>>> under /sys/devices/system/memory I only see one memory entry (before or
>>> after I try to hotadd additional memory).
>>>
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory0uevent
>>> block_size_bytesprobe
>>>
>>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
>>> to 28 and recompile I get the following:
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory7uevent
>>> block_size_bytesprobe
>>>
>>>
>>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
>>> to 27 and recompile I get the following:
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory14uevent
>>> block_size_bytesprobe
>>>
>>> If looks to me like something is not working properly in the ARM64
>>> implementation. I should expect to see multiple memoryX entries under
>>> /sys/devices/system/memory?
>>>
>>
>> Hi Scott,
>>
>> 1. Do you enable the following configs?
>> CONFIG_SPARSEMEM
>> MEMORY_HOTPLUG
>> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
> Yes, these configs are enabled
>>
>> 2. I find you missed create mapping in arch_add_memory(), and x86 has it.
> Could you please explain this further? The patch I submitted hass
> arch_add_memory identical to the ia64 implementation.
Hi Scott,
I think we should create page table first for the new hotadd memory.
e.g. create_mapping_late(start, __phys_to_virt(start), size, PAGE_KERNEL);
I don't know why ia64 don't have this step.
CC Tony
>>
>> 3. We will add memblock first, so pfn_valid() maybe always return true(in the
>> following function), and this will lead __add_section() failed. Please check
>> it.
> You are correct - pfn_valid always returns true. The function is in
> arch/arm64/mm/init.c and different than the one you indicated below:
>
> #ifdef CONFIG_HAVE_ARCH_PFN_VALID
> int pfn_valid(unsigned long pfn)
> {
> return memblock_is_map_memory(pfn << PAGE_SHIFT);
> }
> EXPORT_SYMBOL(pfn_valid);
> #endif
>
>>
>> int pfn_valid(unsigned long pfn)
>> {
>> return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
>> }
>>
>> add_memory
>> add_memory_resource
>> memblock_add_node
>> arch_add_memory
>> __add_pages
>> __add_section
>> pfn_valid
>>
>> Thanks,
>> Xishi Qiu
>>
>>>
>>>
Thanks,
Xishi Qiu
> Scott Branden (2):
> arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
> MEMORY_PROBE
> arm64: defconfig: enable MEMORY_HOTPLUG config options
>
> arch/arm64/Kconfig | 10 ++
> arch/arm64/configs/defconfig | 3 +++
> arch/arm64/mm/init.c | 42
> ++
> 3 files changed, 55 insertions(+)
>
>>>
>>> .
>>>
>>
>>
>>
>
> .
>
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/7 16:43, Scott Branden wrote:
> Hi Xishi,
>
> I followed you suggestions and found pfn_valid is always true. Answers to
> your questions inline.
>
> I could keep debugging this but hope Marcin sends out some code - I'm quite
> willing to test and help clean up the patchset.
>
> On 16-12-01 07:11 PM, Xishi Qiu wrote:
>> On 2016/12/2 10:38, Scott Branden wrote:
>>
>>> Hi Xishi,
>>>
>>> Thanks for the reply - please see comments below.
>>>
>>> On 16-12-01 05:49 PM, Xishi Qiu wrote:
On 2016/12/2 8:19, Scott Branden wrote:
> This patchset is sent for comment to add memory hotplug support for ARM64
> based platforms. It follows hotplug code added for other architectures
> in the linux kernel.
>
> I tried testing the memory hotplug feature following documentation from
> Documentation/memory-hotplug.txt. I don't think it is working as expected
> - see below:
>
> To add memory to the system I did the following:
> echo 0x4 > /sys/devices/system/memory/probe
>
> The memory is displayed as system ram:
> cat /proc/iomem:
> 7400-77ff : System RAM
> 7408-748d : Kernel code
> 7495-749d2fff : Kernel data
> 4-43fff : System RAM
>
> But does not seem to be added to the kernel memory.
> /proc/meminfo did not change.
>
> What else needs to be done so the memory is added to the kernel memory
> pool for normal allocation?
>
Hi Scott,
Do you mean it still don't support hod-add after apply this patchset?
>>>
>>> After applying the patch it appears to partially support hot-add. Please
>>> let me know if you think it is working as expected?
>>>
>>> The memory probe functions in that the memory is registered with the system
>>> and shows up in /proc/iomem. But, the memory is not available in
>>> /proc/meminfo. Do you think something else needs to be adjusted for ARM64
>>> to hotadd the memory
>>>
>>> I just found another clue:
>>> under /sys/devices/system/memory I only see one memory entry (before or
>>> after I try to hotadd additional memory).
>>>
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory0uevent
>>> block_size_bytesprobe
>>>
>>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
>>> to 28 and recompile I get the following:
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory7uevent
>>> block_size_bytesprobe
>>>
>>>
>>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
>>> to 27 and recompile I get the following:
>>> /sys/devices/system/memory # ls
>>> auto_online_blocks memory14uevent
>>> block_size_bytesprobe
>>>
>>> If looks to me like something is not working properly in the ARM64
>>> implementation. I should expect to see multiple memoryX entries under
>>> /sys/devices/system/memory?
>>>
>>
>> Hi Scott,
>>
>> 1. Do you enable the following configs?
>> CONFIG_SPARSEMEM
>> MEMORY_HOTPLUG
>> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
> Yes, these configs are enabled
>>
>> 2. I find you missed create mapping in arch_add_memory(), and x86 has it.
> Could you please explain this further? The patch I submitted hass
> arch_add_memory identical to the ia64 implementation.
Hi Scott,
I think we should create page table first for the new hotadd memory.
e.g. create_mapping_late(start, __phys_to_virt(start), size, PAGE_KERNEL);
I don't know why ia64 don't have this step.
CC Tony
>>
>> 3. We will add memblock first, so pfn_valid() maybe always return true(in the
>> following function), and this will lead __add_section() failed. Please check
>> it.
> You are correct - pfn_valid always returns true. The function is in
> arch/arm64/mm/init.c and different than the one you indicated below:
>
> #ifdef CONFIG_HAVE_ARCH_PFN_VALID
> int pfn_valid(unsigned long pfn)
> {
> return memblock_is_map_memory(pfn << PAGE_SHIFT);
> }
> EXPORT_SYMBOL(pfn_valid);
> #endif
>
>>
>> int pfn_valid(unsigned long pfn)
>> {
>> return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
>> }
>>
>> add_memory
>> add_memory_resource
>> memblock_add_node
>> arch_add_memory
>> __add_pages
>> __add_section
>> pfn_valid
>>
>> Thanks,
>> Xishi Qiu
>>
>>>
>>>
Thanks,
Xishi Qiu
> Scott Branden (2):
> arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
> MEMORY_PROBE
> arm64: defconfig: enable MEMORY_HOTPLUG config options
>
> arch/arm64/Kconfig | 10 ++
> arch/arm64/configs/defconfig | 3 +++
> arch/arm64/mm/init.c | 42
> ++
> 3 files changed, 55 insertions(+)
>
>>>
>>> .
>>>
>>
>>
>>
>
> .
>
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Xishi,
I followed you suggestions and found pfn_valid is always true. Answers
to your questions inline.
I could keep debugging this but hope Marcin sends out some code - I'm
quite willing to test and help clean up the patchset.
On 16-12-01 07:11 PM, Xishi Qiu wrote:
On 2016/12/2 10:38, Scott Branden wrote:
Hi Xishi,
Thanks for the reply - please see comments below.
On 16-12-01 05:49 PM, Xishi Qiu wrote:
On 2016/12/2 8:19, Scott Branden wrote:
This patchset is sent for comment to add memory hotplug support for ARM64
based platforms. It follows hotplug code added for other architectures
in the linux kernel.
I tried testing the memory hotplug feature following documentation from
Documentation/memory-hotplug.txt. I don't think it is working as expected
- see below:
To add memory to the system I did the following:
echo 0x4 > /sys/devices/system/memory/probe
The memory is displayed as system ram:
cat /proc/iomem:
7400-77ff : System RAM
7408-748d : Kernel code
7495-749d2fff : Kernel data
4-43fff : System RAM
But does not seem to be added to the kernel memory.
/proc/meminfo did not change.
What else needs to be done so the memory is added to the kernel memory
pool for normal allocation?
Hi Scott,
Do you mean it still don't support hod-add after apply this patchset?
After applying the patch it appears to partially support hot-add. Please let me
know if you think it is working as expected?
The memory probe functions in that the memory is registered with the system and
shows up in /proc/iomem. But, the memory is not available in /proc/meminfo.
Do you think something else needs to be adjusted for ARM64 to hotadd the memory
I just found another clue:
under /sys/devices/system/memory I only see one memory entry (before or after I
try to hotadd additional memory).
/sys/devices/system/memory # ls
auto_online_blocks memory0uevent
block_size_bytesprobe
In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to
28 and recompile I get the following:
/sys/devices/system/memory # ls
auto_online_blocks memory7uevent
block_size_bytesprobe
In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to
27 and recompile I get the following:
/sys/devices/system/memory # ls
auto_online_blocks memory14uevent
block_size_bytesprobe
If looks to me like something is not working properly in the ARM64
implementation. I should expect to see multiple memoryX entries under
/sys/devices/system/memory?
Hi Scott,
1. Do you enable the following configs?
CONFIG_SPARSEMEM
MEMORY_HOTPLUG
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
Yes, these configs are enabled
2. I find you missed create mapping in arch_add_memory(), and x86 has it.
Could you please explain this further? The patch I submitted hass
arch_add_memory identical to the ia64 implementation.
3. We will add memblock first, so pfn_valid() maybe always return true(in the
following function), and this will lead __add_section() failed. Please check
it.
You are correct - pfn_valid always returns true. The function is in
arch/arm64/mm/init.c and different than the one you indicated below:
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
int pfn_valid(unsigned long pfn)
{
return memblock_is_map_memory(pfn << PAGE_SHIFT);
}
EXPORT_SYMBOL(pfn_valid);
#endif
int pfn_valid(unsigned long pfn)
{
return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
}
add_memory
add_memory_resource
memblock_add_node
arch_add_memory
__add_pages
__add_section
pfn_valid
Thanks,
Xishi Qiu
Thanks,
Xishi Qiu
Scott Branden (2):
arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
MEMORY_PROBE
arm64: defconfig: enable MEMORY_HOTPLUG config options
arch/arm64/Kconfig | 10 ++
arch/arm64/configs/defconfig | 3 +++
arch/arm64/mm/init.c | 42 ++
3 files changed, 55 insertions(+)
.
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Xishi,
I followed you suggestions and found pfn_valid is always true. Answers
to your questions inline.
I could keep debugging this but hope Marcin sends out some code - I'm
quite willing to test and help clean up the patchset.
On 16-12-01 07:11 PM, Xishi Qiu wrote:
On 2016/12/2 10:38, Scott Branden wrote:
Hi Xishi,
Thanks for the reply - please see comments below.
On 16-12-01 05:49 PM, Xishi Qiu wrote:
On 2016/12/2 8:19, Scott Branden wrote:
This patchset is sent for comment to add memory hotplug support for ARM64
based platforms. It follows hotplug code added for other architectures
in the linux kernel.
I tried testing the memory hotplug feature following documentation from
Documentation/memory-hotplug.txt. I don't think it is working as expected
- see below:
To add memory to the system I did the following:
echo 0x4 > /sys/devices/system/memory/probe
The memory is displayed as system ram:
cat /proc/iomem:
7400-77ff : System RAM
7408-748d : Kernel code
7495-749d2fff : Kernel data
4-43fff : System RAM
But does not seem to be added to the kernel memory.
/proc/meminfo did not change.
What else needs to be done so the memory is added to the kernel memory
pool for normal allocation?
Hi Scott,
Do you mean it still don't support hod-add after apply this patchset?
After applying the patch it appears to partially support hot-add. Please let me
know if you think it is working as expected?
The memory probe functions in that the memory is registered with the system and
shows up in /proc/iomem. But, the memory is not available in /proc/meminfo.
Do you think something else needs to be adjusted for ARM64 to hotadd the memory
I just found another clue:
under /sys/devices/system/memory I only see one memory entry (before or after I
try to hotadd additional memory).
/sys/devices/system/memory # ls
auto_online_blocks memory0uevent
block_size_bytesprobe
In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to
28 and recompile I get the following:
/sys/devices/system/memory # ls
auto_online_blocks memory7uevent
block_size_bytesprobe
In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to
27 and recompile I get the following:
/sys/devices/system/memory # ls
auto_online_blocks memory14uevent
block_size_bytesprobe
If looks to me like something is not working properly in the ARM64
implementation. I should expect to see multiple memoryX entries under
/sys/devices/system/memory?
Hi Scott,
1. Do you enable the following configs?
CONFIG_SPARSEMEM
MEMORY_HOTPLUG
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
Yes, these configs are enabled
2. I find you missed create mapping in arch_add_memory(), and x86 has it.
Could you please explain this further? The patch I submitted hass
arch_add_memory identical to the ia64 implementation.
3. We will add memblock first, so pfn_valid() maybe always return true(in the
following function), and this will lead __add_section() failed. Please check
it.
You are correct - pfn_valid always returns true. The function is in
arch/arm64/mm/init.c and different than the one you indicated below:
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
int pfn_valid(unsigned long pfn)
{
return memblock_is_map_memory(pfn << PAGE_SHIFT);
}
EXPORT_SYMBOL(pfn_valid);
#endif
int pfn_valid(unsigned long pfn)
{
return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
}
add_memory
add_memory_resource
memblock_add_node
arch_add_memory
__add_pages
__add_section
pfn_valid
Thanks,
Xishi Qiu
Thanks,
Xishi Qiu
Scott Branden (2):
arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
MEMORY_PROBE
arm64: defconfig: enable MEMORY_HOTPLUG config options
arch/arm64/Kconfig | 10 ++
arch/arm64/configs/defconfig | 3 +++
arch/arm64/mm/init.c | 42 ++
3 files changed, 55 insertions(+)
.
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Maciej, On 16-12-02 02:55 AM, Maciej Bielski wrote: On 02/12/2016 11:49, Will Deacon wrote: On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: Recently we have announced our effort on that: https://lkml.org/lkml/2016/11/17/49 For now we have a working solution for hotplug and we are performing code cleanup to push the patches soon. Are these intended to replace or extend Scott's patches? If the former, please work with Scott's stuff as a base rather than posting a competing series. In the piece of code provided by Scott I have seen similar steps to what is done by us but our work went further since we have the mapping created and everything is working via the sysfs interface. I am now having closer look and comparing them. I would love to see the missing section mapping step and any other additions to test out. Please send additions as soon as you have a chance. Regards, Scott Will
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Maciej, On 16-12-02 02:55 AM, Maciej Bielski wrote: On 02/12/2016 11:49, Will Deacon wrote: On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: Recently we have announced our effort on that: https://lkml.org/lkml/2016/11/17/49 For now we have a working solution for hotplug and we are performing code cleanup to push the patches soon. Are these intended to replace or extend Scott's patches? If the former, please work with Scott's stuff as a base rather than posting a competing series. In the piece of code provided by Scott I have seen similar steps to what is done by us but our work went further since we have the mapping created and everything is working via the sysfs interface. I am now having closer look and comparing them. I would love to see the missing section mapping step and any other additions to test out. Please send additions as soon as you have a chance. Regards, Scott Will
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 02/12/2016 11:49, Will Deacon wrote: > On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: >> Recently we have announced our effort on that: >> https://lkml.org/lkml/2016/11/17/49 >> >> For now we have a working solution for hotplug and we are performing >> code cleanup to push the patches soon. > Are these intended to replace or extend Scott's patches? If the former, > please work with Scott's stuff as a base rather than posting a competing > series. In the piece of code provided by Scott I have seen similar steps to what is done by us but our work went further since we have the mapping created and everything is working via the sysfs interface. I am now having closer look and comparing them. > > Will -- Maciej Bielski
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 02/12/2016 11:49, Will Deacon wrote: > On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: >> Recently we have announced our effort on that: >> https://lkml.org/lkml/2016/11/17/49 >> >> For now we have a working solution for hotplug and we are performing >> code cleanup to push the patches soon. > Are these intended to replace or extend Scott's patches? If the former, > please work with Scott's stuff as a base rather than posting a competing > series. In the piece of code provided by Scott I have seen similar steps to what is done by us but our work went further since we have the mapping created and everything is working via the sysfs interface. I am now having closer look and comparing them. > > Will -- Maciej Bielski
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: > Recently we have announced our effort on that: > https://lkml.org/lkml/2016/11/17/49 > > For now we have a working solution for hotplug and we are performing > code cleanup to push the patches soon. Are these intended to replace or extend Scott's patches? If the former, please work with Scott's stuff as a base rather than posting a competing series. Will
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On Fri, Dec 02, 2016 at 10:13:43AM +0100, Maciej Bielski wrote: > Recently we have announced our effort on that: > https://lkml.org/lkml/2016/11/17/49 > > For now we have a working solution for hotplug and we are performing > code cleanup to push the patches soon. Are these intended to replace or extend Scott's patches? If the former, please work with Scott's stuff as a base rather than posting a competing series. Will
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hello, Recently we have announced our effort on that: https://lkml.org/lkml/2016/11/17/49 For now we have a working solution for hotplug and we are performing code cleanup to push the patches soon. BR, On 02/12/2016 01:19, Scott Branden wrote: > This patchset is sent for comment to add memory hotplug support for ARM64 > based platforms. It follows hotplug code added for other architectures > in the linux kernel. > > I tried testing the memory hotplug feature following documentation from > Documentation/memory-hotplug.txt. I don't think it is working as expected > - see below: > > To add memory to the system I did the following: > echo 0x4 > /sys/devices/system/memory/probe > > The memory is displayed as system ram: > cat /proc/iomem: > 7400-77ff : System RAM > 7408-748d : Kernel code > 7495-749d2fff : Kernel data > 4-43fff : System RAM > > But does not seem to be added to the kernel memory. > /proc/meminfo did not change. > > What else needs to be done so the memory is added to the kernel memory > pool for normal allocation? > > Scott Branden (2): > arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, > MEMORY_PROBE > arm64: defconfig: enable MEMORY_HOTPLUG config options > > arch/arm64/Kconfig | 10 ++ > arch/arm64/configs/defconfig | 3 +++ > arch/arm64/mm/init.c | 42 ++ > 3 files changed, 55 insertions(+) > -- Maciej Bielski
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hello, Recently we have announced our effort on that: https://lkml.org/lkml/2016/11/17/49 For now we have a working solution for hotplug and we are performing code cleanup to push the patches soon. BR, On 02/12/2016 01:19, Scott Branden wrote: > This patchset is sent for comment to add memory hotplug support for ARM64 > based platforms. It follows hotplug code added for other architectures > in the linux kernel. > > I tried testing the memory hotplug feature following documentation from > Documentation/memory-hotplug.txt. I don't think it is working as expected > - see below: > > To add memory to the system I did the following: > echo 0x4 > /sys/devices/system/memory/probe > > The memory is displayed as system ram: > cat /proc/iomem: > 7400-77ff : System RAM > 7408-748d : Kernel code > 7495-749d2fff : Kernel data > 4-43fff : System RAM > > But does not seem to be added to the kernel memory. > /proc/meminfo did not change. > > What else needs to be done so the memory is added to the kernel memory > pool for normal allocation? > > Scott Branden (2): > arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, > MEMORY_PROBE > arm64: defconfig: enable MEMORY_HOTPLUG config options > > arch/arm64/Kconfig | 10 ++ > arch/arm64/configs/defconfig | 3 +++ > arch/arm64/mm/init.c | 42 ++ > 3 files changed, 55 insertions(+) > -- Maciej Bielski
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/2 10:38, Scott Branden wrote:
> Hi Xishi,
>
> Thanks for the reply - please see comments below.
>
> On 16-12-01 05:49 PM, Xishi Qiu wrote:
>> On 2016/12/2 8:19, Scott Branden wrote:
>>
>>> This patchset is sent for comment to add memory hotplug support for ARM64
>>> based platforms. It follows hotplug code added for other architectures
>>> in the linux kernel.
>>>
>>> I tried testing the memory hotplug feature following documentation from
>>> Documentation/memory-hotplug.txt. I don't think it is working as expected
>>> - see below:
>>>
>>> To add memory to the system I did the following:
>>> echo 0x4 > /sys/devices/system/memory/probe
>>>
>>> The memory is displayed as system ram:
>>> cat /proc/iomem:
>>> 7400-77ff : System RAM
>>> 7408-748d : Kernel code
>>> 7495-749d2fff : Kernel data
>>> 4-43fff : System RAM
>>>
>>> But does not seem to be added to the kernel memory.
>>> /proc/meminfo did not change.
>>>
>>> What else needs to be done so the memory is added to the kernel memory
>>> pool for normal allocation?
>>>
>>
>> Hi Scott,
>>
>> Do you mean it still don't support hod-add after apply this patchset?
>
> After applying the patch it appears to partially support hot-add. Please let
> me know if you think it is working as expected?
>
> The memory probe functions in that the memory is registered with the system
> and shows up in /proc/iomem. But, the memory is not available in
> /proc/meminfo. Do you think something else needs to be adjusted for ARM64 to
> hotadd the memory
>
> I just found another clue:
> under /sys/devices/system/memory I only see one memory entry (before or after
> I try to hotadd additional memory).
>
> /sys/devices/system/memory # ls
> auto_online_blocks memory0uevent
> block_size_bytesprobe
>
> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
> to 28 and recompile I get the following:
> /sys/devices/system/memory # ls
> auto_online_blocks memory7uevent
> block_size_bytesprobe
>
>
> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
> to 27 and recompile I get the following:
> /sys/devices/system/memory # ls
> auto_online_blocks memory14uevent
> block_size_bytesprobe
>
> If looks to me like something is not working properly in the ARM64
> implementation. I should expect to see multiple memoryX entries under
> /sys/devices/system/memory?
>
Hi Scott,
1. Do you enable the following configs?
CONFIG_SPARSEMEM
MEMORY_HOTPLUG
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
2. I find you missed create mapping in arch_add_memory(), and x86 has it.
3. We will add memblock first, so pfn_valid() maybe always return true(in the
following function), and this will lead __add_section() failed. Please check
it.
int pfn_valid(unsigned long pfn)
{
return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
}
add_memory
add_memory_resource
memblock_add_node
arch_add_memory
__add_pages
__add_section
pfn_valid
Thanks,
Xishi Qiu
>
>
>>
>> Thanks,
>> Xishi Qiu
>>
>>> Scott Branden (2):
>>> arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
>>> MEMORY_PROBE
>>> arm64: defconfig: enable MEMORY_HOTPLUG config options
>>>
>>> arch/arm64/Kconfig | 10 ++
>>> arch/arm64/configs/defconfig | 3 +++
>>> arch/arm64/mm/init.c | 42
>>> ++
>>> 3 files changed, 55 insertions(+)
>>>
>>
>>
>>
>
> .
>
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/2 10:38, Scott Branden wrote:
> Hi Xishi,
>
> Thanks for the reply - please see comments below.
>
> On 16-12-01 05:49 PM, Xishi Qiu wrote:
>> On 2016/12/2 8:19, Scott Branden wrote:
>>
>>> This patchset is sent for comment to add memory hotplug support for ARM64
>>> based platforms. It follows hotplug code added for other architectures
>>> in the linux kernel.
>>>
>>> I tried testing the memory hotplug feature following documentation from
>>> Documentation/memory-hotplug.txt. I don't think it is working as expected
>>> - see below:
>>>
>>> To add memory to the system I did the following:
>>> echo 0x4 > /sys/devices/system/memory/probe
>>>
>>> The memory is displayed as system ram:
>>> cat /proc/iomem:
>>> 7400-77ff : System RAM
>>> 7408-748d : Kernel code
>>> 7495-749d2fff : Kernel data
>>> 4-43fff : System RAM
>>>
>>> But does not seem to be added to the kernel memory.
>>> /proc/meminfo did not change.
>>>
>>> What else needs to be done so the memory is added to the kernel memory
>>> pool for normal allocation?
>>>
>>
>> Hi Scott,
>>
>> Do you mean it still don't support hod-add after apply this patchset?
>
> After applying the patch it appears to partially support hot-add. Please let
> me know if you think it is working as expected?
>
> The memory probe functions in that the memory is registered with the system
> and shows up in /proc/iomem. But, the memory is not available in
> /proc/meminfo. Do you think something else needs to be adjusted for ARM64 to
> hotadd the memory
>
> I just found another clue:
> under /sys/devices/system/memory I only see one memory entry (before or after
> I try to hotadd additional memory).
>
> /sys/devices/system/memory # ls
> auto_online_blocks memory0uevent
> block_size_bytesprobe
>
> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
> to 28 and recompile I get the following:
> /sys/devices/system/memory # ls
> auto_online_blocks memory7uevent
> block_size_bytesprobe
>
>
> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30
> to 27 and recompile I get the following:
> /sys/devices/system/memory # ls
> auto_online_blocks memory14uevent
> block_size_bytesprobe
>
> If looks to me like something is not working properly in the ARM64
> implementation. I should expect to see multiple memoryX entries under
> /sys/devices/system/memory?
>
Hi Scott,
1. Do you enable the following configs?
CONFIG_SPARSEMEM
MEMORY_HOTPLUG
CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
2. I find you missed create mapping in arch_add_memory(), and x86 has it.
3. We will add memblock first, so pfn_valid() maybe always return true(in the
following function), and this will lead __add_section() failed. Please check
it.
int pfn_valid(unsigned long pfn)
{
return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
}
add_memory
add_memory_resource
memblock_add_node
arch_add_memory
__add_pages
__add_section
pfn_valid
Thanks,
Xishi Qiu
>
>
>>
>> Thanks,
>> Xishi Qiu
>>
>>> Scott Branden (2):
>>> arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
>>> MEMORY_PROBE
>>> arm64: defconfig: enable MEMORY_HOTPLUG config options
>>>
>>> arch/arm64/Kconfig | 10 ++
>>> arch/arm64/configs/defconfig | 3 +++
>>> arch/arm64/mm/init.c | 42
>>> ++
>>> 3 files changed, 55 insertions(+)
>>>
>>
>>
>>
>
> .
>
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Xishi, Thanks for the reply - please see comments below. On 16-12-01 05:49 PM, Xishi Qiu wrote: On 2016/12/2 8:19, Scott Branden wrote: This patchset is sent for comment to add memory hotplug support for ARM64 based platforms. It follows hotplug code added for other architectures in the linux kernel. I tried testing the memory hotplug feature following documentation from Documentation/memory-hotplug.txt. I don't think it is working as expected - see below: To add memory to the system I did the following: echo 0x4 > /sys/devices/system/memory/probe The memory is displayed as system ram: cat /proc/iomem: 7400-77ff : System RAM 7408-748d : Kernel code 7495-749d2fff : Kernel data 4-43fff : System RAM But does not seem to be added to the kernel memory. /proc/meminfo did not change. What else needs to be done so the memory is added to the kernel memory pool for normal allocation? Hi Scott, Do you mean it still don't support hod-add after apply this patchset? After applying the patch it appears to partially support hot-add. Please let me know if you think it is working as expected? The memory probe functions in that the memory is registered with the system and shows up in /proc/iomem. But, the memory is not available in /proc/meminfo. Do you think something else needs to be adjusted for ARM64 to hotadd the memory I just found another clue: under /sys/devices/system/memory I only see one memory entry (before or after I try to hotadd additional memory). /sys/devices/system/memory # ls auto_online_blocks memory0uevent block_size_bytesprobe In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 28 and recompile I get the following: /sys/devices/system/memory # ls auto_online_blocks memory7uevent block_size_bytesprobe In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 27 and recompile I get the following: /sys/devices/system/memory # ls auto_online_blocks memory14uevent block_size_bytesprobe If looks to me like something is not working properly in the ARM64 implementation. I should expect to see multiple memoryX entries under /sys/devices/system/memory? Thanks, Xishi Qiu Scott Branden (2): arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, MEMORY_PROBE arm64: defconfig: enable MEMORY_HOTPLUG config options arch/arm64/Kconfig | 10 ++ arch/arm64/configs/defconfig | 3 +++ arch/arm64/mm/init.c | 42 ++ 3 files changed, 55 insertions(+)
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
Hi Xishi, Thanks for the reply - please see comments below. On 16-12-01 05:49 PM, Xishi Qiu wrote: On 2016/12/2 8:19, Scott Branden wrote: This patchset is sent for comment to add memory hotplug support for ARM64 based platforms. It follows hotplug code added for other architectures in the linux kernel. I tried testing the memory hotplug feature following documentation from Documentation/memory-hotplug.txt. I don't think it is working as expected - see below: To add memory to the system I did the following: echo 0x4 > /sys/devices/system/memory/probe The memory is displayed as system ram: cat /proc/iomem: 7400-77ff : System RAM 7408-748d : Kernel code 7495-749d2fff : Kernel data 4-43fff : System RAM But does not seem to be added to the kernel memory. /proc/meminfo did not change. What else needs to be done so the memory is added to the kernel memory pool for normal allocation? Hi Scott, Do you mean it still don't support hod-add after apply this patchset? After applying the patch it appears to partially support hot-add. Please let me know if you think it is working as expected? The memory probe functions in that the memory is registered with the system and shows up in /proc/iomem. But, the memory is not available in /proc/meminfo. Do you think something else needs to be adjusted for ARM64 to hotadd the memory I just found another clue: under /sys/devices/system/memory I only see one memory entry (before or after I try to hotadd additional memory). /sys/devices/system/memory # ls auto_online_blocks memory0uevent block_size_bytesprobe In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 28 and recompile I get the following: /sys/devices/system/memory # ls auto_online_blocks memory7uevent block_size_bytesprobe In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 27 and recompile I get the following: /sys/devices/system/memory # ls auto_online_blocks memory14uevent block_size_bytesprobe If looks to me like something is not working properly in the ARM64 implementation. I should expect to see multiple memoryX entries under /sys/devices/system/memory? Thanks, Xishi Qiu Scott Branden (2): arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, MEMORY_PROBE arm64: defconfig: enable MEMORY_HOTPLUG config options arch/arm64/Kconfig | 10 ++ arch/arm64/configs/defconfig | 3 +++ arch/arm64/mm/init.c | 42 ++ 3 files changed, 55 insertions(+)
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/2 8:19, Scott Branden wrote: > This patchset is sent for comment to add memory hotplug support for ARM64 > based platforms. It follows hotplug code added for other architectures > in the linux kernel. > > I tried testing the memory hotplug feature following documentation from > Documentation/memory-hotplug.txt. I don't think it is working as expected > - see below: > > To add memory to the system I did the following: > echo 0x4 > /sys/devices/system/memory/probe > > The memory is displayed as system ram: > cat /proc/iomem: > 7400-77ff : System RAM > 7408-748d : Kernel code > 7495-749d2fff : Kernel data > 4-43fff : System RAM > > But does not seem to be added to the kernel memory. > /proc/meminfo did not change. > > What else needs to be done so the memory is added to the kernel memory > pool for normal allocation? > Hi Scott, Do you mean it still don't support hod-add after apply this patchset? Thanks, Xishi Qiu > Scott Branden (2): > arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, > MEMORY_PROBE > arm64: defconfig: enable MEMORY_HOTPLUG config options > > arch/arm64/Kconfig | 10 ++ > arch/arm64/configs/defconfig | 3 +++ > arch/arm64/mm/init.c | 42 ++ > 3 files changed, 55 insertions(+) >
Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
On 2016/12/2 8:19, Scott Branden wrote: > This patchset is sent for comment to add memory hotplug support for ARM64 > based platforms. It follows hotplug code added for other architectures > in the linux kernel. > > I tried testing the memory hotplug feature following documentation from > Documentation/memory-hotplug.txt. I don't think it is working as expected > - see below: > > To add memory to the system I did the following: > echo 0x4 > /sys/devices/system/memory/probe > > The memory is displayed as system ram: > cat /proc/iomem: > 7400-77ff : System RAM > 7408-748d : Kernel code > 7495-749d2fff : Kernel data > 4-43fff : System RAM > > But does not seem to be added to the kernel memory. > /proc/meminfo did not change. > > What else needs to be done so the memory is added to the kernel memory > pool for normal allocation? > Hi Scott, Do you mean it still don't support hod-add after apply this patchset? Thanks, Xishi Qiu > Scott Branden (2): > arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, > MEMORY_PROBE > arm64: defconfig: enable MEMORY_HOTPLUG config options > > arch/arm64/Kconfig | 10 ++ > arch/arm64/configs/defconfig | 3 +++ > arch/arm64/mm/init.c | 42 ++ > 3 files changed, 55 insertions(+) >
[RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
This patchset is sent for comment to add memory hotplug support for ARM64 based platforms. It follows hotplug code added for other architectures in the linux kernel. I tried testing the memory hotplug feature following documentation from Documentation/memory-hotplug.txt. I don't think it is working as expected - see below: To add memory to the system I did the following: echo 0x4 > /sys/devices/system/memory/probe The memory is displayed as system ram: cat /proc/iomem: 7400-77ff : System RAM 7408-748d : Kernel code 7495-749d2fff : Kernel data 4-43fff : System RAM But does not seem to be added to the kernel memory. /proc/meminfo did not change. What else needs to be done so the memory is added to the kernel memory pool for normal allocation? Scott Branden (2): arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, MEMORY_PROBE arm64: defconfig: enable MEMORY_HOTPLUG config options arch/arm64/Kconfig | 10 ++ arch/arm64/configs/defconfig | 3 +++ arch/arm64/mm/init.c | 42 ++ 3 files changed, 55 insertions(+) -- 2.5.0
[RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support
This patchset is sent for comment to add memory hotplug support for ARM64 based platforms. It follows hotplug code added for other architectures in the linux kernel. I tried testing the memory hotplug feature following documentation from Documentation/memory-hotplug.txt. I don't think it is working as expected - see below: To add memory to the system I did the following: echo 0x4 > /sys/devices/system/memory/probe The memory is displayed as system ram: cat /proc/iomem: 7400-77ff : System RAM 7408-748d : Kernel code 7495-749d2fff : Kernel data 4-43fff : System RAM But does not seem to be added to the kernel memory. /proc/meminfo did not change. What else needs to be done so the memory is added to the kernel memory pool for normal allocation? Scott Branden (2): arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE, MEMORY_PROBE arm64: defconfig: enable MEMORY_HOTPLUG config options arch/arm64/Kconfig | 10 ++ arch/arm64/configs/defconfig | 3 +++ arch/arm64/mm/init.c | 42 ++ 3 files changed, 55 insertions(+) -- 2.5.0
