Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Bhupesh, On 2020/10/7 15:07, Bhupesh Sharma wrote: > Hi Catalin, > > On Tue, Oct 6, 2020 at 11:30 PM Catalin Marinas > wrote: >> On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: >>> I think my earlier email with the test results on this series bounced >>> off the mailing list server (for some weird reason), but I still see >>> several issues with this patchset. I will add specific issues in the >>> review comments for each patch again, but overall, with a crashkernel >>> size of say 786M, I see the following issue: >>> >>> # cat /proc/cmdline >>> BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> >>> rd.lvm.lv=<..snip..> crashkernel=786M >>> >>> I see two regions of size 786M and 256M reserved in low and high >>> regions respectively, So we reserve a total of 1042M of memory, which >>> is an incorrect behaviour: >>> >>> # dmesg | grep -i crash >>> [0.00] Reserving 256MB of low memory at 2816MB for crashkernel >>> (System low RAM: 768MB) >>> [0.00] Reserving 786MB of memory at 654158MB for crashkernel >>> (System RAM: 130816MB) >>> [0.00] Kernel command line: >>> BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ >>> root=/dev/mapper/rhel_ampere--hr330a--03-root ro >>> rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap >>> crashkernel=786M cma=1024M >>> >>> # cat /proc/iomem | grep -i crash >>> b000-bfff : Crash kernel (low) >>> bfcbe0-bffcff : Crash kernel >> As Chen said, that's the intended behaviour and how x86 works. The >> requested 768M goes in the high range if there's not enough low memory >> and an additional buffer for swiotlb is allocated, hence the low 256M. > I understand, but why 256M (as low) for arm64? x86_64 setups usually > have more system memory available as compared to several commercially > available arm64 setups. So is the intent, just to keep the behavior > similar between arm64 and x86_64? > > Should we have a CONFIG option / bootarg to help one select the max > 'low_size'? Currently the ' low_size' value is calculated as: > > /* > * two parts from kernel/dma/swiotlb.c: > * -swiotlb size: user-specified with swiotlb= or default. > * > * -swiotlb overflow buffer: now hardcoded to 32k. We round it > * to 8M for other buffers that may need to stay low too. Also > * make sure we allocate enough extra low memory so that we > * don't run out of DMA buffers for 32-bit devices. > */ > low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20); > > Since many arm64 boards ship with swiotlb=0 (turned off) via kernel > bootargs, the low_size, still ends up being 256M in such cases, > whereas this 256M can be used for some other purposes - so should we > be limiting this to 64M and failing the crash kernel allocation > request (gracefully) otherwise? > >> We could (as an additional patch), subtract the 256M from the high >> allocation so that you'd get a low 256M and a high 512M, not sure it's >> worth it. Note that with a "crashkernel=768M,high" option, you still get >> the additional low 256M, otherwise the crashkernel won't be able to >> boot as there's no memory in ZONE_DMA. In the explicit ",high" request >> case, I'm not sure subtracted the 256M is more intuitive. >> In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms >> to cover the entire 32-bit address space (i.e. identical to the current >> ZONE_DMA32). >> >>> IMO, we should test this feature more before including this in 5.11 >> Definitely. That's one of the reasons we haven't queued it yet. So any >> help with testing here is appreciated. > Sure, I am running more checks on this series. I will be soon back > with more updates. Sorry to bother you. I am looking forward to your review comments. Thanks, Chen Zhou > > Regards, > Bhupesh > > . >
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On Wed, Oct 07, 2020 at 12:37:49PM +0530, Bhupesh Sharma wrote: > On Tue, Oct 6, 2020 at 11:30 PM Catalin Marinas > wrote: > > On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: > > > I think my earlier email with the test results on this series bounced > > > off the mailing list server (for some weird reason), but I still see > > > several issues with this patchset. I will add specific issues in the > > > review comments for each patch again, but overall, with a crashkernel > > > size of say 786M, I see the following issue: > > > > > > # cat /proc/cmdline > > > BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> > > > rd.lvm.lv=<..snip..> crashkernel=786M > > > > > > I see two regions of size 786M and 256M reserved in low and high > > > regions respectively, So we reserve a total of 1042M of memory, which > > > is an incorrect behaviour: > > > > > > # dmesg | grep -i crash > > > [0.00] Reserving 256MB of low memory at 2816MB for crashkernel > > > (System low RAM: 768MB) > > > [0.00] Reserving 786MB of memory at 654158MB for crashkernel > > > (System RAM: 130816MB) > > > [0.00] Kernel command line: > > > BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ > > > root=/dev/mapper/rhel_ampere--hr330a--03-root ro > > > rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap > > > crashkernel=786M cma=1024M > > > > > > # cat /proc/iomem | grep -i crash > > > b000-bfff : Crash kernel (low) > > > bfcbe0-bffcff : Crash kernel > > > > As Chen said, that's the intended behaviour and how x86 works. The > > requested 768M goes in the high range if there's not enough low memory > > and an additional buffer for swiotlb is allocated, hence the low 256M. > > I understand, but why 256M (as low) for arm64? x86_64 setups usually > have more system memory available as compared to several commercially > available arm64 setups. So is the intent, just to keep the behavior > similar between arm64 and x86_64? Similar in the sense of the fallback to high memory and some low memory allocation but the amounts can vary per architecture. > Should we have a CONFIG option / bootarg to help one select the max > 'low_size'? Currently the ' low_size' value is calculated as: > > /* > * two parts from kernel/dma/swiotlb.c: > * -swiotlb size: user-specified with swiotlb= or default. > * > * -swiotlb overflow buffer: now hardcoded to 32k. We round it > * to 8M for other buffers that may need to stay low too. Also > * make sure we allocate enough extra low memory so that we > * don't run out of DMA buffers for 32-bit devices. > */ > low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20); > > Since many arm64 boards ship with swiotlb=0 (turned off) via kernel > bootargs, the low_size, still ends up being 256M in such cases, > whereas this 256M can be used for some other purposes - so should we > be limiting this to 64M and failing the crash kernel allocation > request (gracefully) otherwise? I think it makes sense to set a low_size = 0 if swiotlb_size_or_default() is 0. The assumption would be that if the main kernel doesn't need an swiotlb, the crashdump one wouldn't need it either. But this probably needs the ZONE_DMA for non-RPi4 platforms addressed as well (expanded to the whole ZONE_DMA32). -- Catalin
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Catalin, On Tue, Oct 6, 2020 at 11:30 PM Catalin Marinas wrote: > > On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: > > I think my earlier email with the test results on this series bounced > > off the mailing list server (for some weird reason), but I still see > > several issues with this patchset. I will add specific issues in the > > review comments for each patch again, but overall, with a crashkernel > > size of say 786M, I see the following issue: > > > > # cat /proc/cmdline > > BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> > > rd.lvm.lv=<..snip..> crashkernel=786M > > > > I see two regions of size 786M and 256M reserved in low and high > > regions respectively, So we reserve a total of 1042M of memory, which > > is an incorrect behaviour: > > > > # dmesg | grep -i crash > > [0.00] Reserving 256MB of low memory at 2816MB for crashkernel > > (System low RAM: 768MB) > > [0.00] Reserving 786MB of memory at 654158MB for crashkernel > > (System RAM: 130816MB) > > [0.00] Kernel command line: > > BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ > > root=/dev/mapper/rhel_ampere--hr330a--03-root ro > > rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap > > crashkernel=786M cma=1024M > > > > # cat /proc/iomem | grep -i crash > > b000-bfff : Crash kernel (low) > > bfcbe0-bffcff : Crash kernel > > As Chen said, that's the intended behaviour and how x86 works. The > requested 768M goes in the high range if there's not enough low memory > and an additional buffer for swiotlb is allocated, hence the low 256M. I understand, but why 256M (as low) for arm64? x86_64 setups usually have more system memory available as compared to several commercially available arm64 setups. So is the intent, just to keep the behavior similar between arm64 and x86_64? Should we have a CONFIG option / bootarg to help one select the max 'low_size'? Currently the ' low_size' value is calculated as: /* * two parts from kernel/dma/swiotlb.c: * -swiotlb size: user-specified with swiotlb= or default. * * -swiotlb overflow buffer: now hardcoded to 32k. We round it * to 8M for other buffers that may need to stay low too. Also * make sure we allocate enough extra low memory so that we * don't run out of DMA buffers for 32-bit devices. */ low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20); Since many arm64 boards ship with swiotlb=0 (turned off) via kernel bootargs, the low_size, still ends up being 256M in such cases, whereas this 256M can be used for some other purposes - so should we be limiting this to 64M and failing the crash kernel allocation request (gracefully) otherwise? > We could (as an additional patch), subtract the 256M from the high > allocation so that you'd get a low 256M and a high 512M, not sure it's > worth it. Note that with a "crashkernel=768M,high" option, you still get > the additional low 256M, otherwise the crashkernel won't be able to > boot as there's no memory in ZONE_DMA. In the explicit ",high" request > case, I'm not sure subtracted the 256M is more intuitive. > In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms > to cover the entire 32-bit address space (i.e. identical to the current > ZONE_DMA32). > > > IMO, we should test this feature more before including this in 5.11 > > Definitely. That's one of the reasons we haven't queued it yet. So any > help with testing here is appreciated. Sure, I am running more checks on this series. I will be soon back with more updates. Regards, Bhupesh
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On Mon, Oct 05, 2020 at 11:12:10PM +0530, Bhupesh Sharma wrote: > I think my earlier email with the test results on this series bounced > off the mailing list server (for some weird reason), but I still see > several issues with this patchset. I will add specific issues in the > review comments for each patch again, but overall, with a crashkernel > size of say 786M, I see the following issue: > > # cat /proc/cmdline > BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> > crashkernel=786M > > I see two regions of size 786M and 256M reserved in low and high > regions respectively, So we reserve a total of 1042M of memory, which > is an incorrect behaviour: > > # dmesg | grep -i crash > [0.00] Reserving 256MB of low memory at 2816MB for crashkernel > (System low RAM: 768MB) > [0.00] Reserving 786MB of memory at 654158MB for crashkernel (System > RAM: 130816MB) > [0.00] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ > root=/dev/mapper/rhel_ampere--hr330a--03-root ro > rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap > crashkernel=786M cma=1024M > > # cat /proc/iomem | grep -i crash > b000-bfff : Crash kernel (low) > bfcbe0-bffcff : Crash kernel As Chen said, that's the intended behaviour and how x86 works. The requested 768M goes in the high range if there's not enough low memory and an additional buffer for swiotlb is allocated, hence the low 256M. We could (as an additional patch), subtract the 256M from the high allocation so that you'd get a low 256M and a high 512M, not sure it's worth it. Note that with a "crashkernel=768M,high" option, you still get the additional low 256M, otherwise the crashkernel won't be able to boot as there's no memory in ZONE_DMA. In the explicit ",high" request case, I'm not sure subtracted the 256M is more intuitive. In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms to cover the entire 32-bit address space (i.e. identical to the current ZONE_DMA32). > IMO, we should test this feature more before including this in 5.11 Definitely. That's one of the reasons we haven't queued it yet. So any help with testing here is appreciated. Thanks. -- Catalin
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Bhupesh, On 2020/10/6 1:42, Bhupesh Sharma wrote: > Hi Catalin, Chen, > > On Mon, Oct 5, 2020 at 10:39 PM Catalin Marinas > wrote: >> On Sat, Sep 12, 2020 at 06:44:29AM -0500, John Donnelly wrote: >>> On 9/7/20 8:47 AM, Chen Zhou wrote: Chen Zhou (9): x86: kdump: move CRASH_ALIGN to 2M x86: kdump: make the lower bound of crash kernel reservation consistent x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]() x86: kdump: move reserve_crashkernel[_low]() into crash_core.c arm64: kdump: introduce some macroes for crash kernel reservation arm64: kdump: reimplement crashkernel=X kdump: add threshold for the required memory arm64: kdump: add memory for devices by DT property linux,usable-memory-range kdump: update Documentation about crashkernel >> [...] >>> I did a brief unit-test on 5.9-rc4. >>> >>> Please add: >>> >>> Tested-by: John Donnelly >> Thanks for testing. >> >>> This activity is over a year old. It needs accepted. >> It's getting there, hopefully in 5.11. There are some minor tweaks to >> address. > I think my earlier email with the test results on this series bounced > off the mailing list server (for some weird reason), but I still see > several issues with this patchset. I will add specific issues in the > review comments for each patch again, but overall, with a crashkernel > size of say 786M, I see the following issue: > > # cat /proc/cmdline > BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> > rd.lvm.lv=<..snip..> crashkernel=786M > > I see two regions of size 786M and 256M reserved in low and high > regions respectively, So we reserve a total of 1042M of memory, which > is an incorrect behaviour: > > # dmesg | grep -i crash > [0.00] Reserving 256MB of low memory at 2816MB for crashkernel > (System low RAM: 768MB) > [0.00] Reserving 786MB of memory at 654158MB for crashkernel > (System RAM: 130816MB) > [0.00] Kernel command line: > BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ > root=/dev/mapper/rhel_ampere--hr330a--03-root ro > rd.lvm.lv=rhel_ampere-hr330a-03/root > rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M > > # cat /proc/iomem | grep -i crash > b000-bfff : Crash kernel (low) > bfcbe0-bffcff : Crash kernel > > IMO, we should test this feature more before including this in 5.11 Thanks for you test. This behavior is what we what. What is the correct behavior you think? Besides, this feature is been tested by John and PK, and i test for various parameters. We may miss something, any comments are welcome. Thanks, Chen Zhou > > Thanks, > Bhupesh > > . >
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
Hi Catalin, Chen, On Mon, Oct 5, 2020 at 10:39 PM Catalin Marinas wrote: > > On Sat, Sep 12, 2020 at 06:44:29AM -0500, John Donnelly wrote: > > On 9/7/20 8:47 AM, Chen Zhou wrote: > > > Chen Zhou (9): > > >x86: kdump: move CRASH_ALIGN to 2M > > >x86: kdump: make the lower bound of crash kernel reservation > > > consistent > > >x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions > > > reserve_crashkernel[_low]() > > >x86: kdump: move reserve_crashkernel[_low]() into crash_core.c > > >arm64: kdump: introduce some macroes for crash kernel reservation > > >arm64: kdump: reimplement crashkernel=X > > >kdump: add threshold for the required memory > > >arm64: kdump: add memory for devices by DT property > > > linux,usable-memory-range > > >kdump: update Documentation about crashkernel > [...] > > I did a brief unit-test on 5.9-rc4. > > > > Please add: > > > > Tested-by: John Donnelly > > Thanks for testing. > > > This activity is over a year old. It needs accepted. > > It's getting there, hopefully in 5.11. There are some minor tweaks to > address. I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue: # cat /proc/cmdline BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour: # dmesg | grep -i crash [0.00] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB) [0.00] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB) [0.00] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M # cat /proc/iomem | grep -i crash b000-bfff : Crash kernel (low) bfcbe0-bffcff : Crash kernel IMO, we should test this feature more before including this in 5.11 Thanks, Bhupesh
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On Sat, Sep 12, 2020 at 06:44:29AM -0500, John Donnelly wrote: > On 9/7/20 8:47 AM, Chen Zhou wrote: > > Chen Zhou (9): > >x86: kdump: move CRASH_ALIGN to 2M > >x86: kdump: make the lower bound of crash kernel reservation > > consistent > >x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions > > reserve_crashkernel[_low]() > >x86: kdump: move reserve_crashkernel[_low]() into crash_core.c > >arm64: kdump: introduce some macroes for crash kernel reservation > >arm64: kdump: reimplement crashkernel=X > >kdump: add threshold for the required memory > >arm64: kdump: add memory for devices by DT property > > linux,usable-memory-range > >kdump: update Documentation about crashkernel [...] > I did a brief unit-test on 5.9-rc4. > > Please add: > > Tested-by: John Donnelly Thanks for testing. > This activity is over a year old. It needs accepted. It's getting there, hopefully in 5.11. There are some minor tweaks to address. -- Catalin
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
> On Sep 15, 2020, at 2:16 AM, chenzhou wrote:
>
>
>
> On 2020/9/7 21:47, Chen Zhou wrote:
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which
>> will fail when there is no enough low memory.
>> 2. If reserving crashkernel above 4G, in this case, crash dump
>> kernel will boot failure because there is no low memory available
>> for allocation.
>> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
>> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
>> the devices in crash dump kernel need to use ZONE_DMA will alloc
>> fail.
>>
>> To solve these issues, change the behavior of crashkernel=X.
>> crashkernel=X tries low allocation in DMA zone, and fall back to
>> high allocation if it fails.
>> If requized size X is too large and leads to very little low memory
>> in DMA zone after low allocation, the system may not work normally.
>> So add a threshold and go for high allocation directly if the required
>> size is too large. The value of threshold is set as the half of
>> the low memory.
>>
>> We can also use "crashkernel=X,high" to select a high region above
>> DMA zone, which also tries to allocate at least 256M low memory in
>> DMA zone automatically.
>> "crashkernel=Y,low" can be used to allocate specified size low memory.
>> For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
>>
>> When reserving crashkernel in high memory, some low memory is reserved
>> for crash dump kernel devices. So there may be two regions reserved for
>> crash dump kernel.
>> In order to distinct from the high region and make no effect to the use
>> of existing kexec-tools, rename the low region as "Crash kernel (low)",
>> and pass the low region by reusing DT property
>> "linux,usable-memory-range". We made the low memory region as the last
>> range of "linux,usable-memory-range" to keep compatibility with existing
>> user-space and older kdump kernels.
>>
>> Besides, we need to modify kexec-tools:
>> arm64: support more than one crash kernel regions(see [1])
>>
>> Another update is document about DT property 'linux,usable-memory-range':
>> schemas: update 'linux,usable-memory-range' node schema(see [2])
>>
>> This patchset contains the following nine patches:
>> 0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch
>> 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
>> 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
>> 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
>> 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
>> 0006-arm64-kdump-reimplement-crashkernel-X.patch
>> 0007-kdump-add-threshold-for-the-required-memory.patch
>> 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
>> 0009-kdump-update-Documentation-about-crashkernel.patch
>>
>> 0001-0003 are some x86 cleanups which prepares for making
>> functionsreserve_crashkernel[_low]() generic.
>>
>> 0004 makes functions reserve_crashkernel[_low]() generic.
>> 0005-0006 reimplements crashkernel=X.
>> 0007 adds threshold for the required memory.
>> 0008 adds memory for devices by DT property linux,usable-memory-range.
>> 0009 updates the doc.
> Hi Catalin and Dave,
Hi,
This patch set has been going on since May, 2019. When will this be
accepted and integrated into a rc build ?
>
> Any other suggestions about this patchset? Let me know if you have any
> questions.
>
> Thanks,
> Chen Zhou
>>
>> Changes since [v11]
>> - Rebased on top of 5.9-rc4.
>> - Make the function reserve_crashkernel() of x86 generic.
>> Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
>> and arm64 use the generic version to reimplement crashkernel=X.
>>
>> Changes since [v10]
>> - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
>>
>> Changes since [v9]
>> - Patch 1 add Acked-by from Dave.
>> - Update patch 5 according to Dave's comments.
>> - Update chosen schema.
>>
>> Changes since [v8]
>> - Reuse DT property "linux,usable-memory-range".
>> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
>> low
>> memory region.
>> - Fix kdump broken with ZONE_DMA reintroduced.
>> - Update chosen schema.
>>
>> Changes since [v7]
>> - Move x86 CRASH_ALIGN to 2M
>> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
>> - Update Documentation/devicetree/bindings/chosen.txt.
>> Add corresponding documentation to
>> Documentation/devicetree/bindings/chosen.txt
>> suggested by Arnd.
>> - Add Tested-by from Jhon and pk.
>>
>> Changes since [v6]
>> - Fix build errors reported by kbuild test robot.
>>
>> Changes since [v5]
>> - Move reserve_crashkernel_low() into kernel/crash_core.c.
>> - Delete crashkernel=X,high.
>> - Modify crashkernel=X,low.
>> If crashkernel=X,low is specified simultaneously, reserve spcified size low
>> memory for crash kdump kernel devices firstly and then reserve memory above
>> 4G.
>> In
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On 2020/9/7 21:47, Chen Zhou wrote:
> There are following issues in arm64 kdump:
> 1. We use crashkernel=X to reserve crashkernel below 4G, which
> will fail when there is no enough low memory.
> 2. If reserving crashkernel above 4G, in this case, crash dump
> kernel will boot failure because there is no low memory available
> for allocation.
> 3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
> if the memory reserved for crash dump kernel falled in ZONE_DMA32,
> the devices in crash dump kernel need to use ZONE_DMA will alloc
> fail.
>
> To solve these issues, change the behavior of crashkernel=X.
> crashkernel=X tries low allocation in DMA zone, and fall back to
> high allocation if it fails.
> If requized size X is too large and leads to very little low memory
> in DMA zone after low allocation, the system may not work normally.
> So add a threshold and go for high allocation directly if the required
> size is too large. The value of threshold is set as the half of
> the low memory.
>
> We can also use "crashkernel=X,high" to select a high region above
> DMA zone, which also tries to allocate at least 256M low memory in
> DMA zone automatically.
> "crashkernel=Y,low" can be used to allocate specified size low memory.
> For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
>
> When reserving crashkernel in high memory, some low memory is reserved
> for crash dump kernel devices. So there may be two regions reserved for
> crash dump kernel.
> In order to distinct from the high region and make no effect to the use
> of existing kexec-tools, rename the low region as "Crash kernel (low)",
> and pass the low region by reusing DT property
> "linux,usable-memory-range". We made the low memory region as the last
> range of "linux,usable-memory-range" to keep compatibility with existing
> user-space and older kdump kernels.
>
> Besides, we need to modify kexec-tools:
> arm64: support more than one crash kernel regions(see [1])
>
> Another update is document about DT property 'linux,usable-memory-range':
> schemas: update 'linux,usable-memory-range' node schema(see [2])
>
> This patchset contains the following nine patches:
> 0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch
> 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
> 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
> 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
> 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
> 0006-arm64-kdump-reimplement-crashkernel-X.patch
> 0007-kdump-add-threshold-for-the-required-memory.patch
> 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
> 0009-kdump-update-Documentation-about-crashkernel.patch
>
> 0001-0003 are some x86 cleanups which prepares for making
> functionsreserve_crashkernel[_low]() generic.
>
> 0004 makes functions reserve_crashkernel[_low]() generic.
> 0005-0006 reimplements crashkernel=X.
> 0007 adds threshold for the required memory.
> 0008 adds memory for devices by DT property linux,usable-memory-range.
> 0009 updates the doc.
Hi Catalin and Dave,
Any other suggestions about this patchset? Let me know if you have any
questions.
Thanks,
Chen Zhou
>
> Changes since [v11]
> - Rebased on top of 5.9-rc4.
> - Make the function reserve_crashkernel() of x86 generic.
> Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
> and arm64 use the generic version to reimplement crashkernel=X.
>
> Changes since [v10]
> - Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
>
> Changes since [v9]
> - Patch 1 add Acked-by from Dave.
> - Update patch 5 according to Dave's comments.
> - Update chosen schema.
>
> Changes since [v8]
> - Reuse DT property "linux,usable-memory-range".
> Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the
> low
> memory region.
> - Fix kdump broken with ZONE_DMA reintroduced.
> - Update chosen schema.
>
> Changes since [v7]
> - Move x86 CRASH_ALIGN to 2M
> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
> - Update Documentation/devicetree/bindings/chosen.txt.
> Add corresponding documentation to
> Documentation/devicetree/bindings/chosen.txt
> suggested by Arnd.
> - Add Tested-by from Jhon and pk.
>
> Changes since [v6]
> - Fix build errors reported by kbuild test robot.
>
> Changes since [v5]
> - Move reserve_crashkernel_low() into kernel/crash_core.c.
> - Delete crashkernel=X,high.
> - Modify crashkernel=X,low.
> If crashkernel=X,low is specified simultaneously, reserve spcified size low
> memory for crash kdump kernel devices firstly and then reserve memory above
> 4G.
> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then
> pass to crash dump kernel by DT property "linux,low-memory-range".
> - Update Documentation/admin-guide/kdump/kdump.rst.
>
> Changes since [v4]
> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
>
> Changes since [v3]
> - Add
Re: [PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
On 9/7/20 8:47 AM, Chen Zhou wrote:
There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which
will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump
kernel will boot failure because there is no low memory available
for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
if the memory reserved for crash dump kernel falled in ZONE_DMA32,
the devices in crash dump kernel need to use ZONE_DMA will alloc
fail.
To solve these issues, change the behavior of crashkernel=X.
crashkernel=X tries low allocation in DMA zone, and fall back to
high allocation if it fails.
If requized size X is too large and leads to very little low memory
in DMA zone after low allocation, the system may not work normally.
So add a threshold and go for high allocation directly if the required
size is too large. The value of threshold is set as the half of
the low memory.
We can also use "crashkernel=X,high" to select a high region above
DMA zone, which also tries to allocate at least 256M low memory in
DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory.
For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
When reserving crashkernel in high memory, some low memory is reserved
for crash dump kernel devices. So there may be two regions reserved for
crash dump kernel.
In order to distinct from the high region and make no effect to the use
of existing kexec-tools, rename the low region as "Crash kernel (low)",
and pass the low region by reusing DT property
"linux,usable-memory-range". We made the low memory region as the last
range of "linux,usable-memory-range" to keep compatibility with existing
user-space and older kdump kernels.
Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])
Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])
This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch
0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
0006-arm64-kdump-reimplement-crashkernel-X.patch
0007-kdump-add-threshold-for-the-required-memory.patch
0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
0009-kdump-update-Documentation-about-crashkernel.patch
0001-0003 are some x86 cleanups which prepares for making
functionsreserve_crashkernel[_low]() generic.
0004 makes functions reserve_crashkernel[_low]() generic.
0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range.
0009 updates the doc.
Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic.
Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
and arm64 use the generic version to reimplement crashkernel=X.
Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.
Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low
memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.
Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt
suggested by Arnd.
- Add Tested-by from Jhon and pk.
Changes since [v6]
- Fix build errors reported by kbuild test robot.
Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low
memory for crash kdump kernel devices firstly and then reserve memory above 4G.
In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then
pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.
Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.
Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as
two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate
patch.
Changes since [v1]:
- Move common reserve_crashkernel_low() code into
[PATCH v12 0/9] support reserving crashkernel above 4G on arm64 kdump
There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which
will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump
kernel will boot failure because there is no low memory available
for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"),
if the memory reserved for crash dump kernel falled in ZONE_DMA32,
the devices in crash dump kernel need to use ZONE_DMA will alloc
fail.
To solve these issues, change the behavior of crashkernel=X.
crashkernel=X tries low allocation in DMA zone, and fall back to
high allocation if it fails.
If requized size X is too large and leads to very little low memory
in DMA zone after low allocation, the system may not work normally.
So add a threshold and go for high allocation directly if the required
size is too large. The value of threshold is set as the half of
the low memory.
We can also use "crashkernel=X,high" to select a high region above
DMA zone, which also tries to allocate at least 256M low memory in
DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory.
For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
When reserving crashkernel in high memory, some low memory is reserved
for crash dump kernel devices. So there may be two regions reserved for
crash dump kernel.
In order to distinct from the high region and make no effect to the use
of existing kexec-tools, rename the low region as "Crash kernel (low)",
and pass the low region by reusing DT property
"linux,usable-memory-range". We made the low memory region as the last
range of "linux,usable-memory-range" to keep compatibility with existing
user-space and older kdump kernels.
Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])
Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])
This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch
0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch
0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch
0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch
0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch
0006-arm64-kdump-reimplement-crashkernel-X.patch
0007-kdump-add-threshold-for-the-required-memory.patch
0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch
0009-kdump-update-Documentation-about-crashkernel.patch
0001-0003 are some x86 cleanups which prepares for making
functionsreserve_crashkernel[_low]() generic.
0004 makes functions reserve_crashkernel[_low]() generic.
0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range.
0009 updates the doc.
Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic.
Suggested by Catalin, make the function reserve_crashkernel() of x86 generic
and arm64 use the generic version to reimplement crashkernel=X.
Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.
Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.
Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low
memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.
Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt
suggested by Arnd.
- Add Tested-by from Jhon and pk.
Changes since [v6]
- Fix build errors reported by kbuild test robot.
Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low
memory for crash kdump kernel devices firstly and then reserve memory above 4G.
In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then
pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.
Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.
Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.
Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as
two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate
patch.
Changes since [v1]:
- Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
- Remove
