Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 6/19/20 3:21 AM, chenzhou wrote: On 2020/6/19 10:32, John Donnelly wrote: On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: Hi All, On Wed, Jun 3, 2020 at 9:03 PM John Donnelly wrote: On Jun 3, 2020, at 8:20 AM, chenzhou wrote: Hi, On 2020/6/3 19:47, Prabhakar Kushwaha wrote: Hi Chen, On Tue, Jun 2, 2020 at 8:12 PM John Donnelly wrote: On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < prabhakar.p...@gmail.com> wrote: On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi . See below ! On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: Hi John, On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi, On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou < chenzho...@huawei.com> wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ Also **without** this patch-set " https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ " This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.369984] Call trace: [ 30.369989] dump_backtrace+0x0/0x1f8 [ 30.369991] show_stack+0x20/0x30 [ 30.369997] dump_stack+0xc0/0x10c [ 30.370001] warn_alloc+0x10c/0x178 [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0 xb50 [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 [ 30.370008] alloc_page_interleave+0x24/0x98 [ 30.370011] alloc_pages_current+0xe4/0x108 [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 [ 30.370020] do_one_initcall+0x54/0x228 [ 30.370027] kernel_init_freeable+0x228/0x2cc [ 30.370031] kernel_init+0x1c/0x110 [ 30.370034] ret_from_fork+0x10/0x18 [ 30.370036] Mem-Info: [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 [ 30.370064] free:1537719 free_pcp:219 free_cma:0 [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370079] Node 0 DMA free:0kB min:0kB low:0kB high:0kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:128kB managed:0kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 30.370084] lowmem_reserve[]: 0 250 6063 6063 [ 30.370090] Node 0 DMA32 free:256000kB min:408kB low:664kB high:920kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 2020/6/19 10:32, John Donnelly wrote: > > On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: >> On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: >>> Hi All, >>> >>> On Wed, Jun 3, 2020 at 9:03 PM John Donnelly >>> wrote: > On Jun 3, 2020, at 8:20 AM, chenzhou wrote: > > Hi, > > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote: >> Hi Chen, >> >> On Tue, Jun 2, 2020 at 8:12 PM John Donnelly >> wrote: >>> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < prabhakar.p...@gmail.com> wrote: On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < john.p.donne...@oracle.com> wrote: > Hi . See below ! > >> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma >> wrote: >> >> Hi John, >> >> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < >> john.p.donne...@oracle.com> wrote: >>> Hi, >>> >>> >>> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou < chenzho...@huawei.com> wrote: > This patch series enable reserving crashkernel above 4G in > arm64. > > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, > which will fail > when there is no enough low memory. > 2. Currently, crashkernel=Y@X can be used to reserve > crashkernel above 4G, > in this case, if swiotlb or DMA buffers are required, > crash dump kernel > will boot failure because there is no low memory available > for allocation. > We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and >> https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ Also **without** this patch-set " >> https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ " This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in >> https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ >>without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.369984] Call trace: [ 30.369989] dump_backtrace+0x0/0x1f8 [ 30.369991] show_stack+0x20/0x30 [ 30.369997] dump_stack+0xc0/0x10c [ 30.370001] warn_alloc+0x10c/0x178 [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0 xb50 [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 [ 30.370008] alloc_page_interleave+0x24/0x98 [ 30.370011] alloc_pages_current+0xe4/0x108 [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 [ 30.370020] do_one_initcall+0x54/0x228 [ 30.370027] kernel_init_freeable+0x228/0x2cc [ 30.370031] kernel_init+0x1c/0x110 [ 30.370034] ret_from_fork+0x10/0x18 [ 30.370036] Mem-Info: [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 [ 30.370064]
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: Hi All, On Wed, Jun 3, 2020 at 9:03 PM John Donnelly wrote: On Jun 3, 2020, at 8:20 AM, chenzhou wrote: Hi, On 2020/6/3 19:47, Prabhakar Kushwaha wrote: Hi Chen, On Tue, Jun 2, 2020 at 8:12 PM John Donnelly wrote: On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < prabhakar.p...@gmail.com> wrote: On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi . See below ! On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: Hi John, On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi, On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou < chenzho...@huawei.com> wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ Also **without** this patch-set " https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ " This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.369984] Call trace: [ 30.369989] dump_backtrace+0x0/0x1f8 [ 30.369991] show_stack+0x20/0x30 [ 30.369997] dump_stack+0xc0/0x10c [ 30.370001] warn_alloc+0x10c/0x178 [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0 xb50 [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 [ 30.370008] alloc_page_interleave+0x24/0x98 [ 30.370011] alloc_pages_current+0xe4/0x108 [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 [ 30.370020] do_one_initcall+0x54/0x228 [ 30.370027] kernel_init_freeable+0x228/0x2cc [ 30.370031] kernel_init+0x1c/0x110 [ 30.370034] ret_from_fork+0x10/0x18 [ 30.370036] Mem-Info: [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 [ 30.370064] free:1537719 free_pcp:219 free_cma:0 [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370079] Node 0 DMA free:0kB min:0kB low:0kB high:0kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:128kB managed:0kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 30.370084] lowmem_reserve[]: 0 250 6063 6063 [ 30.370090] Node 0 DMA32 free:256000kB min:408kB low:664kB high:920kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:269700kB managed:256000kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On Thu, 2020-06-04 at 21:26 -0500, John Donnelly wrote: > On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: > > On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: > > > Hi All, > > > > > > On Wed, Jun 3, 2020 at 9:03 PM John Donnelly > > > wrote: > > > > > On Jun 3, 2020, at 8:20 AM, chenzhou wrote: > > > > > > > > > > Hi, > > > > > > > > > > > > > > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote: > > > > > > Hi Chen, > > > > > > > > > > > > On Tue, Jun 2, 2020 at 8:12 PM John Donnelly < > > > > > > john.p.donne...@oracle.com > > > > > > > wrote: > > > > > > > > > > > > > > > On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < > > > > > > > > prabhakar.p...@gmail.com> wrote: > > > > > > > > > > > > > > > > On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < > > > > > > > > john.p.donne...@oracle.com> wrote: > > > > > > > > > Hi . See below ! > > > > > > > > > > > > > > > > > > > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma < > > > > > > > > > > bhsha...@redhat.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi John, > > > > > > > > > > > > > > > > > > > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < > > > > > > > > > > john.p.donne...@oracle.com> wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > > > > > > > > > > > > Hi Chen, > > > > > > > > > > > > > > > > > > > > > > > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou < > > > > > > > > > > > > chenzho...@huawei.com> wrote: > > > > > > > > > > > > > This patch series enable reserving crashkernel above > > > > > > > > > > > > > 4G in > > > > > > > > > > > > > arm64. > > > > > > > > > > > > > > > > > > > > > > > > > > There are following issues in arm64 kdump: > > > > > > > > > > > > > 1. We use crashkernel=X to reserve crashkernel below > > > > > > > > > > > > > 4G, > > > > > > > > > > > > > which will fail > > > > > > > > > > > > > when there is no enough low memory. > > > > > > > > > > > > > 2. Currently, crashkernel=Y@X can be used to reserve > > > > > > > > > > > > > crashkernel above 4G, > > > > > > > > > > > > > in this case, if swiotlb or DMA buffers are required, > > > > > > > > > > > > > crash dump kernel > > > > > > > > > > > > > will boot failure because there is no low memory > > > > > > > > > > > > > available > > > > > > > > > > > > > for allocation. > > > > > > > > > > > > > > > > > > > > > > > > > We are getting "warn_alloc" [1] warning during boot of > > > > > > > > > > > > kdump > > > > > > > > > > > > kernel > > > > > > > > > > > > with bootargs as [2] of primary kernel. > > > > > > > > > > > > This error observed on ThunderX2 ARM64 platform. > > > > > > > > > > > > > > > > > > > > > > > > It is observed with latest upstream tag (v5.7-rc3) with > > > > > > > > > > > > this > > > > > > > > > > > > patch set > > > > > > > > > > > > and > > > > > > > > > > > > > > https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ > > > > > > > > > > > > Also **without** this patch-set > > > > > > > > > > > > " > > > > > > > > > > > > > > https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ > > > > > > > > > > > > " > > > > > > > > > > > > > > > > > > > > > > > > This issue comes whenever crashkernel memory is reserved > > > > > > > > > > > > after 0xc000_. > > > > > > > > > > > > More details discussed earlier in > > > > > > > > > > > > > > https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ > >without > > > > > > > > > > > > any > > > > > > > > > > > > solution > > > > > > > > > > > > > > > > > > > > > > > > This patch-set is expected to solve similar kind of > > > > > > > > > > > > issue. > > > > > > > > > > > > i.e. low memory is only targeted for DMA, swiotlb; So > > > > > > > > > > > > above > > > > > > > > > > > > mentioned > > > > > > > > > > > > observation should be considered/fixed. . > > > > > > > > > > > > > > > > > > > > > > > > --pk > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS > > > > > > > > > > > > TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ > > > > > > > > > > > > [ 30.367696] NET: Registered protocol family 16 > > > > > > > > > > > > [ 30.369973] swapper/0: page allocation failure: > > > > > > > > > > > > order:6, > > > > > > > > > > > > mode:0x1(GFP_DMA), > > > > > > > > > > > > nodemask=(null),cpuset=/,mems_allowed=0 > > > > > > > > > > > > [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted > > > > > > > > > > > > 5.7.0-rc3+ #121 > > > > > > > > > > > > [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, > > > > > > > > > > >
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 6/4/20 12:01 PM, Nicolas Saenz Julienne wrote: On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: Hi All, On Wed, Jun 3, 2020 at 9:03 PM John Donnelly wrote: On Jun 3, 2020, at 8:20 AM, chenzhou wrote: Hi, On 2020/6/3 19:47, Prabhakar Kushwaha wrote: Hi Chen, On Tue, Jun 2, 2020 at 8:12 PM John Donnelly wrote: On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < prabhakar.p...@gmail.com> wrote: On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi . See below ! On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: Hi John, On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < john.p.donne...@oracle.com> wrote: Hi, On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou < chenzho...@huawei.com> wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ Also **without** this patch-set " https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ " This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.369984] Call trace: [ 30.369989] dump_backtrace+0x0/0x1f8 [ 30.369991] show_stack+0x20/0x30 [ 30.369997] dump_stack+0xc0/0x10c [ 30.370001] warn_alloc+0x10c/0x178 [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0 xb50 [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 [ 30.370008] alloc_page_interleave+0x24/0x98 [ 30.370011] alloc_pages_current+0xe4/0x108 [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 [ 30.370020] do_one_initcall+0x54/0x228 [ 30.370027] kernel_init_freeable+0x228/0x2cc [ 30.370031] kernel_init+0x1c/0x110 [ 30.370034] ret_from_fork+0x10/0x18 [ 30.370036] Mem-Info: [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 [ 30.370064] free:1537719 free_pcp:219 free_cma:0 [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370079] Node 0 DMA free:0kB min:0kB low:0kB high:0kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:128kB managed:0kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [ 30.370084] lowmem_reserve[]: 0 250 6063 6063 [ 30.370090] Node 0 DMA32 free:256000kB min:408kB low:664kB high:920kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:269700kB managed:256000kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On Thu, 2020-06-04 at 01:17 +0530, Bhupesh Sharma wrote: > Hi All, > > On Wed, Jun 3, 2020 at 9:03 PM John Donnelly > wrote: > > > > > > > On Jun 3, 2020, at 8:20 AM, chenzhou wrote: > > > > > > Hi, > > > > > > > > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote: > > > > Hi Chen, > > > > > > > > On Tue, Jun 2, 2020 at 8:12 PM John Donnelly > > > > wrote: > > > > > > > > > > > On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha < > > > > > > prabhakar.p...@gmail.com> wrote: > > > > > > > > > > > > On Tue, Jun 2, 2020 at 3:29 AM John Donnelly < > > > > > > john.p.donne...@oracle.com> wrote: > > > > > > > Hi . See below ! > > > > > > > > > > > > > > > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma > > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi John, > > > > > > > > > > > > > > > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly < > > > > > > > > john.p.donne...@oracle.com> wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > > > > > > > > > > Hi Chen, > > > > > > > > > > > > > > > > > > > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou < > > > > > > > > > > chenzho...@huawei.com> wrote: > > > > > > > > > > > This patch series enable reserving crashkernel above 4G in > > > > > > > > > > > arm64. > > > > > > > > > > > > > > > > > > > > > > There are following issues in arm64 kdump: > > > > > > > > > > > 1. We use crashkernel=X to reserve crashkernel below 4G, > > > > > > > > > > > which will fail > > > > > > > > > > > when there is no enough low memory. > > > > > > > > > > > 2. Currently, crashkernel=Y@X can be used to reserve > > > > > > > > > > > crashkernel above 4G, > > > > > > > > > > > in this case, if swiotlb or DMA buffers are required, > > > > > > > > > > > crash dump kernel > > > > > > > > > > > will boot failure because there is no low memory available > > > > > > > > > > > for allocation. > > > > > > > > > > > > > > > > > > > > > We are getting "warn_alloc" [1] warning during boot of kdump > > > > > > > > > > kernel > > > > > > > > > > with bootargs as [2] of primary kernel. > > > > > > > > > > This error observed on ThunderX2 ARM64 platform. > > > > > > > > > > > > > > > > > > > > It is observed with latest upstream tag (v5.7-rc3) with this > > > > > > > > > > patch set > > > > > > > > > > and > > > > > > > > > > https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ > > > > > > > > > > Also **without** this patch-set > > > > > > > > > > " > > > > > > > > > > https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ > > > > > > > > > > " > > > > > > > > > > > > > > > > > > > > This issue comes whenever crashkernel memory is reserved > > > > > > > > > > after 0xc000_. > > > > > > > > > > More details discussed earlier in > > > > > > > > > > https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ without > > > > > > > > > > any > > > > > > > > > > solution > > > > > > > > > > > > > > > > > > > > This patch-set is expected to solve similar kind of issue. > > > > > > > > > > i.e. low memory is only targeted for DMA, swiotlb; So above > > > > > > > > > > mentioned > > > > > > > > > > observation should be considered/fixed. . > > > > > > > > > > > > > > > > > > > > --pk > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS > > > > > > > > > > TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ > > > > > > > > > > [ 30.367696] NET: Registered protocol family 16 > > > > > > > > > > [ 30.369973] swapper/0: page allocation failure: order:6, > > > > > > > > > > mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 > > > > > > > > > > [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted > > > > > > > > > > 5.7.0-rc3+ #121 > > > > > > > > > > [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS > > > > > > > > > > TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ > > > > > > > > > > [ 30.369984] Call trace: > > > > > > > > > > [ 30.369989] dump_backtrace+0x0/0x1f8 > > > > > > > > > > [ 30.369991] show_stack+0x20/0x30 > > > > > > > > > > [ 30.369997] dump_stack+0xc0/0x10c > > > > > > > > > > [ 30.370001] warn_alloc+0x10c/0x178 > > > > > > > > > > [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0 > > > > > > > > > > xb50 > > > > > > > > > > [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 > > > > > > > > > > [ 30.370008] alloc_page_interleave+0x24/0x98 > > > > > > > > > > [ 30.370011] alloc_pages_current+0xe4/0x108 > > > > > > > > > > [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 > > > > > > > > > > [ 30.370020]
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On Thu, Jun 04, 2020 at 01:17:06AM +0530, Bhupesh Sharma wrote:
> On Wed, Jun 3, 2020 at 9:03 PM John Donnelly
> wrote:
> > > On Jun 3, 2020, at 8:20 AM, chenzhou wrote:
> > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote:
> > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > index 7f9e5a6dc48c..bd67b90d35bd 100644
> > --- a/kernel/crash_core.c
> > +++ b/kernel/crash_core.c
> > @@ -354,7 +354,7 @@ int __init reserve_crashkernel_low(void)
> > return 0;
> > }
> >
> > - low_base = memblock_find_in_range(0, 1ULL << 32, low_size,
> > CRASH_ALIGN);
> > + low_base = memblock_find_in_range(0,0xc000, low_size,
> > CRASH_ALIGN);
> > if (!low_base) {
> > pr_err("Cannot reserve %ldMB crashkernel low memory,
> > please try smaller size.\n",
> > (unsigned long)(low_size >> 20));
> >
> >
> > >>>I suspect 0xc000 would need to be a CONFIG item and not
> > >>> hard-coded.
> > >>>
> > >> if you consider this as valid change, can you please incorporate as
> > >> part of your patch-set.
> > >
> > > After commit 1a8e1cef7 ("arm64: use both ZONE_DMA and
> > > ZONE_DMA32")，the 0-4G memory is splited to DMA [mem
> > > 0x-0x3fff] and DMA32 [mem
> > > 0x4000-0x] on arm64.
> > >
> > > From the above discussion, on your platform, the low crashkernel fall
> > > in DMA32 region, but your environment needs to access DMA region, so
> > > there is the call trace.
> > >
> > > I have a question, why do you choose 0xc000 here?
> > >
> > > Besides, this is common code, we also need to consider about x86.
> > >
> >
> > + nsaenzjulie...@suse.de
> >
> > Exactly . This is why it needs to be a CONFIG option for Raspberry
> > .., or device tree option.
> >
> >
> > We could revert 1a8e1cef7 since it broke Arm kdump too.
>
> Well, unfortunately the patch for commit 1a8e1cef7603 ("arm64: use
> both ZONE_DMA and ZONE_DMA32") was not Cc'ed to the kexec mailing
> list, thus we couldn't get many eyes on it for a thorough review from
> kexec/kdump p-o-v.
>
> Also we historically never had distinction in common arch code on the
> basis of the intended end use-case: embedded, server or automotive, so
> I am not sure introducing a Raspberry specific CONFIG option would be
> a good idea.
Right, we need a fix that works for everybody, since we try hard for a
single Image that works for all platforms.
What I don't really understand is why, with Chen's patches applied, we can't
just keep the crashkernel out of the DMA zones altogether when no base is
specified. I guess I'll just look out for your patch!
Will
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi All, On Wed, Jun 3, 2020 at 9:03 PM John Donnelly wrote: > > > > > On Jun 3, 2020, at 8:20 AM, chenzhou wrote: > > > > Hi, > > > > > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote: > >> Hi Chen, > >> > >> On Tue, Jun 2, 2020 at 8:12 PM John Donnelly > >> wrote: > >>> > >>> > On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha > wrote: > > On Tue, Jun 2, 2020 at 3:29 AM John Donnelly > wrote: > > Hi . See below ! > > > >> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > >> > >> Hi John, > >> > >> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > >> wrote: > >>> Hi, > >>> > >>> > >>> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > Hi Chen, > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou > wrote: > > This patch series enable reserving crashkernel above 4G in arm64. > > > > There are following issues in arm64 kdump: > > 1. We use crashkernel=X to reserve crashkernel below 4G, which will > > fail > > when there is no enough low memory. > > 2. Currently, crashkernel=Y@X can be used to reserve crashkernel > > above 4G, > > in this case, if swiotlb or DMA buffers are required, crash dump > > kernel > > will boot failure because there is no low memory available for > > allocation. > > > We are getting "warn_alloc" [1] warning during boot of kdump kernel > with bootargs as [2] of primary kernel. > This error observed on ThunderX2 ARM64 platform. > > It is observed with latest upstream tag (v5.7-rc3) with this patch > set > and > https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ > Also **without** this patch-set > "https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$; > > This issue comes whenever crashkernel memory is reserved after > 0xc000_. > More details discussed earlier in > https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ > without any > solution > > This patch-set is expected to solve similar kind of issue. > i.e. low memory is only targeted for DMA, swiotlb; So above mentioned > observation should be considered/fixed. . > > --pk > > [1] > [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS > TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ > [ 30.367696] NET: Registered protocol family 16 > [ 30.369973] swapper/0: page allocation failure: order:6, > mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 > [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ > #121 > [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS > TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ > [ 30.369984] Call trace: > [ 30.369989] dump_backtrace+0x0/0x1f8 > [ 30.369991] show_stack+0x20/0x30 > [ 30.369997] dump_stack+0xc0/0x10c > [ 30.370001] warn_alloc+0x10c/0x178 > [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0xb50 > [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 > [ 30.370008] alloc_page_interleave+0x24/0x98 > [ 30.370011] alloc_pages_current+0xe4/0x108 > [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 > [ 30.370020] do_one_initcall+0x54/0x228 > [ 30.370027] kernel_init_freeable+0x228/0x2cc > [ 30.370031] kernel_init+0x1c/0x110 > [ 30.370034] ret_from_fork+0x10/0x18 > [ 30.370036] Mem-Info: > [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 > [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 > [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 > [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 > [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 > [ 30.370064] free:1537719 free_pcp:219 free_cma:0 > [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB > active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB > isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB > shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB > unstable:0kB all_unreclaimable? no > [ 30.370073] Node 1 active_anon:0kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
> On Jun 3, 2020, at 8:20 AM, chenzhou wrote: > > Hi, > > > On 2020/6/3 19:47, Prabhakar Kushwaha wrote: >> Hi Chen, >> >> On Tue, Jun 2, 2020 at 8:12 PM John Donnelly >> wrote: >>> >>> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha wrote: On Tue, Jun 2, 2020 at 3:29 AM John Donnelly wrote: > Hi . See below ! > >> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: >> >> Hi John, >> >> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly >> wrote: >>> Hi, >>> >>> >>> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: > This patch series enable reserving crashkernel above 4G in arm64. > > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which will > fail > when there is no enough low memory. > 2. Currently, crashkernel=Y@X can be used to reserve crashkernel > above 4G, > in this case, if swiotlb or DMA buffers are required, crash dump > kernel > will boot failure because there is no low memory available for > allocation. > We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ Also **without** this patch-set "https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$; This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.369984] Call trace: [ 30.369989] dump_backtrace+0x0/0x1f8 [ 30.369991] show_stack+0x20/0x30 [ 30.369997] dump_stack+0xc0/0x10c [ 30.370001] warn_alloc+0x10c/0x178 [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0xb50 [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 [ 30.370008] alloc_page_interleave+0x24/0x98 [ 30.370011] alloc_pages_current+0xe4/0x108 [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 [ 30.370020] do_one_initcall+0x54/0x228 [ 30.370027] kernel_init_freeable+0x228/0x2cc [ 30.370031] kernel_init+0x1c/0x110 [ 30.370034] ret_from_fork+0x10/0x18 [ 30.370036] Mem-Info: [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 [ 30.370064] free:1537719 free_pcp:219 free_cma:0 [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi, On 2020/6/3 19:47, Prabhakar Kushwaha wrote: > Hi Chen, > > On Tue, Jun 2, 2020 at 8:12 PM John Donnelly > wrote: >> >> >>> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha >>> wrote: >>> >>> On Tue, Jun 2, 2020 at 3:29 AM John Donnelly >>> wrote: Hi . See below ! > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > > Hi John, > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > wrote: >> Hi, >> >> >> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: >>> Hi Chen, >>> >>> On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. >>> We are getting "warn_alloc" [1] warning during boot of kdump kernel >>> with bootargs as [2] of primary kernel. >>> This error observed on ThunderX2 ARM64 platform. >>> >>> It is observed with latest upstream tag (v5.7-rc3) with this patch set >>> and >>> https://urldefense.com/v3/__https://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiIAAlzu$ >>> Also **without** this patch-set >>> "https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$; >>> >>> This issue comes whenever crashkernel memory is reserved after >>> 0xc000_. >>> More details discussed earlier in >>> https://urldefense.com/v3/__https://www.spinics.net/lists/arm-kernel/msg806882.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbjC6ujMA$ >>> without any >>> solution >>> >>> This patch-set is expected to solve similar kind of issue. >>> i.e. low memory is only targeted for DMA, swiotlb; So above mentioned >>> observation should be considered/fixed. . >>> >>> --pk >>> >>> [1] >>> [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS >>> TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ >>> [ 30.367696] NET: Registered protocol family 16 >>> [ 30.369973] swapper/0: page allocation failure: order:6, >>> mode:0x1(GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 >>> [ 30.369980] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc3+ #121 >>> [ 30.369981] Hardware name: Cavium Inc. Saber/Saber, BIOS >>> TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ >>> [ 30.369984] Call trace: >>> [ 30.369989] dump_backtrace+0x0/0x1f8 >>> [ 30.369991] show_stack+0x20/0x30 >>> [ 30.369997] dump_stack+0xc0/0x10c >>> [ 30.370001] warn_alloc+0x10c/0x178 >>> [ 30.370004] __alloc_pages_slowpath.constprop.111+0xb10/0xb50 >>> [ 30.370006] __alloc_pages_nodemask+0x2b4/0x300 >>> [ 30.370008] alloc_page_interleave+0x24/0x98 >>> [ 30.370011] alloc_pages_current+0xe4/0x108 >>> [ 30.370017] dma_atomic_pool_init+0x44/0x1a4 >>> [ 30.370020] do_one_initcall+0x54/0x228 >>> [ 30.370027] kernel_init_freeable+0x228/0x2cc >>> [ 30.370031] kernel_init+0x1c/0x110 >>> [ 30.370034] ret_from_fork+0x10/0x18 >>> [ 30.370036] Mem-Info: >>> [ 30.370064] active_anon:0 inactive_anon:0 isolated_anon:0 >>> [ 30.370064] active_file:0 inactive_file:0 isolated_file:0 >>> [ 30.370064] unevictable:0 dirty:0 writeback:0 unstable:0 >>> [ 30.370064] slab_reclaimable:34 slab_unreclaimable:4438 >>> [ 30.370064] mapped:0 shmem:0 pagetables:14 bounce:0 >>> [ 30.370064] free:1537719 free_pcp:219 free_cma:0 >>> [ 30.370070] Node 0 active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB >>> isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB >>> shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB >>> unstable:0kB all_unreclaimable? no >>> [ 30.370073] Node 1 active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB >>> isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB >>> shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB >>> unstable:0kB all_unreclaimable? no >>> [ 30.370079] Node 0 DMA free:0kB min:0kB low:0kB high:0kB >>> reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB >>> active_file:0kB inactive_file:0kB unevictable:0kB
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi Chen, On Tue, Jun 2, 2020 at 8:12 PM John Donnelly wrote: > > > > > On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha > > wrote: > > > > On Tue, Jun 2, 2020 at 3:29 AM John Donnelly > > wrote: > >> > >> Hi . See below ! > >> > >>> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > >>> > >>> Hi John, > >>> > >>> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > >>> wrote: > > Hi, > > > On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > > Hi Chen, > > > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: > >> This patch series enable reserving crashkernel above 4G in arm64. > >> > >> There are following issues in arm64 kdump: > >> 1. We use crashkernel=X to reserve crashkernel below 4G, which will > >> fail > >> when there is no enough low memory. > >> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above > >> 4G, > >> in this case, if swiotlb or DMA buffers are required, crash dump kernel > >> will boot failure because there is no low memory available for > >> allocation. > >> > >> To solve these issues, introduce crashkernel=X,low to reserve specified > >> size low memory. > >> Crashkernel=X tries to reserve memory for the crash dump kernel under > >> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified > >> size low memory for crash kdump kernel devices firstly and then reserve > >> memory above 4G. > >> > >> When crashkernel is reserved above 4G in memory, that is, > >> crashkernel=X,low > >> is specified simultaneously, kernel should reserve specified size low > >> memory > >> for crash dump kernel devices. So there may be two crash kernel > >> regions, one > >> is below 4G, the other is above 4G. > >> In order to distinct from the high region and make no effect to the > >> use of > >> kexec-tools, rename the low region as "Crash kernel (low)", and add DT > >> property > >> "linux,low-memory-range" to crash dump kernel's dtb to pass the low > >> region. > >> > >> Besides, we need to modify kexec-tools: > >> arm64: kdump: add another DT property to crash dump kernel's dtb(see > >> [1]) > >> > >> The previous changes and discussions can be retrieved from: > >> > >> Changes since [v7] > >> - Move x86 CRASH_ALIGN to 2M > >> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > >> - Update Documentation/devicetree/bindings/chosen.txt > >> Add corresponding documentation to > >> Documentation/devicetree/bindings/chosen.txt suggested by Arnd. > >> - Add Tested-by from Jhon and pk > >> > >> Changes since [v6] > >> - Fix build errors reported by kbuild test robot. > >> > >> Changes since [v5] > >> - Move reserve_crashkernel_low() into kernel/crash_core.c. > >> - Delete crashkernel=X,high. > >> - Modify crashkernel=X,low. > >> If crashkernel=X,low is specified simultaneously, reserve spcified > >> size low > >> memory for crash kdump kernel devices firstly and then reserve memory > >> above 4G. > >> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, > >> and then > >> pass to crash dump kernel by DT property "linux,low-memory-range". > >> - Update Documentation/admin-guide/kdump/kdump.rst. > >> > >> Changes since [v4] > >> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > >> > >> Changes since [v3] > >> - Add memblock_cap_memory_ranges back for multiple ranges. > >> - Fix some compiling warnings. > >> > >> Changes since [v2] > >> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > >> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a > >> separate > >> patch. > >> > >> Changes since [v1]: > >> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > >> - Remove memblock_cap_memory_ranges() i added in v1 and implement that > >> in fdt_enforce_memory_region(). > >> There are at most two crash kernel regions, for two crash kernel > >> regions > >> case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > >> and then remove the memory range in the middle. > >> > >> [1]: > >> https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ > >> [v1]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ > >> [v2]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ > >> [v3]: > >>
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
> On Jun 2, 2020, at 12:38 AM, Prabhakar Kushwaha > wrote: > > On Tue, Jun 2, 2020 at 3:29 AM John Donnelly > wrote: >> >> Hi . See below ! >> >>> On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: >>> >>> Hi John, >>> >>> On Tue, Jun 2, 2020 at 1:01 AM John Donnelly >>> wrote: Hi, On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > Hi Chen, > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: >> This patch series enable reserving crashkernel above 4G in arm64. >> >> There are following issues in arm64 kdump: >> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail >> when there is no enough low memory. >> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above >> 4G, >> in this case, if swiotlb or DMA buffers are required, crash dump kernel >> will boot failure because there is no low memory available for >> allocation. >> >> To solve these issues, introduce crashkernel=X,low to reserve specified >> size low memory. >> Crashkernel=X tries to reserve memory for the crash dump kernel under >> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified >> size low memory for crash kdump kernel devices firstly and then reserve >> memory above 4G. >> >> When crashkernel is reserved above 4G in memory, that is, >> crashkernel=X,low >> is specified simultaneously, kernel should reserve specified size low >> memory >> for crash dump kernel devices. So there may be two crash kernel regions, >> one >> is below 4G, the other is above 4G. >> In order to distinct from the high region and make no effect to the use >> of >> kexec-tools, rename the low region as "Crash kernel (low)", and add DT >> property >> "linux,low-memory-range" to crash dump kernel's dtb to pass the low >> region. >> >> Besides, we need to modify kexec-tools: >> arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) >> >> The previous changes and discussions can be retrieved from: >> >> Changes since [v7] >> - Move x86 CRASH_ALIGN to 2M >> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. >> - Update Documentation/devicetree/bindings/chosen.txt >> Add corresponding documentation to >> Documentation/devicetree/bindings/chosen.txt suggested by Arnd. >> - Add Tested-by from Jhon and pk >> >> Changes since [v6] >> - Fix build errors reported by kbuild test robot. >> >> Changes since [v5] >> - Move reserve_crashkernel_low() into kernel/crash_core.c. >> - Delete crashkernel=X,high. >> - Modify crashkernel=X,low. >> If crashkernel=X,low is specified simultaneously, reserve spcified size >> low >> memory for crash kdump kernel devices firstly and then reserve memory >> above 4G. >> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, >> and then >> pass to crash dump kernel by DT property "linux,low-memory-range". >> - Update Documentation/admin-guide/kdump/kdump.rst. >> >> Changes since [v4] >> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. >> >> Changes since [v3] >> - Add memblock_cap_memory_ranges back for multiple ranges. >> - Fix some compiling warnings. >> >> Changes since [v2] >> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as >> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate >> patch. >> >> Changes since [v1]: >> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. >> - Remove memblock_cap_memory_ranges() i added in v1 and implement that >> in fdt_enforce_memory_region(). >> There are at most two crash kernel regions, for two crash kernel regions >> case, we cap the memory range [min(regs[*].start), max(regs[*].end)] >> and then remove the memory range in the middle. >> >> [1]: >> https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ >> [v1]: >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ >> [v2]: >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ >> [v3]: >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbnc4zPPV$ >> [v4]: >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvsAsZLu$ >> [v5]: >>
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On Tue, Jun 2, 2020 at 3:29 AM John Donnelly wrote: > > Hi . See below ! > > > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > > > > Hi John, > > > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > > wrote: > >> > >> Hi, > >> > >> > >> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > >>> Hi Chen, > >>> > >>> On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: > This patch series enable reserving crashkernel above 4G in arm64. > > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail > when there is no enough low memory. > 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above > 4G, > in this case, if swiotlb or DMA buffers are required, crash dump kernel > will boot failure because there is no low memory available for > allocation. > > To solve these issues, introduce crashkernel=X,low to reserve specified > size low memory. > Crashkernel=X tries to reserve memory for the crash dump kernel under > 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified > size low memory for crash kdump kernel devices firstly and then reserve > memory above 4G. > > When crashkernel is reserved above 4G in memory, that is, > crashkernel=X,low > is specified simultaneously, kernel should reserve specified size low > memory > for crash dump kernel devices. So there may be two crash kernel regions, > one > is below 4G, the other is above 4G. > In order to distinct from the high region and make no effect to the use > of > kexec-tools, rename the low region as "Crash kernel (low)", and add DT > property > "linux,low-memory-range" to crash dump kernel's dtb to pass the low > region. > > Besides, we need to modify kexec-tools: > arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) > > The previous changes and discussions can be retrieved from: > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > - Update Documentation/devicetree/bindings/chosen.txt > Add corresponding documentation to > Documentation/devicetree/bindings/chosen.txt suggested by Arnd. > - Add Tested-by from Jhon and pk > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size > low > memory for crash kdump kernel devices firstly and then reserve memory > above 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, > and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_cap_memory_ranges back for multiple ranges. > - Fix some compiling warnings. > > Changes since [v2] > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > patch. > > Changes since [v1]: > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > in fdt_enforce_memory_region(). > There are at most two crash kernel regions, for two crash kernel regions > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > and then remove the memory range in the middle. > > [1]: > https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ > [v1]: > https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ > [v2]: > https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ > [v3]: > https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbnc4zPPV$ > [v4]: > https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvsAsZLu$ > [v5]: >
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi . See below ! > On Jun 1, 2020, at 4:02 PM, Bhupesh Sharma wrote: > > Hi John, > > On Tue, Jun 2, 2020 at 1:01 AM John Donnelly > wrote: >> >> Hi, >> >> >> On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: >>> Hi Chen, >>> >>> On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. To solve these issues, introduce crashkernel=X,low to reserve specified size low memory. Crashkernel=X tries to reserve memory for the crash dump kernel under 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low is specified simultaneously, kernel should reserve specified size low memory for crash dump kernel devices. So there may be two crash kernel regions, one is below 4G, the other is above 4G. In order to distinct from the high region and make no effect to the use of kexec-tools, rename the low region as "Crash kernel (low)", and add DT property "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. Besides, we need to modify kexec-tools: arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) The previous changes and discussions can be retrieved from: Changes since [v7] - Move x86 CRASH_ALIGN to 2M Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. - Update Documentation/devicetree/bindings/chosen.txt Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd. - Add Tested-by from Jhon and pk Changes since [v6] - Fix build errors reported by kbuild test robot. Changes since [v5] - Move reserve_crashkernel_low() into kernel/crash_core.c. - Delete crashkernel=X,high. - Modify crashkernel=X,low. If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range". - Update Documentation/admin-guide/kdump/kdump.rst. Changes since [v4] - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. Changes since [v3] - Add memblock_cap_memory_ranges back for multiple ranges. - Fix some compiling warnings. Changes since [v2] - Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch. Changes since [v1]: - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. - Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region(). There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle. [1]: https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ [v1]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ [v2]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ [v3]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbnc4zPPV$ [v4]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvsAsZLu$ [v5]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbl24n-79$ [v6]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbs7r8G2a$ [v7]:
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi John, On Tue, Jun 2, 2020 at 1:01 AM John Donnelly wrote: > > Hi, > > > On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: > > Hi Chen, > > > > On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: > >> This patch series enable reserving crashkernel above 4G in arm64. > >> > >> There are following issues in arm64 kdump: > >> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail > >> when there is no enough low memory. > >> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, > >> in this case, if swiotlb or DMA buffers are required, crash dump kernel > >> will boot failure because there is no low memory available for allocation. > >> > >> To solve these issues, introduce crashkernel=X,low to reserve specified > >> size low memory. > >> Crashkernel=X tries to reserve memory for the crash dump kernel under > >> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified > >> size low memory for crash kdump kernel devices firstly and then reserve > >> memory above 4G. > >> > >> When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low > >> is specified simultaneously, kernel should reserve specified size low > >> memory > >> for crash dump kernel devices. So there may be two crash kernel regions, > >> one > >> is below 4G, the other is above 4G. > >> In order to distinct from the high region and make no effect to the use of > >> kexec-tools, rename the low region as "Crash kernel (low)", and add DT > >> property > >> "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. > >> > >> Besides, we need to modify kexec-tools: > >> arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) > >> > >> The previous changes and discussions can be retrieved from: > >> > >> Changes since [v7] > >> - Move x86 CRASH_ALIGN to 2M > >> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > >> - Update Documentation/devicetree/bindings/chosen.txt > >> Add corresponding documentation to > >> Documentation/devicetree/bindings/chosen.txt suggested by Arnd. > >> - Add Tested-by from Jhon and pk > >> > >> Changes since [v6] > >> - Fix build errors reported by kbuild test robot. > >> > >> Changes since [v5] > >> - Move reserve_crashkernel_low() into kernel/crash_core.c. > >> - Delete crashkernel=X,high. > >> - Modify crashkernel=X,low. > >> If crashkernel=X,low is specified simultaneously, reserve spcified size low > >> memory for crash kdump kernel devices firstly and then reserve memory > >> above 4G. > >> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and > >> then > >> pass to crash dump kernel by DT property "linux,low-memory-range". > >> - Update Documentation/admin-guide/kdump/kdump.rst. > >> > >> Changes since [v4] > >> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > >> > >> Changes since [v3] > >> - Add memblock_cap_memory_ranges back for multiple ranges. > >> - Fix some compiling warnings. > >> > >> Changes since [v2] > >> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > >> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > >> patch. > >> > >> Changes since [v1]: > >> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > >> - Remove memblock_cap_memory_ranges() i added in v1 and implement that > >> in fdt_enforce_memory_region(). > >> There are at most two crash kernel regions, for two crash kernel regions > >> case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > >> and then remove the memory range in the middle. > >> > >> [1]: > >> https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ > >> [v1]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ > >> [v2]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ > >> [v3]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbnc4zPPV$ > >> [v4]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvsAsZLu$ > >> [v5]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbl24n-79$ > >> [v6]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbs7r8G2a$ > >> [v7]: > >> https://urldefense.com/v3/__https://lkml.org/lkml/2019/12/23/411__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiFUH90G$ > >> > >> Chen
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi, On 6/1/20 7:02 AM, Prabhakar Kushwaha wrote: Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. To solve these issues, introduce crashkernel=X,low to reserve specified size low memory. Crashkernel=X tries to reserve memory for the crash dump kernel under 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low is specified simultaneously, kernel should reserve specified size low memory for crash dump kernel devices. So there may be two crash kernel regions, one is below 4G, the other is above 4G. In order to distinct from the high region and make no effect to the use of kexec-tools, rename the low region as "Crash kernel (low)", and add DT property "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. Besides, we need to modify kexec-tools: arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) The previous changes and discussions can be retrieved from: Changes since [v7] - Move x86 CRASH_ALIGN to 2M Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. - Update Documentation/devicetree/bindings/chosen.txt Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd. - Add Tested-by from Jhon and pk Changes since [v6] - Fix build errors reported by kbuild test robot. Changes since [v5] - Move reserve_crashkernel_low() into kernel/crash_core.c. - Delete crashkernel=X,high. - Modify crashkernel=X,low. If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range". - Update Documentation/admin-guide/kdump/kdump.rst. Changes since [v4] - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. Changes since [v3] - Add memblock_cap_memory_ranges back for multiple ranges. - Fix some compiling warnings. Changes since [v2] - Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch. Changes since [v1]: - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. - Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region(). There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle. [1]: https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvpn1uM1$ [v1]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbt0xN9PE$ [v2]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbub7yUQH$ [v3]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbnc4zPPV$ [v4]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbvsAsZLu$ [v5]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbl24n-79$ [v6]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbs7r8G2a$ [v7]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/12/23/411__;!!GqivPVa7Brio!LnTSARkCt0V0FozR0KmqooaH5ADtdXvs3mPdP3KRVqALmvSK2VmCkIPIhsaxbiFUH90G$ Chen Zhou (5): x86: kdump: move reserve_crashkernel_low() into crash_core.c arm64: kdump: reserve crashkenel above 4G for crash dump kernel arm64: kdump: add memory for devices by DT property, low-memory-range kdump: update Documentation about crashkernel on arm64 dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi Chen, On Thu, May 21, 2020 at 3:05 PM Chen Zhou wrote: > > This patch series enable reserving crashkernel above 4G in arm64. > > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail > when there is no enough low memory. > 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, > in this case, if swiotlb or DMA buffers are required, crash dump kernel > will boot failure because there is no low memory available for allocation. > > To solve these issues, introduce crashkernel=X,low to reserve specified > size low memory. > Crashkernel=X tries to reserve memory for the crash dump kernel under > 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified > size low memory for crash kdump kernel devices firstly and then reserve > memory above 4G. > > When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low > is specified simultaneously, kernel should reserve specified size low memory > for crash dump kernel devices. So there may be two crash kernel regions, one > is below 4G, the other is above 4G. > In order to distinct from the high region and make no effect to the use of > kexec-tools, rename the low region as "Crash kernel (low)", and add DT > property > "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. > > Besides, we need to modify kexec-tools: > arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) > > The previous changes and discussions can be retrieved from: > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > - Update Documentation/devicetree/bindings/chosen.txt > Add corresponding documentation to > Documentation/devicetree/bindings/chosen.txt suggested by Arnd. > - Add Tested-by from Jhon and pk > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size low > memory for crash kdump kernel devices firstly and then reserve memory above > 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_cap_memory_ranges back for multiple ranges. > - Fix some compiling warnings. > > Changes since [v2] > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > patch. > > Changes since [v1]: > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > in fdt_enforce_memory_region(). > There are at most two crash kernel regions, for two crash kernel regions > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > and then remove the memory range in the middle. > > [1]: http://lists.infradead.org/pipermail/kexec/2020-May/025128.html > [v1]: https://lkml.org/lkml/2019/4/2/1174 > [v2]: https://lkml.org/lkml/2019/4/9/86 > [v3]: https://lkml.org/lkml/2019/4/9/306 > [v4]: https://lkml.org/lkml/2019/4/15/273 > [v5]: https://lkml.org/lkml/2019/5/6/1360 > [v6]: https://lkml.org/lkml/2019/8/30/142 > [v7]: https://lkml.org/lkml/2019/12/23/411 > > Chen Zhou (5): > x86: kdump: move reserve_crashkernel_low() into crash_core.c > arm64: kdump: reserve crashkenel above 4G for crash dump kernel > arm64: kdump: add memory for devices by DT property, low-memory-range > kdump: update Documentation about crashkernel on arm64 > dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump > We are getting "warn_alloc" [1] warning during boot of kdump kernel with bootargs as [2] of primary kernel. This error observed on ThunderX2 ARM64 platform. It is observed with latest upstream tag (v5.7-rc3) with this patch set and https://lists.infradead.org/pipermail/kexec/2020-May/025128.html Also **without** this patch-set "https://www.spinics.net/lists/arm-kernel/msg806882.html; This issue comes whenever crashkernel memory is reserved after 0xc000_. More details discussed earlier in https://www.spinics.net/lists/arm-kernel/msg806882.html without any solution This patch-set is expected to solve similar kind of issue. i.e. low memory is only targeted for DMA, swiotlb; So above mentioned observation should be considered/fixed. . --pk [1] [ 30.366695] DMI: Cavium Inc. Saber/Saber, BIOS TX2-FW-Release-3.1-build_01-2803-g74253a541a mm/dd/ [ 30.367696] NET: Registered protocol family 16 [ 30.369973] swapper/0: page allocation failure: order:6, mode:0x1(GFP_DMA),
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
[+James Morse] On Thu, May 28, 2020 at 05:20:34PM -0500, John Donnelly wrote: > On 5/25/20 8:42 PM, Baoquan He wrote: > > On 05/21/20 at 05:38pm, Chen Zhou wrote: > > > This patch series enable reserving crashkernel above 4G in arm64. [...] > > > Chen Zhou (5): > > >x86: kdump: move reserve_crashkernel_low() into crash_core.c > > >arm64: kdump: reserve crashkenel above 4G for crash dump kernel > > >arm64: kdump: add memory for devices by DT property, low-memory-range > > >kdump: update Documentation about crashkernel on arm64 > > >dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump > > > > > > Documentation/admin-guide/kdump/kdump.rst | 13 ++- > > > .../admin-guide/kernel-parameters.txt | 12 ++- > > > Documentation/devicetree/bindings/chosen.txt | 25 ++ > > > arch/arm64/kernel/setup.c | 8 +- > > > arch/arm64/mm/init.c | 61 - > > > arch/x86/kernel/setup.c | 66 ++ > > > include/linux/crash_core.h| 3 + > > > include/linux/kexec.h | 2 - > > > kernel/crash_core.c | 85 +++ > > > kernel/kexec_core.c | 17 > > > 10 files changed, 208 insertions(+), 84 deletions(-) > > > > This proposal to improve vmcore creation on Arm has been going on for > almost a year now. > > Who is the final maintainer that needs to approve and except these ? > > What are the lingering issues that are remaining so we get these accepted > into a upstream commit ? The arm64 bits need an Ack from James Morse, but he's not on CC despite offering feedback on earlier versions. Will
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 5/25/20 8:42 PM, Baoquan He wrote: On 05/21/20 at 05:38pm, Chen Zhou wrote: This patch series enable reserving crashkernel above 4G in arm64. There are following issues in arm64 kdump: 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory. 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, in this case, if swiotlb or DMA buffers are required, crash dump kernel will boot failure because there is no low memory available for allocation. To solve these issues, introduce crashkernel=X,low to reserve specified size low memory. Crashkernel=X tries to reserve memory for the crash dump kernel under 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low is specified simultaneously, kernel should reserve specified size low memory for crash dump kernel devices. So there may be two crash kernel regions, one is below 4G, the other is above 4G. In order to distinct from the high region and make no effect to the use of kexec-tools, rename the low region as "Crash kernel (low)", and add DT property "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. Besides, we need to modify kexec-tools: arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) The previous changes and discussions can be retrieved from: Changes since [v7] - Move x86 CRASH_ALIGN to 2M Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. OK, moving x86 CRASH_ALIGN to 2M is suggested by Dave. Because CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M. So 2M seems good. But, anyway, we should tell the reason why it need be changed in commit log. arch/x86/Kconfig: config PHYSICAL_ALIGN hex "Alignment value to which kernel should be aligned" default "0x20" range 0x2000 0x100 if X86_32 range 0x20 0x100 if X86_64 - Update Documentation/devicetree/bindings/chosen.txt Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd. - Add Tested-by from Jhon and pk Changes since [v6] - Fix build errors reported by kbuild test robot. Changes since [v5] - Move reserve_crashkernel_low() into kernel/crash_core.c. - Delete crashkernel=X,high. And the crashkernel=X,high being deleted need be told too. Otherwise people reading the commit have to check why themselves. I didn't follow the old version, can't see why ,high can't be specified explicitly. - Modify crashkernel=X,low. If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range". - Update Documentation/admin-guide/kdump/kdump.rst. Changes since [v4] - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. Changes since [v3] - Add memblock_cap_memory_ranges back for multiple ranges. - Fix some compiling warnings. Changes since [v2] - Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch. Changes since [v1]: - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. - Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region(). There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle. [1]: https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-May/025128.html__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJwQs3C4x$ [v1]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJ6e-mIEp$ [v2]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJyUVjUta$ [v3]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJ3CXBRdT$ [v4]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJ7SxW1Vj$ [v5]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJ2wyJ9tj$ [v6]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!NHQIQVbVz5bR1SSP7U7SwT3uHb6OnycPGa6nM0oLTaQdZT4pjRsjrMjn5GqOJzvGhWBh$ [v7]:
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
Hi Baoquan, Thanks for your suggestions. You are right, some details should be made in the commit log. Thanks, Chen Zhou On 2020/5/26 9:42, Baoquan He wrote: > On 05/21/20 at 05:38pm, Chen Zhou wrote: >> This patch series enable reserving crashkernel above 4G in arm64. >> >> There are following issues in arm64 kdump: >> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail >> when there is no enough low memory. >> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, >> in this case, if swiotlb or DMA buffers are required, crash dump kernel >> will boot failure because there is no low memory available for allocation. >> >> To solve these issues, introduce crashkernel=X,low to reserve specified >> size low memory. >> Crashkernel=X tries to reserve memory for the crash dump kernel under >> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified >> size low memory for crash kdump kernel devices firstly and then reserve >> memory above 4G. >> >> When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low >> is specified simultaneously, kernel should reserve specified size low memory >> for crash dump kernel devices. So there may be two crash kernel regions, one >> is below 4G, the other is above 4G. >> In order to distinct from the high region and make no effect to the use of >> kexec-tools, rename the low region as "Crash kernel (low)", and add DT >> property >> "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. >> >> Besides, we need to modify kexec-tools: >> arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) >> >> The previous changes and discussions can be retrieved from: >> >> Changes since [v7] >> - Move x86 CRASH_ALIGN to 2M >> Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. > OK, moving x86 CRASH_ALIGN to 2M is suggested by Dave. Because > CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M. So 2M seems good. > But, anyway, we should tell the reason why it need be changed in commit > log. > > > arch/x86/Kconfig: > config PHYSICAL_ALIGN > hex "Alignment value to which kernel should be aligned" > default "0x20" > range 0x2000 0x100 if X86_32 > range 0x20 0x100 if X86_64 > >> - Update Documentation/devicetree/bindings/chosen.txt >> Add corresponding documentation to >> Documentation/devicetree/bindings/chosen.txt suggested by Arnd. >> - Add Tested-by from Jhon and pk >> >> Changes since [v6] >> - Fix build errors reported by kbuild test robot. >> >> Changes since [v5] >> - Move reserve_crashkernel_low() into kernel/crash_core.c. >> - Delete crashkernel=X,high. > And the crashkernel=X,high being deleted need be told too. Otherwise > people reading the commit have to check why themselves. I didn't follow > the old version, can't see why ,high can't be specified explicitly. > >> - Modify crashkernel=X,low. >> If crashkernel=X,low is specified simultaneously, reserve spcified size low >> memory for crash kdump kernel devices firstly and then reserve memory above >> 4G. >> In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and >> then >> pass to crash dump kernel by DT property "linux,low-memory-range". >> - Update Documentation/admin-guide/kdump/kdump.rst. >> >> Changes since [v4] >> - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. >> >> Changes since [v3] >> - Add memblock_cap_memory_ranges back for multiple ranges. >> - Fix some compiling warnings. >> >> Changes since [v2] >> - Split patch "arm64: kdump: support reserving crashkernel above 4G" as >> two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate >> patch. >> >> Changes since [v1]: >> - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. >> - Remove memblock_cap_memory_ranges() i added in v1 and implement that >> in fdt_enforce_memory_region(). >> There are at most two crash kernel regions, for two crash kernel regions >> case, we cap the memory range [min(regs[*].start), max(regs[*].end)] >> and then remove the memory range in the middle. >> >> [1]: http://lists.infradead.org/pipermail/kexec/2020-May/025128.html >> [v1]: https://lkml.org/lkml/2019/4/2/1174 >> [v2]: https://lkml.org/lkml/2019/4/9/86 >> [v3]: https://lkml.org/lkml/2019/4/9/306 >> [v4]: https://lkml.org/lkml/2019/4/15/273 >> [v5]: https://lkml.org/lkml/2019/5/6/1360 >> [v6]: https://lkml.org/lkml/2019/8/30/142 >> [v7]: https://lkml.org/lkml/2019/12/23/411 >> >> Chen Zhou (5): >> x86: kdump: move reserve_crashkernel_low() into crash_core.c >> arm64: kdump: reserve crashkenel above 4G for crash dump kernel >> arm64: kdump: add memory for devices by DT property, low-memory-range >> kdump: update Documentation about crashkernel on arm64 >> dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump >> >> Documentation/admin-guide/kdump/kdump.rst | 13 ++- >> .../admin-guide/kernel-parameters.txt
Re: [PATCH v8 0/5] support reserving crashkernel above 4G on arm64 kdump
On 05/21/20 at 05:38pm, Chen Zhou wrote: > This patch series enable reserving crashkernel above 4G in arm64. > > There are following issues in arm64 kdump: > 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail > when there is no enough low memory. > 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G, > in this case, if swiotlb or DMA buffers are required, crash dump kernel > will boot failure because there is no low memory available for allocation. > > To solve these issues, introduce crashkernel=X,low to reserve specified > size low memory. > Crashkernel=X tries to reserve memory for the crash dump kernel under > 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified > size low memory for crash kdump kernel devices firstly and then reserve > memory above 4G. > > When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low > is specified simultaneously, kernel should reserve specified size low memory > for crash dump kernel devices. So there may be two crash kernel regions, one > is below 4G, the other is above 4G. > In order to distinct from the high region and make no effect to the use of > kexec-tools, rename the low region as "Crash kernel (low)", and add DT > property > "linux,low-memory-range" to crash dump kernel's dtb to pass the low region. > > Besides, we need to modify kexec-tools: > arm64: kdump: add another DT property to crash dump kernel's dtb(see [1]) > > The previous changes and discussions can be retrieved from: > > Changes since [v7] > - Move x86 CRASH_ALIGN to 2M > Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M. OK, moving x86 CRASH_ALIGN to 2M is suggested by Dave. Because CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M. So 2M seems good. But, anyway, we should tell the reason why it need be changed in commit log. arch/x86/Kconfig: config PHYSICAL_ALIGN hex "Alignment value to which kernel should be aligned" default "0x20" range 0x2000 0x100 if X86_32 range 0x20 0x100 if X86_64 > - Update Documentation/devicetree/bindings/chosen.txt > Add corresponding documentation to > Documentation/devicetree/bindings/chosen.txt suggested by Arnd. > - Add Tested-by from Jhon and pk > > Changes since [v6] > - Fix build errors reported by kbuild test robot. > > Changes since [v5] > - Move reserve_crashkernel_low() into kernel/crash_core.c. > - Delete crashkernel=X,high. And the crashkernel=X,high being deleted need be told too. Otherwise people reading the commit have to check why themselves. I didn't follow the old version, can't see why ,high can't be specified explicitly. > - Modify crashkernel=X,low. > If crashkernel=X,low is specified simultaneously, reserve spcified size low > memory for crash kdump kernel devices firstly and then reserve memory above > 4G. > In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then > pass to crash dump kernel by DT property "linux,low-memory-range". > - Update Documentation/admin-guide/kdump/kdump.rst. > > Changes since [v4] > - Reimplement memblock_cap_memory_ranges for multiple ranges by Mike. > > Changes since [v3] > - Add memblock_cap_memory_ranges back for multiple ranges. > - Fix some compiling warnings. > > Changes since [v2] > - Split patch "arm64: kdump: support reserving crashkernel above 4G" as > two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate > patch. > > Changes since [v1]: > - Move common reserve_crashkernel_low() code into kernel/kexec_core.c. > - Remove memblock_cap_memory_ranges() i added in v1 and implement that > in fdt_enforce_memory_region(). > There are at most two crash kernel regions, for two crash kernel regions > case, we cap the memory range [min(regs[*].start), max(regs[*].end)] > and then remove the memory range in the middle. > > [1]: http://lists.infradead.org/pipermail/kexec/2020-May/025128.html > [v1]: https://lkml.org/lkml/2019/4/2/1174 > [v2]: https://lkml.org/lkml/2019/4/9/86 > [v3]: https://lkml.org/lkml/2019/4/9/306 > [v4]: https://lkml.org/lkml/2019/4/15/273 > [v5]: https://lkml.org/lkml/2019/5/6/1360 > [v6]: https://lkml.org/lkml/2019/8/30/142 > [v7]: https://lkml.org/lkml/2019/12/23/411 > > Chen Zhou (5): > x86: kdump: move reserve_crashkernel_low() into crash_core.c > arm64: kdump: reserve crashkenel above 4G for crash dump kernel > arm64: kdump: add memory for devices by DT property, low-memory-range > kdump: update Documentation about crashkernel on arm64 > dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump > > Documentation/admin-guide/kdump/kdump.rst | 13 ++- > .../admin-guide/kernel-parameters.txt | 12 ++- > Documentation/devicetree/bindings/chosen.txt | 25 ++ > arch/arm64/kernel/setup.c | 8 +- > arch/arm64/mm/init.c | 61 - > arch/x86/kernel/setup.c | 66 ++ >
