RE: [PATCH v2 0/6] arm/arm64: Allow the rescheduling IPI to bypass irq_enter/exit
Hi Marc, I plan to add NMI patches which enables IPI_CPU_CRASH_STOP IPI as pseudo-NMI[1]. But I know need to resolve the instrumentation issues before that. I think need to moving arm64 entry code over to the generic entry code(kernel/entry/common.c) for that, is this right? Can you tell me current status? Let me know if there's anything I can do to help. [1]https://lore.kernel.org/lkml/20201104080539.3205889-1-ito-yui...@fujitsu.com/ Thanks, Yuichi Ito
RE: [PATCH v7 2/7] irqchip/gic-v3: Enable support for SGIs to act as NMIs
Hi Sumit,
> Subject: [PATCH v7 2/7] irqchip/gic-v3: Enable support for SGIs to act as
> NMIs
>
> Add support to handle SGIs as pseudo NMIs. As SGIs or IPIs default to a
> special flow handler: handle_percpu_devid_fasteoi_ipi(), so skip NMI handler
> update in case of SGIs.
>
> Also, enable NMI support prior to gic_smp_init() as allocation of SGIs as
> IRQs/NMIs happen as part of this routine.
>
> Signed-off-by: Sumit Garg
> ---
> drivers/irqchip/irq-gic-v3.c | 29 +
> 1 file changed, 21 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index
> 16fecc0..7010ae2 100644
> --- a/drivers/irqchip/irq-gic-v3.c
> +++ b/drivers/irqchip/irq-gic-v3.c
> @@ -461,6 +461,7 @@ static u32 gic_get_ppi_index(struct irq_data *d)
> static int gic_irq_nmi_setup(struct irq_data *d) {
> struct irq_desc *desc = irq_to_desc(d->irq);
> + u32 idx;
>
> if (!gic_supports_nmi())
> return -EINVAL;
> @@ -478,16 +479,22 @@ static int gic_irq_nmi_setup(struct irq_data *d)
> return -EINVAL;
>
> /* desc lock should already be held */
> - if (gic_irq_in_rdist(d)) {
> - u32 idx = gic_get_ppi_index(d);
> + switch (get_intid_range(d)) {
> + case SGI_RANGE:
> + break;
> + case PPI_RANGE:
> + case EPPI_RANGE:
> + idx = gic_get_ppi_index(d);
>
> /* Setting up PPI as NMI, only switch handler for first NMI */
> if (!refcount_inc_not_zero(_nmi_refs[idx])) {
> refcount_set(_nmi_refs[idx], 1);
> desc->handle_irq =
> handle_percpu_devid_fasteoi_nmi;
> }
> - } else {
> + break;
> + default:
> desc->handle_irq = handle_fasteoi_nmi;
> + break;
> }
>
> gic_irq_set_prio(d, GICD_INT_NMI_PRI); @@ -498,6 +505,7 @@
> static int gic_irq_nmi_setup(struct irq_data *d) static void
> gic_irq_nmi_teardown(struct irq_data *d) {
> struct irq_desc *desc = irq_to_desc(d->irq);
> + u32 idx;
>
> if (WARN_ON(!gic_supports_nmi()))
> return;
> @@ -515,14 +523,20 @@ static void gic_irq_nmi_teardown(struct irq_data *d)
> return;
>
> /* desc lock should already be held */
> - if (gic_irq_in_rdist(d)) {
> - u32 idx = gic_get_ppi_index(d);
> + switch (get_intid_range(d)) {
> + case SGI_RANGE:
> + break;
> + case PPI_RANGE:
> + case EPPI_RANGE:
> + idx = gic_get_ppi_index(d);
>
> /* Tearing down NMI, only switch handler for last NMI */
> if (refcount_dec_and_test(_nmi_refs[idx]))
> desc->handle_irq = handle_percpu_devid_irq;
> - } else {
> + break;
> + default:
> desc->handle_irq = handle_fasteoi_irq;
> + break;
> }
>
> gic_irq_set_prio(d, GICD_INT_DEF_PRI); @@ -1708,6 +1722,7 @@
> static int __init gic_init_bases(void __iomem *dist_base,
>
> gic_dist_init();
> gic_cpu_init();
> + gic_enable_nmi_support();
> gic_smp_init();
> gic_cpu_pm_init();
>
> @@ -1719,8 +1734,6 @@ static int __init gic_init_bases(void __iomem
> *dist_base,
> gicv2m_init(handle, gic_data.domain);
> }
>
> - gic_enable_nmi_support();
> -
> return 0;
>
> out_free:
> --
> 2.7.4
I checked for this patch and I think that's good.
I've tested this patch with FX1000 used my other patches.
The result is as follows.
$ echo 1 > /proc/sys/kernel/panic_on_rcu_stal
$ echo HARDLOCKUP > /sys/kernel/debug/provoke-crash/DIRECT
: kernel panics and crash kernel boot
: makedumpfile saves the system state at HARDLOCKUP in vmcore.
crash utility:
#7 [fe0029e4fd30] lkdtm_HARDLOCKUP at fe0010856eec
#8 [fe0029e4fd40] direct_entry at fe0010856c94
#9 [fe0029e4fd90] full_proxy_write at fe001055ced0
#10 [fe0029e4fdd0] vfs_write at fe001047436c
#11 [fe0029e4fe10] ksys_write at fe001047466c
#12 [fe0029e4fe60] __arm64_sys_write at fe0010474718
#13 [fe0029e4fe70] do_el0_svc at fe00101590cc
#14 [fe0029e4fea0] el0_svc at fe0010147a30
#15 [fe0029e4feb0] el0_sync_handler at fe001014835c
#16 [fe0029e4fff0] el0_sync at fe0010142c14
Best regards,
Yuichi Ito
RE: [PATCH v2 0/3] Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI
Hi Marc, Sumit What should I do to merge this patch. I would appreciate if you have any advice. I have not tested it with ThunderX2 yet. Best regards, Yuichi Ito > -Original Message- > From: Yuichi Ito > Sent: Wednesday, November 4, 2020 5:06 PM > To: m...@kernel.org; sumit.g...@linaro.org; t...@linutronix.de; > ja...@lakedaemon.net; catalin.mari...@arm.com; w...@kernel.org > Cc: linux-arm-ker...@lists.infradead.org; linux-kernel@vger.kernel.org; Ito, > Yuichi/伊藤 有一 > Subject: [PATCH v2 0/3] Enable support IPI_CPU_CRASH_STOP to be > pseudo-NMI > > This patchset enables IPI_CPU_CRASH_STOP IPI to be pseudo-NMI. > This allows kdump to collect system information even when the CPU is in a > HARDLOCKUP state. > > Only IPI_CPU_CRASH_STOP uses NMI and the other IPIs remain normal > IRQs. > > The patch has been tested on FX1000. > > It also uses some of Sumit's IPI patch set for NMI.[1] > > [1] > https://lore.kernel.org/lkml/1603983387-8738-3-git-send-email-sumit.garg@l > inaro.org/ > > $ echo 1 > /proc/sys/kernel/panic_on_rcu_stal > $ echo HARDLOCKUP > /sys/kernel/debug/provoke-crash/DIRECT >: kernel panics and crash kernel boot >: makedumpfile saves the system state at HARDLOCKUP in vmcore. > > crash utility: > #7 [fe00290afd30] lkdtm_HARDLOCKUP at fe0010857ee8 > #8 [fe00290afd40] direct_entry at fe0010857c94 > #9 [fe00290afd90] full_proxy_write at fe001055dea0 > #10 [fe00290afdd0] vfs_write at fe001047533c > #11 [fe00290afe10] ksys_write at fe001047563c > #12 [fe00290afe60] __arm64_sys_write at fe00104756e8 > #13 [fe00290afe70] do_el0_svc at fe00101590cc > #14 [fe00290afea0] el0_svc at fe0010147a30 > #15 [fe00290afeb0] el0_sync_handler at fe001014835c > #16 [fe00290afff0] el0_sync at fe0010142c14 > > Changes in v1: > - Rebased to head of upstream master. > - Rebased to Sumit's latest IPIs patch-set [1]. > > [1] > https://lore.kernel.org/lkml/1603983387-8738-3-git-send-email-sumit.garg@l > inaro.org/ > > - Add conditional branch of local_irq_disable(). > > Sumit Garg (1): > irqchip/gic-v3: Enable support for SGIs to act as NMIs > > Yuichi Ito (2): > arch64: smp: Register IPI_CPU_CRASH_STOP IPI as pseudo-NMI > arch64: smp: Disable priority masking when received NMI on PSR.I section > > arch/arm64/kernel/smp.c | 44 > +++- > drivers/irqchip/irq-gic-v3.c | 29 + > 2 files changed, 56 insertions(+), 17 deletions(-) > > -- > 1.8.3.1
RE: [PATCH 0/2] Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI
Hi Marc > > On 2020-09-29 06:50, ito-yui...@fujitsu.com wrote: > > Hi Marc > > [...] > > >> >> The patch has been tested on ThunderX. > >> > >> Which ThunderX? TX2 (at least the incarnation I used in the past) > >> wasn't able > >> to correctly deal with priorities. > > > > I tried it with ThunderX CN8890. > > If you tell me steps to reproduce the problem of TX2, I will > > investigate it with TX as well. > > PMR_EL1 reporting fantasy values, non-uniform priority support across > the interrupt classes, and generally prone to lockups. The original TX > is a very different machine though (TX 1 and 2 only share the engraving > of the manufacturer on the heat-spreader). Thank you for the information. I will check if we have a ThunderX1 or X2 environment. If we have either one, I will investigate it. > M. > -- > Jazz is not dead. It just smells funny... Thank you and best regards, Yuichi Ito
RE: [PATCH 0/2] Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI
Hi Marc Thank you for your reply. > On 2020-09-28 03:43, ito-yui...@fujitsu.com wrote: > > Hi Marc, Sumit > > > > I would appreciate if you have any advice on this patch. > > I haven't had a chance to look into it, as I'm not even sure I'll take the > core > series in the first place (there are outstanding regressions I can't > reproduce, > let alone fix them). > I understand it. Please let me know if there is anything I can do. I sincerely hope that your patches will be merged into the mainline. > > > > Yuichi Ito > > > >> Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI > >> > >> This patchset enables IPI_CPU_CRASH_STOP IPI to be pseudo-NMI. > >> This allows kdump to collect system information even when the CPU is > >> in a HARDLOCKUP state. > >> > >> Only IPI_CPU_CRASH_STOP uses NMI and the other IPIs remain normal > >> IRQs. > >> > >> The patch has been tested on ThunderX. > > Which ThunderX? TX2 (at least the incarnation I used in the past) wasn't able > to correctly deal with priorities. I tried it with ThunderX CN8890. If you tell me steps to reproduce the problem of TX2, I will investigate it with TX as well. > M. > -- > Jazz is not dead. It just smells funny... Thank you and best regards, Yuichi Ito
RE: [PATCH 0/2] Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI
Hi Marc, Sumit I would appreciate if you have any advice on this patch. Yuichi Ito > -Original Message- > From: Yuichi Ito > Sent: Thursday, September 24, 2020 1:43 PM > To: m...@kernel.org; sumit.g...@linaro.org; t...@linutronix.de; > ja...@lakedaemon.net; catalin.mari...@arm.com; w...@kernel.org > Cc: linux-arm-ker...@lists.infradead.org; linux-kernel@vger.kernel.org; Ito, > Yuichi/伊藤 有一 > Subject: [PATCH 0/2] Enable support IPI_CPU_CRASH_STOP to be > pseudo-NMI > > Enable support IPI_CPU_CRASH_STOP to be pseudo-NMI > > This patchset enables IPI_CPU_CRASH_STOP IPI to be pseudo-NMI. > This allows kdump to collect system information even when the CPU is in a > HARDLOCKUP state. > > Only IPI_CPU_CRASH_STOP uses NMI and the other IPIs remain normal > IRQs. > > The patch has been tested on ThunderX. > > This patch assumes Marc's latest IPIs patch-set. [1] It also uses some of > Sumit's IPI patch set for NMI.[2] > > [1] > https://lore.kernel.org/linux-arm-kernel/20200901144324.1071694-1-maz@ke > rnel.org/ > [2] > https://lore.kernel.org/linux-arm-kernel/1599830924-13990-3-git-send-email > -sumit.g...@linaro.org/ > > $ echo 1 > /proc/sys/kernel/panic_on_rcu_stal > $ echo HARDLOCKUP > /sys/kernel/debug/provoke-crash/DIRECT >: kernel panics and crash kernel boot >: makedumpfile saves the system state at HARDLOCKUP in vmcore. > > crash utility: > crash> bt > PID: 3213 TASK: fd001adc5940 CPU: 8 COMMAND: "bash" > #0 [fe0022fefcf0] lkdtm_HARDLOCKUP at fe0010888ab4 > #1 [fe0022fefd10] lkdtm_do_action at fe00108882bc > #2 [fe0022fefd20] direct_entry at fe0010888720 > #3 [fe0022fefd70] full_proxy_write at fe001058cfe4 > #4 [fe0022fefdb0] vfs_write at fe00104a4c2c > #5 [fe0022fefdf0] ksys_write at fe00104a4f0c > #6 [fe0022fefe40] __arm64_sys_write at fe00104a4fbc > #7 [fe0022fefe50] el0_svc_common.constprop.0 at fe0010159e38 > #8 [fe0022fefe80] do_el0_svc at fe0010159fa0 > #9 [fe0022fefe90] el0_svc at fe00101481d0 > #10 [fe0022fefea0] el0_sync_handler at fe00101484b4 > #11 [fe0022fefff0] el0_sync at fe0010142b7c > > > Sumit Garg (1): > irqchip/gic-v3: Enable support for SGIs to act as NMIs > > Yuichi Ito (1): > Register IPI_CPU_CRASH_STOP IPI as pseudo-NMI > > arch/arm64/kernel/smp.c | 39 > > drivers/irqchip/irq-gic-v3.c | 13 ++-- > 2 files changed, 42 insertions(+), 10 deletions(-) > > -- > 2.25.1
