[PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
This is a regression fix.
Before, to fix erratum AVR31, commit 522e66464467 ("x86/apic: Disable
I/O APIC before shutdown of the local APIC") moved lapic_shutdown()
calling after disable_IO_APIC() in reboot and kexec/kdump code path.
This introdued a regression. The root cause is that disable_IO_APIC()
not only clears IO_APIC, also restore boot irq mode by setting
LAPIC/APIC/IMCR, calling lapic_shutdown() after disable_IO_APIC() will
disable LAPIC and ruin the possible virtual wire mode setting which
the code has been trying to do all along.
The consequence is, in KVM guest kernel always prints warning as below
during kexec/kdump kernel boots up. That happened in setup_local_APIC()
since 'do { xxx } while (queued && max_loops > 0)' loop does not function
well any more if pending irq exists in APIC IRR after LAPIC is disabled.
[0.001000] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/apic/apic.c:1467
setup_local_APIC+0x228/0x330
[0.001000] Modules linked in:
[0.001000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc5+ #3
[0.001000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.10.2-1.fc26 04/01/2014
[0.001000] RIP: 0010:setup_local_APIC+0x228/0x330
[0.001000] RSP: :b6e03eb8 EFLAGS: 00010286
[0.001000] RAX: 009edb4c4d84 RBX: RCX: b099d800
[0.001000] RDX: 009e RSI: RDI: 0810
[0.001000] RBP: R08: R09: 0001
[0.001000] R10: 98ce6a801c00 R11: 0761076d072f0776 R12: 0001
[0.001000] R13: 00f0 R14: 4000 R15: c6ff
[0.001000] FS: () GS:98ce6bc0()
knlGS:
[0.001000] CS: 0010 DS: ES: CR0: 80050033
[0.001000] CR2: CR3: 22209000 CR4: 000406b0
[0.001000] Call Trace:
[0.001000] apic_bsp_setup+0x56/0x74
[0.001000] x86_late_time_init+0x11/0x16
[0.001000] start_kernel+0x3c9/0x486
[0.001000] secondary_startup_64+0xa5/0xb0
[0.001000] Code: 00 85 c9 74 2d 0f 31 c1 e1 0a 48 c1 e2 20 41 89 cf 4c 03
7c 24 08 48 09 d0 49 29 c7 4c 89 3c 24 48 83 3c 24 00 0f 8f 8f fe ff
ff <0f> ff e9 10 ff ff ff 48 83 2c 24 01 eb e7 48 83 c4 18 5b 5d 41
[0.001000] ---[ end trace b88e71b9a6ebebdd ]---
[0.001000] masked ExtINT on CPU#0
To fix this, just call clear_IO_APIC() to stop IO_APIC where
disable_IO_APIC() was called, and call restore_boot_irq_mode() to
restore boot irq mode before reboot or kexec/kdump jump.
Signed-off-by: Baoquan He
Fixes: commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the
local APIC")
Cc: sta...@vger.kernel.org
---
v4->v5:
Take the change related to KEXEC_JUMP out to new patch 0002.
v4->v3:
Eric pointed out the change related to KEXEC_JUMP is not right.
Correct it.
Add Fixes tag and Cc to stable.
arch/x86/kernel/crash.c | 3 ++-
arch/x86/kernel/reboot.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 10e74d4778a1..1f6680427ff0 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -199,9 +199,10 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
#ifdef CONFIG_X86_IO_APIC
/* Prevent crash_kexec() from deadlocking on ioapic_lock. */
ioapic_zap_locks();
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
#endif
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 2126b9d27c34..725624b6c0c0 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -666,7 +666,7 @@ void native_machine_shutdown(void)
* Even without the erratum, it still makes sense to quiet IO APIC
* before disabling Local APIC.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
#ifdef CONFIG_SMP
@@ -680,6 +680,7 @@ void native_machine_shutdown(void)
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
--
2.13.6
[PATCH v5 3/6] x86/apic: Fix restoring boot irq mode in reboot and kexec/kdump
This is a regression fix.
Before, to fix erratum AVR31, commit 522e66464467 ("x86/apic: Disable
I/O APIC before shutdown of the local APIC") moved lapic_shutdown()
calling after disable_IO_APIC() in reboot and kexec/kdump code path.
This introdued a regression. The root cause is that disable_IO_APIC()
not only clears IO_APIC, also restore boot irq mode by setting
LAPIC/APIC/IMCR, calling lapic_shutdown() after disable_IO_APIC() will
disable LAPIC and ruin the possible virtual wire mode setting which
the code has been trying to do all along.
The consequence is, in KVM guest kernel always prints warning as below
during kexec/kdump kernel boots up. That happened in setup_local_APIC()
since 'do { xxx } while (queued && max_loops > 0)' loop does not function
well any more if pending irq exists in APIC IRR after LAPIC is disabled.
[0.001000] WARNING: CPU: 0 PID: 0 at arch/x86/kernel/apic/apic.c:1467
setup_local_APIC+0x228/0x330
[0.001000] Modules linked in:
[0.001000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-rc5+ #3
[0.001000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.10.2-1.fc26 04/01/2014
[0.001000] RIP: 0010:setup_local_APIC+0x228/0x330
[0.001000] RSP: :b6e03eb8 EFLAGS: 00010286
[0.001000] RAX: 009edb4c4d84 RBX: RCX: b099d800
[0.001000] RDX: 009e RSI: RDI: 0810
[0.001000] RBP: R08: R09: 0001
[0.001000] R10: 98ce6a801c00 R11: 0761076d072f0776 R12: 0001
[0.001000] R13: 00f0 R14: 4000 R15: c6ff
[0.001000] FS: () GS:98ce6bc0()
knlGS:
[0.001000] CS: 0010 DS: ES: CR0: 80050033
[0.001000] CR2: CR3: 22209000 CR4: 000406b0
[0.001000] Call Trace:
[0.001000] apic_bsp_setup+0x56/0x74
[0.001000] x86_late_time_init+0x11/0x16
[0.001000] start_kernel+0x3c9/0x486
[0.001000] secondary_startup_64+0xa5/0xb0
[0.001000] Code: 00 85 c9 74 2d 0f 31 c1 e1 0a 48 c1 e2 20 41 89 cf 4c 03
7c 24 08 48 09 d0 49 29 c7 4c 89 3c 24 48 83 3c 24 00 0f 8f 8f fe ff
ff <0f> ff e9 10 ff ff ff 48 83 2c 24 01 eb e7 48 83 c4 18 5b 5d 41
[0.001000] ---[ end trace b88e71b9a6ebebdd ]---
[0.001000] masked ExtINT on CPU#0
To fix this, just call clear_IO_APIC() to stop IO_APIC where
disable_IO_APIC() was called, and call restore_boot_irq_mode() to
restore boot irq mode before reboot or kexec/kdump jump.
Signed-off-by: Baoquan He
Fixes: commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the
local APIC")
Cc: sta...@vger.kernel.org
---
v4->v5:
Take the change related to KEXEC_JUMP out to new patch 0002.
v4->v3:
Eric pointed out the change related to KEXEC_JUMP is not right.
Correct it.
Add Fixes tag and Cc to stable.
arch/x86/kernel/crash.c | 3 ++-
arch/x86/kernel/reboot.c | 3 ++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 10e74d4778a1..1f6680427ff0 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -199,9 +199,10 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
#ifdef CONFIG_X86_IO_APIC
/* Prevent crash_kexec() from deadlocking on ioapic_lock. */
ioapic_zap_locks();
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
#endif
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 2126b9d27c34..725624b6c0c0 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -666,7 +666,7 @@ void native_machine_shutdown(void)
* Even without the erratum, it still makes sense to quiet IO APIC
* before disabling Local APIC.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
#ifdef CONFIG_SMP
@@ -680,6 +680,7 @@ void native_machine_shutdown(void)
#endif
lapic_shutdown();
+ restore_boot_irq_mode();
#ifdef CONFIG_HPET_TIMER
hpet_disable();
--
2.13.6
