Re: Panic (gpf) in early boot after upgrading FreeBSD 10.4 -> 11.2 on Ganeti

2018-11-12 Thread Rob Austein
On Mon, 12 Nov 2018 01:58:12 -0500, Konstantin Belousov wrote:
> On Sun, Nov 11, 2018 at 11:24:48PM -0500, Rob Austein wrote:
> > 
> > panic: general protection fault
> > cpuid = 0
> > KDB: stack backtrace:
> > #0 0x80b3d587 at kdb_backtrace+0x67
> > #1 0x80af6b27 at vpanic+0x177
> > #2 0x80af69a3 at panic+0x43
> > #3 0x80f77fdf at trap_fatal+0x35f
> > #4 0x80f7759e at trap+0x5e
> > #5 0x80f57fbc at calltrap+0x8
> > #6 0x810ec5f3 at apic_setup_io+0x53
> > #7 0x80a92898 at mi_startup+0x118
> > #8 0x8031002c at btext+0x2c
> > 
> Try to issue the following commands at the loader prompt:
>   set hw.x2apic_enable=0
>   boot

That worked.  Added to loader.conf.

Thank you!
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Panic (gpf) in early boot after upgrading FreeBSD 10.4 -> 11.2 on Ganeti

2018-11-11 Thread Konstantin Belousov
On Sun, Nov 11, 2018 at 11:24:48PM -0500, Rob Austein wrote:
> Belated upgrade (don't ask) of a pair of FreeBSD 10.4 VMs to 11.2.
> Each VM got as far as:
> 
>   freebsd-update -r 11.2-RELEASE update
>   freebsd-update install
>   reboot
> 
> Each VM got an immediate kernel panic after the reboot (log below).
> 
> The two VMs are basically identical at the system level, but run in
> separate Ganeti clusters on opposite coasts, so no hardware in common.
> A dozen or so other VMs run in each cluster without issues (including
> at least one other FreeBSD 11.2 VM), and the VMs I'm trying to upgrade
> have also been just fine until now, so the problem seems unlikely to
> be hardware per se.
> 
> GENERIC, amd64, UFS2, no non-/boot/kernel modules, one CPU per VM.
> The only things even slightly unusual about these VMs are:
> 
> * They're running in Ganeti clusters:
>   * Ganeti version 2.15.2
>   * KVM hypervisor
> 
> * They use virtio, so:
>   * Disk is vtbd0
>   * Net is vtnet0
> 
> Log of attempted boot with new kernel:
> 
> /boot/kernel/kernel text=0x1547d48 data=0x144138+0x4e9818 
> syms=[0x8+0x16aef8+0x8+0x183f99]
> Booting...
> Copyright (c) 1992-2018 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>   The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018
>   r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on 
> LLVM 6.0.0)
> VT(vga): text 80x25
> CPU: QEMU Virtual CPU version 2.1.2 (2666.81-MHz K8-class CPU)
>   Origin="GenuineIntel"  Id=0x663  Family=0x6  Model=0x6  Stepping=3
>   
> Features=0x783fbfd
>   Features2=0x80a02001
>   AMD Features=0x20100800
>   AMD Features2=0x1
> Hypervisor: Origin = "KVMKVMKVM"
> real memory  = 4294967296 (4096 MB)
> avail memory = 4088406016 (3899 MB)
> Event timer "LAPIC" quality 100
> ACPI APIC Table: 
> ioapic0  irqs 0-23 on motherboard
> kernel trap 9 with interrupts disabled
> 
> 
> Fatal trap 9: general protection fault while in kernel mode
> cpuid = 0; apic id = 00
> instruction pointer   = 0x20:0x810e9ae6
> stack pointer = 0x28:0x82272c20
> frame pointer = 0x28:0x82272c80
> code segment  = base 0x0, limit 0xf, type 0x1b
>   = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags  = resume, IOPL = 0
> current process   = 0 (swapper)
> trap number   = 9
> panic: general protection fault
> cpuid = 0
> KDB: stack backtrace:
> #0 0x80b3d587 at kdb_backtrace+0x67
> #1 0x80af6b27 at vpanic+0x177
> #2 0x80af69a3 at panic+0x43
> #3 0x80f77fdf at trap_fatal+0x35f
> #4 0x80f7759e at trap+0x5e
> #5 0x80f57fbc at calltrap+0x8
> #6 0x810ec5f3 at apic_setup_io+0x53
> #7 0x80a92898 at mi_startup+0x118
> #8 0x8031002c at btext+0x2c
> Uptime: 1s
> Automatic reboot in 15 seconds - press a key on the console to abort
> --> Press a key on the console to reboot,
> --> or switch off the system now.
> 
> Goggling turned up a few theories about bad memory and incompatible
> changes to video drivers, none of which seem likely to apply here.
> 
> Cluebat, please, somebody?
Try to issue the following commands at the loader prompt:
set hw.x2apic_enable=0
boot
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Panic (gpf) in early boot after upgrading FreeBSD 10.4 -> 11.2 on Ganeti

2018-11-11 Thread Rob Austein
Belated upgrade (don't ask) of a pair of FreeBSD 10.4 VMs to 11.2.
Each VM got as far as:

  freebsd-update -r 11.2-RELEASE update
  freebsd-update install
  reboot

Each VM got an immediate kernel panic after the reboot (log below).

The two VMs are basically identical at the system level, but run in
separate Ganeti clusters on opposite coasts, so no hardware in common.
A dozen or so other VMs run in each cluster without issues (including
at least one other FreeBSD 11.2 VM), and the VMs I'm trying to upgrade
have also been just fine until now, so the problem seems unlikely to
be hardware per se.

GENERIC, amd64, UFS2, no non-/boot/kernel modules, one CPU per VM.
The only things even slightly unusual about these VMs are:

* They're running in Ganeti clusters:
  * Ganeti version 2.15.2
  * KVM hypervisor

* They use virtio, so:
  * Disk is vtbd0
  * Net is vtnet0

Log of attempted boot with new kernel:

/boot/kernel/kernel text=0x1547d48 data=0x144138+0x4e9818 
syms=[0x8+0x16aef8+0x8+0x183f99]
Booting...
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018
r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 
6.0.0)
VT(vga): text 80x25
CPU: QEMU Virtual CPU version 2.1.2 (2666.81-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x663  Family=0x6  Model=0x6  Stepping=3
  
Features=0x783fbfd
  Features2=0x80a02001
  AMD Features=0x20100800
  AMD Features2=0x1
Hypervisor: Origin = "KVMKVMKVM"
real memory  = 4294967296 (4096 MB)
avail memory = 4088406016 (3899 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: 
ioapic0  irqs 0-23 on motherboard
kernel trap 9 with interrupts disabled


Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer = 0x20:0x810e9ae6
stack pointer   = 0x28:0x82272c20
frame pointer   = 0x28:0x82272c80
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= resume, IOPL = 0
current process = 0 (swapper)
trap number = 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
#0 0x80b3d587 at kdb_backtrace+0x67
#1 0x80af6b27 at vpanic+0x177
#2 0x80af69a3 at panic+0x43
#3 0x80f77fdf at trap_fatal+0x35f
#4 0x80f7759e at trap+0x5e
#5 0x80f57fbc at calltrap+0x8
#6 0x810ec5f3 at apic_setup_io+0x53
#7 0x80a92898 at mi_startup+0x118
#8 0x8031002c at btext+0x2c
Uptime: 1s
Automatic reboot in 15 seconds - press a key on the console to abort
--> Press a key on the console to reboot,
--> or switch off the system now.

Goggling turned up a few theories about bad memory and incompatible
changes to video drivers, none of which seem likely to apply here.

Cluebat, please, somebody?

Thanks for reading.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"