Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 11:19, Eggert, Lars wrote: Hi, I'm trying to run a recent-CURRENT XEN i386 kernel in a Xen 4.2 PV domU. I had to add the following options to the kernel config file in order to compile the XEN kernel: makeoptions WITHOUT_MODULES+=mlx4 mlx4ib mlxen mthca The resulting kernel compiles, but fails to boot. Attached is the log. Any ideas? This is due to the unification of event channel implementations. I'm attaching a patch that should fix the issue (at least when using only one vcpu). The i386 Xen PV port doesn't fill the data in struct pcpu, so we end up with uninitialised values there which are required for the new event channel implementation. Maybe someone with more knowledge of the Xen i386 PV port can provide a better solution. If you plan to use the i386 PV port you will also need to revert r244237 in order for it to work. May I ask if there's anything that prevents you from switching to the PVHVM port? I wouldn't recommend using the i386 PV port. Roger. From 0429a1ba5bdbca844e9a798df13396e1ee11aae0 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne roger@citrix.com Date: Mon, 9 Sep 2013 11:53:15 +0200 Subject: [PATCH] xen: set acpi_id for i386 PV port Set a 'fake' acpi_id for the i386 PV port, it is needed in order to use VIRQs or IPI event channels. --- sys/i386/xen/mp_machdep.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/sys/i386/xen/mp_machdep.c b/sys/i386/xen/mp_machdep.c index 274598f..4e4c07d 100644 --- a/sys/i386/xen/mp_machdep.c +++ b/sys/i386/xen/mp_machdep.c @@ -251,6 +251,9 @@ cpu_add(u_int apic_id, char boot_cpu) if (bootverbose) printf(SMP: Added CPU %d (%s)\n, apic_id, boot_cpu ? BSP : AP); + + /* Set the ACPI id (it is needed by VCPU operations) */ + pcpu_find(apic_id)-pc_acpi_id = apic_id; } void -- 1.7.7.5 (Apple Git-26) ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, thanks, let me try your suggestion/patch. On Sep 9, 2013, at 13:53, Roger Pau Monné roger@citrix.com wrote: May I ask if there's anything that prevents you from switching to the PVHVM port? I wouldn't recommend using the i386 PV port. I was thinking that performance would be better, and resource requirements lower. If that's not the case, I can certainly go the PVHVM route. Lars signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 14:06, Eggert, Lars wrote: Hi, thanks, let me try your suggestion/patch. On Sep 9, 2013, at 13:53, Roger Pau Monné roger@citrix.com wrote: May I ask if there's anything that prevents you from switching to the PVHVM port? I wouldn't recommend using the i386 PV port. I was thinking that performance would be better, and resource requirements lower. If that's not the case, I can certainly go the PVHVM route. Performance will probably be better on most workloads with PVHVM if you have HAP (Intel's Extended Page Table or AMD's Nested Page Tables). And also, i386 PV port is limited to only one vcpu, while PVHVM will allow you to use as many vcpus as you want. PVHVM certainly has a higher resource requirement because it requires you to have a Qemu instance for each running VM, but that process is going to be idle most of the time (Qemu is mainly needed for booting). Roger. ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, On Sep 9, 2013, at 14:06, Eggert, Lars l...@netapp.com wrote: thanks, let me try your suggestion/patch. I applied the patch and tried booting. That failed (as expected) with a bad pte panic: ... Trying to mount root from ufs:/dev/ada0s1a []... start_init: trying /sbin/init dumpon: /dev/da1s1a: No such file or directory pid 40 (sh), uid 0: exited on signal 11 TPTE at 0xbf840258 IS ZERO @ VA 0804b000 panic: bad pte cpuid = 0 KDB: enter: panic [ thread pid 40 tid 100032 ] Stopped at kdb_enter+0x3d: movl$0,kdb_why I then reverted r244237. The domU now boots a bit further, but there still seem to be significant issues. Below is the complete log. Lars root@six:/home/elars# xl create -c pvm-freebsd.cfg Parsing config from pvm-freebsd.cfg Daemon running with PID 13661 WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #26 bc8458f-dirty: Mon Sep 9 15:11:32 CEST 2013 el...@stanley.muccbc.hq.netapp.com:/usr/home/elars/obj/i386.i386/usr/home/elars/src/sys/FAS3270 i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 Xen reported: 3000.214 MHz processor. CPU: Intel(R) Xeon(R) CPU E5240 @ 3.00GHz (3000.21-MHz 686-class CPU) Origin = GenuineIntel Id = 0x1067a Family = 0x6 Model = 0x17 Stepping = 10 Features=0xbfe3fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xc0ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE,OSXSAVE AMD Features=0x2010NX,LM AMD Features2=0x1LAHF Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 6144 kbytes, 16-way associative, 64 bytes/line real memory = 268435456 (256 MB) Physical memory chunk(s): 0x00564000 - 0x0fb23fff, 257687552 bytes (62912 pages) avail memory = 255848448 (243 MB) random device not loaded; using insecure entropy ULE: setup cpu 0 Event-channel device installed. nfslock: pseudo-device io: I/O random: Software, Yarrow initialized mem: memory Pentium Pro MTRR support enabled null: null device, zero device xc0: Xen Console on motherboard xenstore0: XenStore on motherboard Grant table initialized xen_et0: Xen PV Clock on motherboard Event timer XENTIMER frequency 10 Hz quality 950 Timecounter XENTIMER frequency 10 Hz quality 950 xen_et0: registered as a time-of-day clock (resolution 1000us, adjustment 5.0s) Device configuration finished. procfs registered Timecounters tick every 10.000 msec tcp_init: net.inet.tcp.tcbhashsize auto tuned to 2048 lo0: bpf attached xctrl0: Xen Control Device on xenstore0 xenbusb_front0: Xen Frontend Devices on xenstore0 xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 xn0: Virtual Network Interface at device/vif/0 on xenbusb_front0 xn0: bpf attached xn0: Ethernet address: 00:16:3e:13:dd:43 xenbusb_back0: Xen Backend Devices on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xbd0: 384MB Virtual Block Device at device/vbd/769 on xenbusb_front0 xbd0: attaching as ada0 xbd0: features: write_barrier xbd0: synchronize cache commands enabled. GEOM: new disk ada0 [XEN] IPI cpu=0 port=6 vector=RESCHEDULE_VECTOR (0) [XEN] IPI cpu=0 port=7 vector=CALL_FUNCTION_VECTOR (1) TSC timecounter discards lower 1 bit(s) Timecounter TSC-low frequency 1500107000 Hz quality 800 Trying to mount root from ufs:/dev/ada0s1a []... start_init: trying /sbin/init dumpon: /dev/da1s1a: No such file or directory pid 42 (logger), uid 0: exited on signal 8 Floating point exception /etc/rc: WARNING: unable to specify /dev/da1s1a as a dump device Entropy harvesting: interrupts ethernet point_to_pointpid 50 (dd), uid 0: exited on signal 8 pid 53 (df), uid 0: exited on signal 8 Floating point exception pid 54 (ps), uid 0: exited on signal 8 Floating point exception pid 55 (date), uid 0: exited on signal 8 Floating point exception Floating point exception pid 58 (dd), uid 0: exited on signal 8 Floating point exception kickstart. eval: cannot create /etc/hostid: Read-only file system pid 76 (logger), uid 0: exited on signal 8 Floating point exception /etc/rc: WARNING: could not store hostuuid in /etc/hostid. pid 82 (newfs), uid 0: exited on signal 8 mount_mfs: mount exited with error code 1 pid 87 (newfs), uid 0: exited on signal 8 mount_mfs: mount exited with error code 1 Mounting /etc/fstab filesystems failed, startup aborted ERROR: ABORTING BOOT (sending
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 15:13, Eggert, Lars wrote: Hi, On Sep 9, 2013, at 14:06, Eggert, Lars l...@netapp.com wrote: thanks, let me try your suggestion/patch. I applied the patch and tried booting. That failed (as expected) with a bad pte panic: ... Trying to mount root from ufs:/dev/ada0s1a []... start_init: trying /sbin/init dumpon: /dev/da1s1a: No such file or directory pid 40 (sh), uid 0: exited on signal 11 TPTE at 0xbf840258 IS ZERO @ VA 0804b000 panic: bad pte cpuid = 0 KDB: enter: panic [ thread pid 40 tid 100032 ] Stopped at kdb_enter+0x3d: movl$0,kdb_why I then reverted r244237. The domU now boots a bit further, but there still seem to be significant issues. Below is the complete log. Applying the patch I've sent and reverting r244237 seems to be enough for me, I can get the guest to boot fine. This is the cmdline that I use for booting: boot_verbose=1,vfs.root.mountfrom=ufs:/dev/xbd0p2,kern.hz=100,hw.mca.enabled=0 WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #72 06b2ac0: Mon Sep 9 15:54:59 CEST 2013 root@:/usr/obj/usr/src/sys/XEN i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 3066.888 MHz processor. CPU: Intel(R) Xeon(R) CPU W3550 @ 3.07GHz (3066.89-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 0x6 Model = 0x1a Stepping = 5 Features=0xbfe3fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x9ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT AMD Features=0x2810NX,RDTSCP,LM AMD Features2=0x1LAHF Data TLB: 4 KB pages, 4-way set associative, 64 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 256 kbytes, 8-way associative, 64 bytes/line real memory = 536870912 (512 MB) Physical memory chunk(s): 0x0085f000 - 0x1f6a0fff, 518266880 bytes (126530 pages) avail memory = 515629056 (491 MB) random device not loaded; using insecure entropy ULE: setup cpu 0 Event-channel device installed. null: null device, zero device nfslock: pseudo-device io: I/O random: Software, Yarrow initialized mem: memory Pentium Pro MTRR support enabled xc0: Xen Console on motherboard xenstore0: XenStore on motherboard Grant table initialized xen_et0: Xen PV Clock on motherboard Event timer XENTIMER frequency 10 Hz quality 950 Timecounter XENTIMER frequency 10 Hz quality 950 xen_et0: registered as a time-of-day clock (resolution 1000us, adjustment 5.0s) Device configuration finished. procfs registered Timecounters tick every 10.000 msec tcp_init: net.inet.tcp.tcbhashsize auto tuned to 4096 lo0: bpf attached xctrl0: Xen Control Device on xenstore0 xenbusb_front0: Xen Frontend Devices on xenstore0 xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 xn0: Virtual Network Interface at device/vif/0 on xenbusb_front0 xn0: bpf attached xn0: Ethernet address: 00:16:3e:47:d4:52 xenbusb_back0: Xen Backend Devices on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xbd0: 20480MB Virtual Block Device at device/vbd/51712 on xenbusb_front0 xbd0: features: flush, write_barrier xbd0: synchronize cache commands enabled. GEOM: new disk xbd0 TSC timecounter discards lower 1 bit(s) Timecounter TSC-low frequency 1533444000 Hz quality 800 WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/xbd0p2 []... WARNING: / was not properly dismounted WARNING: /: mount pending error: blocks 0 files 1 start_init: trying /sbin/init Setting hostuuid: ee840805-6f57-4031-b4b8-919d4ac8021a. Setting hostid: 0xf9f637d0. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No such file or directory kickstart. swapon: /dev/ad0p3: No such file or directory Starting file system checks: ** SU+J Recovering /dev/xbd0p2 ** Reading 33554432 byte journal from inode 4. ** Building recovery table. ** Resolving unreferenced inode list. ** Processing journal entries. ** 17 journal records in 1024 bytes for 53.12% utilization ** Freed 0 inodes (0 dirs) 0 blocks, and 0 frags. * FILE SYSTEM MARKED CLEAN * Mounting local file systems:. Writing entropy file:. xn0: link state changed to DOWN xn0: link state changed to UP Starting Network: lo0 xn0. lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384 options=63RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, On Sep 9, 2013, at 16:02, Roger Pau Monné roger@citrix.com wrote: Applying the patch I've sent and reverting r244237 seems to be enough for me, I can get the guest to boot fine. same here, now that I removed the TARGET_CPUTYPE=core2 flag that had been left over in the local setup. Thanks, Lars signature.asc Description: Message signed with OpenPGP using GPGMail
