Re: [Xen-devel] FreeBSD PVHVM call for testing
On 05/29/13 10:45, Roger Pau Monné wrote: Oh, sure, more changes where needed in order to get it to work, like using acpi_id to map the vcpu_info and perform the cpu bindings. Ah, that explains it. I looked at timer.c for other places where that change was needed but it didn't occur to me that the same problem would exist in other parts of the tree. On 29/05/13 19:22, Matt Wilson wrote: On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote: http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12 Colin, can you build an AMI with this new kernel? Done, ami-95177dfc in us-east-1. This is now booting successfully on cr1.8xlarge; are there any other instance types I should test? I don't know which Xen versions you have deployed across the entire fleet. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: [Xen-devel] FreeBSD PVHVM call for testing
On 05/28/13 12:18, Matt Wilson wrote: VCPUOP_set_singleshot_timer returns -EINVAL when: 1) the specified vCPU ID is out of range (0 or MAX_VIRT_CPUS) 2) the specified vCPU ID doesn't match the running vCPU. It seems that there is a confusion between the logical vCPU ID and the local APIC physical ID. [...] (XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1 [...] APIC: CPU 1 has ACPI ID 16 Thanks Matt! Looks like we need to pass our acpi_id to the Xen hypercall instead of our cpuid. Roger, changing the line int cpu = PCPU_GET(cpuid); to int cpu = PCPU_GET(acpi_id); in xentimer_et_start and xentimer_et_stop fixes this panic and gets me slightly further; the following lines are now added to the console output prior to the system appearing to hang: ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 2 vector 48 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 3 vector 48 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 4 vector 48 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 5 vector 48 ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 6 vector 48 ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 7 vector 48 TSC timecounter discards lower 1 bit(s) Timecounter TSC-low frequency 1300024860 Hz quality -100 WARNING: WITNESS option enabled, expect reduced performance. On a cc2.8xlarge EC2 instance, the lines which come after this are GEOM: new disk xbd1 GEOM: new disk xbd2 GEOM: new disk xbd3 GEOM: new disk xbd4 Trying to mount root from ufs:/dev/ad0a [rw]... start_init: trying /sbin/init and then the userland boot process; have you made any bug fixes after your pvhvm_v7 which would explain why tasting disks was hanging? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/22/13 04:45, Roger Pau Monné wrote: On 18/05/13 17:44, Colin Percival wrote: That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? I have not tested ZFS, that might be a good one. If you are running this on Xen 3.4 the behaviour should be the same as without this patches, so there shouldn't be many differences. I don't use ZFS personally, so I'm not sure exactly what tests to run on it; hopefully someone else can take care of that. If you could try that on Xen 4.0 at least (if I remember correctly that's when the vector callback was introduced), you should see the PV timer getting attached, and a performance increase. Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with a panic -- console output below. I can get a backtrace and possibly even a dump if those would help. Booting [/boot/kernel/kernel]... -\|/-\|GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb SMAP type=01 base= len=0009e000 SMAP type=02 base=0009e000 len=2000 SMAP type=02 base=000e len=0002 SMAP type=01 base=0010 len=eff0 SMAP type=02 base=fc00 len=0400 SMAP type=01 base=0001 len=003c1900 Table 'FACP' at 0xfc014980 Table 'APIC' at 0xfc014a80 APIC: Found table at 0xfc014a80 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 1: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 2: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 3: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 4: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 5: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 12 ACPI ID 6: enabled SMP: Added CPU 12 (AP) MADT: Found CPU APIC ID 14 ACPI ID 7: enabled SMP: Added CPU 14 (AP) MADT: Found CPU APIC ID 32 ACPI ID 8: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 9: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 10: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 11: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 12: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 13: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 44 ACPI ID 14: enabled SMP: Added CPU 44 (AP) MADT: Found CPU APIC ID 46 ACPI ID 15: enabled SMP: Added CPU 46 (AP) MADT: Found CPU APIC ID 1 ACPI ID 16: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 3 ACPI ID 17: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 5 ACPI ID 18: enabled SMP: Added CPU 5 (AP) MADT: Found CPU APIC ID 7 ACPI ID 19: enabled SMP: Added CPU 7 (AP) MADT: Found CPU APIC ID 9 ACPI ID 20: enabled SMP: Added CPU 9 (AP) MADT: Found CPU APIC ID 11 ACPI ID 21: enabled SMP: Added CPU 11 (AP) MADT: Found CPU APIC ID 13 ACPI ID 22: enabled SMP: Added CPU 13 (AP) MADT: Found CPU APIC ID 15 ACPI ID 23: enabled SMP: Added CPU 15 (AP) MADT: Found CPU APIC ID 33 ACPI ID 24: enabled SMP: Added CPU 33 (AP) MADT: Found CPU APIC ID 35 ACPI ID 25: enabled SMP: Added CPU 35 (AP) MADT: Found CPU APIC ID 37 ACPI ID 26: enabled SMP: Added CPU 37 (AP) MADT: Found CPU APIC ID 39 ACPI ID 27: enabled SMP: Added CPU 39 (AP) MADT: Found CPU APIC ID 41 ACPI ID 28: enabled SMP: Added CPU 41 (AP) MADT: Found CPU APIC ID 43 ACPI ID 29: enabled SMP: Added CPU 43 (AP) MADT: Found CPU APIC ID 45 ACPI ID 30: enabled SMP: Added CPU 45 (AP) MADT: Found CPU APIC ID 47 ACPI ID 31: enabled SMP: Added CPU 47 (AP) MADT: Found CPU APIC ID 0 ACPI ID 32: disabled MADT: Found CPU APIC ID 0 ACPI ID 33: disabled MADT: Found CPU APIC ID 0 ACPI ID 34: disabled MADT: Found CPU APIC ID 0 ACPI ID 35: disabled MADT: Found CPU APIC ID 0 ACPI ID 36: disabled MADT: Found CPU APIC ID 0 ACPI ID 37: disabled MADT: Found CPU APIC ID 0 ACPI ID 38: disabled MADT: Found CPU APIC ID 0 ACPI ID 39: disabled MADT: Found CPU APIC ID 0 ACPI ID 40: disabled MADT: Found CPU APIC ID 0 ACPI ID 41: disabled MADT: Found CPU APIC ID 0 ACPI ID 42: disabled MADT: Found CPU APIC ID 0 ACPI ID 43: disabled MADT: Found CPU APIC ID 0 ACPI ID 44: disabled MADT: Found CPU APIC ID 0 ACPI ID 45: disabled MADT: Found CPU APIC ID 0 ACPI ID 46: disabled MADT: Found CPU APIC ID 0 ACPI ID 47: disabled MADT: Found CPU APIC ID 0 ACPI ID 48: disabled MADT: Found CPU APIC ID 0 ACPI ID 49: disabled MADT: Found CPU APIC ID 0 ACPI ID 50: disabled MADT: Found CPU APIC ID 0 ACPI ID 51: disabled MADT: Found CPU APIC ID 0 ACPI ID 52: disabled MADT: Found CPU APIC ID 0 ACPI ID 53: disabled MADT: Found CPU APIC ID 0 ACPI ID 54: disabled MADT: Found CPU APIC ID 0 ACPI ID 55: disabled MADT: Found CPU APIC ID 0 ACPI ID 56: disabled MADT: Found CPU APIC ID 0 ACPI ID 57: disabled MADT: Found CPU APIC ID 0 ACPI ID 58: disabled MADT: Found CPU APIC ID 0 ACPI ID 59: disabled MADT: Found CPU APIC ID 0 ACPI
Re: FreeBSD PVHVM call for testing
On 05/18/13 02:50, Roger Pau Monné wrote: On 17/05/13 05:07, Colin Percival wrote: On 05/16/13 17:43, Roger Pau Monné wrote: Thanks for testing this on EC2, could you post the full dmesg? So I can see the hypervisor version and if the PV timer is loaded or not. Here's what I get on a cc2.8xlarge with boot_verbose=YES: I've pushed a new branch to my repository, pvhvm_v7 that should work, there was a bug with PCI event channel interrupt set up. I've tested with 3.4 and seems OK, but of course it doesn't support the vector callback injection. That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? If anyone else wants to play with this, you can launch ami-e75c358e in the EC2 us-east-1 region. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid Table 'FACP' at 0xfc005ee0 Table 'APIC' at 0xfc005fe0 APIC: Found table at 0xfc005fe0 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 1: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 2: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 3: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 4: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 5: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 12 ACPI ID 6: enabled SMP: Added CPU 12 (AP) MADT: Found CPU APIC ID 14 ACPI ID 7: enabled SMP: Added CPU 14 (AP) MADT: Found CPU APIC ID 32 ACPI ID 8: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 9: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 10: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 11: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 12: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 13: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 44 ACPI ID 14: enabled SMP: Added CPU 44 (AP) MADT: Found CPU APIC ID 46 ACPI ID 15: enabled SMP: Added CPU 46 (AP) MADT: Found CPU APIC ID 1 ACPI ID 16: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 3 ACPI ID 17: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 5 ACPI ID 18: enabled SMP: Added CPU 5 (AP) MADT: Found CPU APIC ID 7 ACPI ID 19: enabled SMP: Added CPU 7 (AP) MADT: Found CPU APIC ID 9 ACPI ID 20: enabled SMP: Added CPU 9 (AP) MADT: Found CPU APIC ID 11 ACPI ID 21: enabled SMP: Added CPU 11 (AP) MADT: Found CPU APIC ID 13 ACPI ID 22: enabled SMP: Added CPU 13 (AP) MADT: Found CPU APIC ID 15 ACPI ID 23: enabled SMP: Added CPU 15 (AP) MADT: Found CPU APIC ID 33 ACPI ID 24: enabled SMP: Added CPU 33 (AP) MADT: Found CPU APIC ID 35 ACPI ID 25: enabled SMP: Added CPU 35 (AP) MADT: Found CPU APIC ID 37 ACPI ID 26: enabled SMP: Added CPU 37 (AP) MADT: Found CPU APIC ID 39 ACPI ID 27: enabled SMP: Added CPU 39 (AP) MADT: Found CPU APIC ID 41 ACPI ID 28: enabled SMP: Added CPU 41 (AP) MADT: Found CPU APIC ID 43 ACPI ID 29: enabled SMP: Added CPU 43 (AP) MADT: Found CPU APIC ID 45 ACPI ID 30: enabled SMP: Added CPU 45 (AP) MADT: Found CPU APIC ID 47 ACPI ID 31: enabled SMP: Added CPU 47 (AP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r+9b25356: Sat May 18 14:46:16 UTC 2013 root@ip-10-140-132-115:/usr/obj/usr/src/sys/XENHVM amd64 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. XEN: Hypervisor version 3.4 detected. XEN: Disabling emulated block and network devices Preloaded elf kernel /boot/kernel/kernel at 0x81912000. Hypervisor: Origin = XenVMMXenVMM Calibrating TSC clock ... TSC clock: 2593802768 Hz CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2593.80-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206d6 Family = 0x6 Model = 0x2d Stepping = 6 Features=0x1781fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x9c982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,OSXSAVE,AVX,HV AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF real memory = 65011712000 (62000 MB) Physical memory chunk(s): 0x1000 - 0x0009bfff, 634880 bytes (155 pages) 0x0010 - 0x001f, 1048576 bytes (256 pages) 0x01972000 - 0xbfff, 3194544128 bytes (779918 pages) 0x0001 - 0x000efeb95fff, 60108136448 bytes (14674838 pages) avail memory = 60563271680 (57757 MB) Event timer LAPIC quality 400 ACPI APIC Table: Xen HVM INTR: Adding local APIC 1 as a target INTR: Adding local APIC 2 as a target INTR: Adding local APIC 3 as a target INTR: Adding local APIC 4 as a target INTR: Adding local APIC 5 as a target INTR
Re: FreeBSD PVHVM call for testing
On 05/13/13 11:32, Roger Pau Monné wrote: Right now the code is in a state where it can be tested by users, so we would like to encourage FreeBSD and Xen users to test it and provide feedback. The code is available in the following git repository, under the branch pvhvm_v5: http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary Also, I've created a wiki page that explains how to set up a FreeBSD PVHVM for testing: http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM I built a XENHVM kernel with this code on EC2, and it hanged after xenbusb_front0: Xen Frontend Devices on xenstore0 With a XENHVM kernel from FreeBSD HEAD the next line after that is xbd0: 10240MB Virtual Block Device at device/vbd/768 on xenbusb_front0 Any ideas? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xb) for rid 0 of orm0 ahc_isa_identify 0: ioport 0xc00 alloc failed ahc_isa_identify 1: ioport 0x1c00 alloc failed ahc_isa_identify 2: ioport 0x2c00 alloc failed ahc_isa_identify 3: ioport 0x3c00 alloc failed ahc_isa_identify 4: ioport 0x4c00 alloc failed ahc_isa_identify 5: ioport 0x5c00 alloc failed ahc_isa_identify 6: ioport 0x6c00 alloc failed ahc_isa_identify 7: ioport 0x7c00 alloc failed ahc_isa_identify 8: ioport 0x8c00 alloc failed ahc_isa_identify 9: ioport 0x9c00 alloc failed ahc_isa_identify 10: ioport 0xac00 alloc failed ahc_isa_identify 11: ioport 0xbc00 alloc failed ahc_isa_identify 12: ioport 0xcc00 alloc failed ahc_isa_identify 13: ioport 0xdc00 alloc failed ahc_isa_identify 14: ioport 0xec00 alloc failed isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices sc0: System console at flags 0x100 on isa0 sc0: VGA 16 virtual consoles, flags=0x100 sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: Generic ISA VGA at port 0x3c0-0x3df iomem 0xa-0xb on isa0 pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0 pcib0: allocated type 3 (0xa-0xb) for rid 0 of vga0 fdc0: No FDOUT register! fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 ppc0: cannot reserve I/O port range ppc0 failed to probe at irq 7 on isa0 pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1 uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices Device configuration finished. procfs registered lapic: Divisor 2, Frequency 5453 Hz Timecounters tick every 10.000 msec vlan: initialized, using hash tables with chaining tcp_init: net.inet.tcp.tcbhashsize auto tuned to 524288 lo0: bpf attached hptrr: no controller detected. hpt27xx: no controller detected. xenbusb_front0: Xen Frontend Devices on xenstore0 ata0: reset tp1 mask=03 ostat0=00 ostat1=00 ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=00 stat1=00 devices=0x0 ata1: reset tp1 mask=03 ostat0=00 ostat1=00 ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00 ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 ata1: reset tp2 stat0=00 stat1=00 devices=0x0 -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/13/13 11:52, Michael Sierchio wrote: I think should be encouraged. We're eagerly awaiting the ability to run FreeBSD in AWS in something other than t1.micro or cluster compute instances. Should we keep holding out hope, or will AWS make HVM available for all instance types before this happens? Err... my AMIs run on all EC2 instance types. On some you have to pay the Windows rate, that's all. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/13/13 12:08, Michael Sierchio wrote: The Windoze tax is unacceptable for a number of reasons - the primary reason is that I'm not running Windows. I don't think the licensing scheme is unfair for those actually running Windows, mind you. Right, it's definitely annoying having to pay more -- I just wanted to point out that the ability does exist, if you're willing to pay the price. At the AWS Summit, an assertion was made that HVM support might be coming soon for all instance types for *NIX OSes. I hope that's true. Was it indeed? I must not have been present for that... it certainly would be good news. Certainly all the new instance types they've released in the past few years have had UNIX HVM support. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
