Re: [Xen-devel] FreeBSD PVHVM call for testing
On 21/05/13 19:40, Konrad Rzeszutek Wilk wrote: On Mon, May 13, 2013 at 08:32:56PM +0200, Roger Pau Monné wrote: Hello, Recently Justin T Gibbs, Will Andrews and myself have been working on improving the Xen support in FreeBSD. The main goal of this was to bring full PVHVM support to FreeBSD, right now FreeBSD is only using PV interfaces for disk and network interfaces when running as a HVM guest. The main benefits of this changes are that Xen virtual interrupts (event channels) are now delivered to the guest using a vector callback injection, that is a per-cpu mechanism that allows each vCPU to have different interrupts assigned, so for example network and disk interrupts are delivered to different vCPUs in order to improve performance. With this changes FreeBSD also uses PV timers when running as an HVM guest, which should provide better time keeping and reduce the virtualization overhead, since emulated timers are no longer used. PV IPIs can also be used inside a HVM guest, but this will be implemented later. Right now the code is in a state where it can be tested by users, so we would like to encourage FreeBSD and Xen users to test it and provide feedback. The code is available in the following git repository, under the branch pvhvm_v5: http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary Also, I've created a wiki page that explains how to set up a FreeBSD PVHVM for testing: http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM I tried on my Linux box to do this: HEAD is now at 9b25356... xen-netfront: fix detach of network interfaces konrad@phenom:~/git/freebsd$ make kernel-toolchain make buildkernel KERNCONF=XENHVM make installkernel KERNCONF=XENHVM Makefile:123: *** missing separator. Stop. As I thought it would compile the same way you can compile NetBSD - that is even on non-BSD distros. Is that not the case? Should I only do this under a FreeBSD guest? I have never tried to run the FreeBSD build system under something different than FreeBSD, also keep in mind that installkernel will place a bunch of FreeBSD files on your Linux box if you manage to run it. The error itself can probably be fixed by installing and using BSD make (bmake), but anyway you need a FreeBSD HVM DomU in order to test the kernel, so why not do the compilation on it? ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 18/05/13 17:44, Colin Percival wrote: On 05/18/13 02:50, Roger Pau Monné wrote: On 17/05/13 05:07, Colin Percival wrote: On 05/16/13 17:43, Roger Pau Monné wrote: Thanks for testing this on EC2, could you post the full dmesg? So I can see the hypervisor version and if the PV timer is loaded or not. Here's what I get on a cc2.8xlarge with boot_verbose=YES: I've pushed a new branch to my repository, pvhvm_v7 that should work, there was a bug with PCI event channel interrupt set up. I've tested with 3.4 and seems OK, but of course it doesn't support the vector callback injection. That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? I have not tested ZFS, that might be a good one. If you are running this on Xen 3.4 the behaviour should be the same as without this patches, so there shouldn't be many differences. If you could try that on Xen 4.0 at least (if I remember correctly that's when the vector callback was introduced), you should see the PV timer getting attached, and a performance increase. If anyone else wants to play with this, you can launch ami-e75c358e in the EC2 us-east-1 region. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: [Xen-devel] FreeBSD PVHVM call for testing
On Wed, May 22, 2013 at 01:41:02PM +0200, Roger Pau Monné wrote: On 21/05/13 19:40, Konrad Rzeszutek Wilk wrote: On Mon, May 13, 2013 at 08:32:56PM +0200, Roger Pau Monné wrote: Hello, Recently Justin T Gibbs, Will Andrews and myself have been working on improving the Xen support in FreeBSD. The main goal of this was to bring full PVHVM support to FreeBSD, right now FreeBSD is only using PV interfaces for disk and network interfaces when running as a HVM guest. The main benefits of this changes are that Xen virtual interrupts (event channels) are now delivered to the guest using a vector callback injection, that is a per-cpu mechanism that allows each vCPU to have different interrupts assigned, so for example network and disk interrupts are delivered to different vCPUs in order to improve performance. With this changes FreeBSD also uses PV timers when running as an HVM guest, which should provide better time keeping and reduce the virtualization overhead, since emulated timers are no longer used. PV IPIs can also be used inside a HVM guest, but this will be implemented later. Right now the code is in a state where it can be tested by users, so we would like to encourage FreeBSD and Xen users to test it and provide feedback. The code is available in the following git repository, under the branch pvhvm_v5: http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary Also, I've created a wiki page that explains how to set up a FreeBSD PVHVM for testing: http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM I tried on my Linux box to do this: HEAD is now at 9b25356... xen-netfront: fix detach of network interfaces konrad@phenom:~/git/freebsd$ make kernel-toolchain make buildkernel KERNCONF=XENHVM make installkernel KERNCONF=XENHVM Makefile:123: *** missing separator. Stop. As I thought it would compile the same way you can compile NetBSD - that is even on non-BSD distros. Is that not the case? Should I only do this under a FreeBSD guest? I have never tried to run the FreeBSD build system under something different than FreeBSD, also keep in mind that installkernel will place a bunch of FreeBSD files on your Linux box if you manage to run it. The error itself can probably be fixed by installing and using BSD make (bmake), but anyway you need a FreeBSD HVM DomU in order to test the kernel, so why not do the compilation on it? Oh, sure. Just thought that the NetBSD method (which also allows one to build the kernel and the iso) was the same on FreeBSd. Will of course install first the HVM domU and do it again. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/22/13 04:45, Roger Pau Monné wrote: On 18/05/13 17:44, Colin Percival wrote: That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? I have not tested ZFS, that might be a good one. If you are running this on Xen 3.4 the behaviour should be the same as without this patches, so there shouldn't be many differences. I don't use ZFS personally, so I'm not sure exactly what tests to run on it; hopefully someone else can take care of that. If you could try that on Xen 4.0 at least (if I remember correctly that's when the vector callback was introduced), you should see the PV timer getting attached, and a performance increase. Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with a panic -- console output below. I can get a backtrace and possibly even a dump if those would help. Booting [/boot/kernel/kernel]... -\|/-\|GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb SMAP type=01 base= len=0009e000 SMAP type=02 base=0009e000 len=2000 SMAP type=02 base=000e len=0002 SMAP type=01 base=0010 len=eff0 SMAP type=02 base=fc00 len=0400 SMAP type=01 base=0001 len=003c1900 Table 'FACP' at 0xfc014980 Table 'APIC' at 0xfc014a80 APIC: Found table at 0xfc014a80 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 1: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 2: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 3: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 4: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 5: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 12 ACPI ID 6: enabled SMP: Added CPU 12 (AP) MADT: Found CPU APIC ID 14 ACPI ID 7: enabled SMP: Added CPU 14 (AP) MADT: Found CPU APIC ID 32 ACPI ID 8: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 9: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 10: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 11: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 12: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 13: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 44 ACPI ID 14: enabled SMP: Added CPU 44 (AP) MADT: Found CPU APIC ID 46 ACPI ID 15: enabled SMP: Added CPU 46 (AP) MADT: Found CPU APIC ID 1 ACPI ID 16: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 3 ACPI ID 17: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 5 ACPI ID 18: enabled SMP: Added CPU 5 (AP) MADT: Found CPU APIC ID 7 ACPI ID 19: enabled SMP: Added CPU 7 (AP) MADT: Found CPU APIC ID 9 ACPI ID 20: enabled SMP: Added CPU 9 (AP) MADT: Found CPU APIC ID 11 ACPI ID 21: enabled SMP: Added CPU 11 (AP) MADT: Found CPU APIC ID 13 ACPI ID 22: enabled SMP: Added CPU 13 (AP) MADT: Found CPU APIC ID 15 ACPI ID 23: enabled SMP: Added CPU 15 (AP) MADT: Found CPU APIC ID 33 ACPI ID 24: enabled SMP: Added CPU 33 (AP) MADT: Found CPU APIC ID 35 ACPI ID 25: enabled SMP: Added CPU 35 (AP) MADT: Found CPU APIC ID 37 ACPI ID 26: enabled SMP: Added CPU 37 (AP) MADT: Found CPU APIC ID 39 ACPI ID 27: enabled SMP: Added CPU 39 (AP) MADT: Found CPU APIC ID 41 ACPI ID 28: enabled SMP: Added CPU 41 (AP) MADT: Found CPU APIC ID 43 ACPI ID 29: enabled SMP: Added CPU 43 (AP) MADT: Found CPU APIC ID 45 ACPI ID 30: enabled SMP: Added CPU 45 (AP) MADT: Found CPU APIC ID 47 ACPI ID 31: enabled SMP: Added CPU 47 (AP) MADT: Found CPU APIC ID 0 ACPI ID 32: disabled MADT: Found CPU APIC ID 0 ACPI ID 33: disabled MADT: Found CPU APIC ID 0 ACPI ID 34: disabled MADT: Found CPU APIC ID 0 ACPI ID 35: disabled MADT: Found CPU APIC ID 0 ACPI ID 36: disabled MADT: Found CPU APIC ID 0 ACPI ID 37: disabled MADT: Found CPU APIC ID 0 ACPI ID 38: disabled MADT: Found CPU APIC ID 0 ACPI ID 39: disabled MADT: Found CPU APIC ID 0 ACPI ID 40: disabled MADT: Found CPU APIC ID 0 ACPI ID 41: disabled MADT: Found CPU APIC ID 0 ACPI ID 42: disabled MADT: Found CPU APIC ID 0 ACPI ID 43: disabled MADT: Found CPU APIC ID 0 ACPI ID 44: disabled MADT: Found CPU APIC ID 0 ACPI ID 45: disabled MADT: Found CPU APIC ID 0 ACPI ID 46: disabled MADT: Found CPU APIC ID 0 ACPI ID 47: disabled MADT: Found CPU APIC ID 0 ACPI ID 48: disabled MADT: Found CPU APIC ID 0 ACPI ID 49: disabled MADT: Found CPU APIC ID 0 ACPI ID 50: disabled MADT: Found CPU APIC ID 0 ACPI ID 51: disabled MADT: Found CPU APIC ID 0 ACPI ID 52: disabled MADT: Found CPU APIC ID 0 ACPI ID 53: disabled MADT: Found CPU APIC ID 0 ACPI ID 54: disabled MADT: Found CPU APIC ID 0 ACPI ID 55: disabled MADT: Found CPU APIC ID 0 ACPI ID 56: disabled MADT: Found CPU APIC ID 0 ACPI ID 57: disabled MADT: Found CPU APIC ID 0 ACPI ID 58: disabled MADT: Found CPU APIC ID 0 ACPI ID 59: disabled MADT: Found CPU APIC ID 0 ACPI
