Re: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support
On Thu, Nov 06, 2008 at 04:40:21PM -0600, Anthony Liguori wrote: > We've been talking about avoiding hardware passthrough entirely and > just backing a virtio-net backend driver by a dedicated VF in the > host. That avoids a huge amount of guest-facing complexity, let's > migration Just Work, and should give the same level of performance. I don't believe that it will, and every benchmark I've seen or have done so far shows a significant performance gap between virtio and direct assignment, even on 1G ethernet. I am willing however to reserve judgement until someone implements your suggestion and actually measures it, preferably on 10G ethernet. No doubt device assignment---and SR-IOV in particular---are complex, but I hardly think ignoring it as you seem to propose is the right approach. Cheers, Muli -- The First Workshop on I/O Virtualization (WIOV '08) Dec 2008, San Diego, CA, http://www.usenix.org/wiov08/ <-> SYSTOR 2009---The Israeli Experimental Systems Conference http://www.haifa.il.ibm.com/conferences/systor2009/ -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Preventing conflicts between kernel header versions?
Hi kvm team, I'm tracking Linus.git, kvm.git, and VirtualBox.svn, among other open source projects. I'm finding a conflict between kvm.git and VirtualBox.svn over the version of my linux kernel headers. Specifically, kvm wants the files asm/kvm_para.h (and friends), but apparently looks in /usr/include/asm for those files and can't find them --> because my linux distro installs kernel headers from 2.6.26 in /usr/include/linux and /usr/include/asm. If I make /usr/include/linux and /usr/include/asm symlinks to my Linus.git repository, then kvm is happy and I'm happy -- until I try to build VirtualBox.svn, which is now broken because of those very same symlinks. Because kvm is very fussy about using exactly the correct version of the linux kernel headers, wouldn't it be reasonable to teach kvm.git to look in the right place for those headers instead of in /usr/include? I.e. /lib/modules/2.6.28-rc3/build/ for example. Is there an easier way to fix this problem? Thanks! -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 15/15] Revert "x86: default to reboot via ACPI"
On Friday, 7 of November 2008, Robert Hancock wrote: > Matthew Garrett wrote: > > On Fri, Nov 07, 2008 at 09:01:19AM +0800, Zhao Yakui wrote: > > > >> With the help of KVM I find that the windows will be rebooted by writing > >> RESET_VALUE to RESET_REG I/O port if the RESET_REG_SUP bit is not > >> zero(It indicates whether ACPI reboot is supported). > >> IMO maybe the ACPI reboot is the first choice. If it can't, then it will > >> fall back to other mode. > > > > Hmm. But we're seeing some machines that end up very confused if > > rebooted via ACPI. I guess we need to run Vista on them to find out how > > they behave. What OSI strings did your KVM setup expose? We know that > > Windows changes behaviour under various circumstances depending on which > > OS the firmware requests, so it's almost possible that this is another > > of those cases. > > Given that Windows behavior, this patch seems suspicious: > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=8fd145917fb62368a9b80db59562c20576238f5a > > This patch ignores the RESET_REG_SUP flag and just tries using the reset > register anyway if it thinks it's valid. So we may attempt ACPI reset on > machines which don't indicate it's supported. > > The patch description mentioned that some machines didn't reboot after > S3 suspend without this patch. However, we recently had this patch merged: > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=a68823ee5285e65b51ceb96f8b13a5b4f99a6888 > > Is it possible that the problem fixed there is the true cause of this > reboot after S3 problem? Generally, it is. Should it regarded as -stable material, BTW, or is it already in -stable? Matthew? Thanks, Rafael -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[ kvm-Bugs-2235570 ] 100% cpu usage with KVM-78
Bugs item #2235570, was opened at 2008-11-07 17:58 Message generated for change (Comment added) made by dgym You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2235570&group_id=180599 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Bailey (dgym) Assigned to: Nobody/Anonymous (nobody) Summary: 100% cpu usage with KVM-78 Initial Comment: When I start a guest it consumes 100% CPU on the host, even after it has booted and is sitting idle at a login prompt. The odd thing is that if I then live migrate the guest to another (identical) machine, the problem goes away. The guest continues to run just fine on the new host, and the new host's CPU usage is normal. I have tried the obvious: starting on the other machine and migrating to the first, and even multiple migrations. It is always the same, the qemu-system-x86_64 process sits at 100% unless it was started with -incoming ... Migrating machines every time you start up is not a very convenient work around, so it would be nice to find out what is different between the normal start up and the -incoming start up and fix the former. Versions and settings: KVM: 78 Host Kernel: Vanilla 2.6.25.2 Compiled with: gcc version 4.1.2 CPU: AMD Phenom Guest OS: Linux (have tried a few distros) Guest Kernels: Debian etch, and an OpenVZ 2.6.18 Command line: qemu-system-x86_64 -m 128 -smp 1 -drive file=/dev/drbd0 -vnc :1 Things I have tried which have not worked: Using -nographics. Using SDL graphics. Using -snapshot, and doing a savevm and loadvm. Things I have tried which have worked: Using -no-kvm. I have attached gdb and found the busy thread, here is its backtrace: #0 0x7f06f017ea17 in ioctl () from /lib/libc.so.6 #1 0x0051b423 in kvm_run (kvm=0xa93040, vcpu=0) at libkvm.c:892 #2 0x004f1116 in kvm_cpu_exec (env=) at /opt/setup/kvm-78/qemu/qemu-kvm.c:230 #3 0x004f13e4 in ap_main_loop (_env=) at /opt/setup/kvm-78/qemu/qemu-kvm.c:432 #4 0x7f06f0565135 in start_thread () from /lib/libpthread.so.0 #5 0x7f06f01852ce in clone () from /lib/libc.so.6 #6 0x in ?? () Because this indicates business within the kernel module it is as far as I have got. I will attempt to identify the previous working version, I know I never had this problem with 68, but I haven't yet tried anything in between. -- Comment By: James Bailey (dgym) Date: 2008-11-08 21:35 Message: Using the -no-kvm-pit option fixes the CPU problem. There were some changes to arch/x86/kvm/i8254.c so this definitely looks PIT related. -- Comment By: James Bailey (dgym) Date: 2008-11-08 20:28 Message: I have been able to narrow the problem down to a single commit. KVM-76 was fine, but I get this broken behaviour with KVM-77. I checked out the KVM userspace and went back to 76, and I also checked out the KVM kernel and tried different versions. Commit 666c4a43cba0cbaa30cd3c86b515dfdab2a6fa98 on git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm.git is the first version to show the 100% CPU behaviour. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2235570&group_id=180599 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[ kvm-Bugs-2235570 ] 100% cpu usage with KVM-78
Bugs item #2235570, was opened at 2008-11-07 17:58 Message generated for change (Comment added) made by dgym You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2235570&group_id=180599 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Bailey (dgym) Assigned to: Nobody/Anonymous (nobody) Summary: 100% cpu usage with KVM-78 Initial Comment: When I start a guest it consumes 100% CPU on the host, even after it has booted and is sitting idle at a login prompt. The odd thing is that if I then live migrate the guest to another (identical) machine, the problem goes away. The guest continues to run just fine on the new host, and the new host's CPU usage is normal. I have tried the obvious: starting on the other machine and migrating to the first, and even multiple migrations. It is always the same, the qemu-system-x86_64 process sits at 100% unless it was started with -incoming ... Migrating machines every time you start up is not a very convenient work around, so it would be nice to find out what is different between the normal start up and the -incoming start up and fix the former. Versions and settings: KVM: 78 Host Kernel: Vanilla 2.6.25.2 Compiled with: gcc version 4.1.2 CPU: AMD Phenom Guest OS: Linux (have tried a few distros) Guest Kernels: Debian etch, and an OpenVZ 2.6.18 Command line: qemu-system-x86_64 -m 128 -smp 1 -drive file=/dev/drbd0 -vnc :1 Things I have tried which have not worked: Using -nographics. Using SDL graphics. Using -snapshot, and doing a savevm and loadvm. Things I have tried which have worked: Using -no-kvm. I have attached gdb and found the busy thread, here is its backtrace: #0 0x7f06f017ea17 in ioctl () from /lib/libc.so.6 #1 0x0051b423 in kvm_run (kvm=0xa93040, vcpu=0) at libkvm.c:892 #2 0x004f1116 in kvm_cpu_exec (env=) at /opt/setup/kvm-78/qemu/qemu-kvm.c:230 #3 0x004f13e4 in ap_main_loop (_env=) at /opt/setup/kvm-78/qemu/qemu-kvm.c:432 #4 0x7f06f0565135 in start_thread () from /lib/libpthread.so.0 #5 0x7f06f01852ce in clone () from /lib/libc.so.6 #6 0x in ?? () Because this indicates business within the kernel module it is as far as I have got. I will attempt to identify the previous working version, I know I never had this problem with 68, but I haven't yet tried anything in between. -- Comment By: James Bailey (dgym) Date: 2008-11-08 20:28 Message: I have been able to narrow the problem down to a single commit. KVM-76 was fine, but I get this broken behaviour with KVM-77. I checked out the KVM userspace and went back to 76, and I also checked out the KVM kernel and tried different versions. Commit 666c4a43cba0cbaa30cd3c86b515dfdab2a6fa98 on git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm.git is the first version to show the 100% CPU behaviour. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2235570&group_id=180599 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support
> -Original Message- > From: Fischer, Anna [mailto:[EMAIL PROTECTED] > Sent: Saturday, November 08, 2008 3:10 AM > To: Greg KH; Yu Zhao > Cc: Matthew Wilcox; Anthony Liguori; H L; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; Chiang, Alexander; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; kvm@vger.kernel.org; > [EMAIL PROTECTED]; [EMAIL PROTECTED]; Leonid Grossman; > [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support > > > But would such an api really take advantage of the new IOV interfaces > > that are exposed by the new device type? > > I agree with what Yu says. The idea is to have hardware capabilities to > virtualize a PCI device in a way that those virtual devices can represent > full PCI devices. The advantage of that is that those virtual device can > then be used like any other standard PCI device, meaning we can use > existing > OS tools, configuration mechanism etc. to start working with them. Also, > when > using a virtualization-based system, e.g. Xen or KVM, we do not need > to introduce new mechanisms to make use of SR-IOV, because we can handle > VFs as full PCI devices. > > A virtual PCI device in hardware (a VF) can be as powerful or complex as > you like, or it can be very simple. But the big advantage of SR-IOV is > that hardware presents a complete PCI device to the OS - as opposed to > some resources, or queues, that need specific new configuration and > assignment mechanisms in order to use them with a guest OS (like, for > example, VMDq or similar technologies). > > Anna Ditto. Taking netdev interface as an example - a queue pair is a great way to scale across cpu cores in a single OS image, but it is just not a good way to share device across multiple OS images. The best unit of virtualization is a VF that is implemented as a complete netdev pci device (not a subset of a pci device). This way, native netdev device drivers can work for direct hw access to a VF "as is", and most/all Linux networking features (including VMQ) will work in a guest. Also, guest migration for netdev interfaces (both direct and virtual) can be supported via native Linux mechanism (bonding driver), while Dom0 can retain "veto power" over any guest direct interface operation it deems privileged (vlan, mac address, promisc mode, bandwidth allocation between VFs, etc.). Leonid -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support
> Subject: Re: [PATCH 0/16 v6] PCI: Linux kernel SR-IOV support > Importance: High > > On Fri, Nov 07, 2008 at 11:17:40PM +0800, Yu Zhao wrote: > > While we are arguing what the software model the SR-IOV should be, > let me > > ask two simple questions first: > > > > 1, What does the SR-IOV looks like? > > 2, Why do we need to support it? > > I don't think we need to worry about those questions, as we can see > what > the SR-IOV interface looks like by looking at the PCI spec, and we know > Linux needs to support it, as Linux needs to support everything :) > > (note, community members that can not see the PCI specs at this point > in > time, please know that we are working on resolving these issues, > hopefully we will have some good news within a month or so.) > > > As you know the Linux kernel is the base of various virtual machine > > monitors such as KVM, Xen, OpenVZ and VServer. We need SR-IOV support > in > > the kernel because mostly it helps high-end users (IT departments, > HPC, > > etc.) to share limited hardware resources among hundreds or even > thousands > > virtual machines and hence reduce the cost. How can we make these > virtual > > machine monitors utilize the advantage of SR-IOV without spending too > much > > effort meanwhile remaining architectural correctness? I believe > making VF > > represent as much closer as a normal PCI device (struct pci_dev) is > the > > best way in current situation, because this is not only what the > hardware > > designers expect us to do but also the usage model that KVM, Xen and > other > > VMMs have already supported. > > But would such an api really take advantage of the new IOV interfaces > that are exposed by the new device type? I agree with what Yu says. The idea is to have hardware capabilities to virtualize a PCI device in a way that those virtual devices can represent full PCI devices. The advantage of that is that those virtual device can then be used like any other standard PCI device, meaning we can use existing OS tools, configuration mechanism etc. to start working with them. Also, when using a virtualization-based system, e.g. Xen or KVM, we do not need to introduce new mechanisms to make use of SR-IOV, because we can handle VFs as full PCI devices. A virtual PCI device in hardware (a VF) can be as powerful or complex as you like, or it can be very simple. But the big advantage of SR-IOV is that hardware presents a complete PCI device to the OS - as opposed to some resources, or queues, that need specific new configuration and assignment mechanisms in order to use them with a guest OS (like, for example, VMDq or similar technologies). Anna -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
