Re: RE: Some more basic questions..
One more bump:) MW.. On Wed, Jun 4, 2014 at 9:42 PM, Marcus White roastedseawee...@gmail.com wrote: Hello Just a bump to see if anyone can help:) MW On Tue, Jun 3, 2014 at 6:08 PM, Marcus White roastedseawee...@gmail.com wrote: Cant the pages be locked down by the host when it gets the call down for an operation with a buffer(like if a usual user mode process case), and once locked down in the host kernel, the physical address be used directly for DMA, whether send or recieve? Not sure if I fully understand why a copy will be done irrespective. Can you please clarify? Also in the QEMU case, since they are all part of the same process (VM and QEMU), where is the data copy? Apologies if these are basic questions, trying to understand the path here. Thanks in advance. MW On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam venkateswararao.nandi...@citrix.com wrote: I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is realtively zero copy. I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided. Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space. -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Zhang Haoyu Sent: Tuesday, June 03, 2014 4:30 PM To: Venkateswara Rao Nandigam; Marcus White Cc: kvm Subject: Re: RE: Some more basic questions.. A few additional questions:) 1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also. An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively. I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RE: Some more basic questions..
Hello Just a bump to see if anyone can help:) MW On Tue, Jun 3, 2014 at 6:08 PM, Marcus White roastedseawee...@gmail.com wrote: Cant the pages be locked down by the host when it gets the call down for an operation with a buffer(like if a usual user mode process case), and once locked down in the host kernel, the physical address be used directly for DMA, whether send or recieve? Not sure if I fully understand why a copy will be done irrespective. Can you please clarify? Also in the QEMU case, since they are all part of the same process (VM and QEMU), where is the data copy? Apologies if these are basic questions, trying to understand the path here. Thanks in advance. MW On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam venkateswararao.nandi...@citrix.com wrote: I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is realtively zero copy. I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided. Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space. -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Zhang Haoyu Sent: Tuesday, June 03, 2014 4:30 PM To: Venkateswara Rao Nandigam; Marcus White Cc: kvm Subject: Re: RE: Some more basic questions.. A few additional questions:) 1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also. An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively. I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RE: Some more basic questions..
Cant the pages be locked down by the host when it gets the call down for an operation with a buffer(like if a usual user mode process case), and once locked down in the host kernel, the physical address be used directly for DMA, whether send or recieve? Not sure if I fully understand why a copy will be done irrespective. Can you please clarify? Also in the QEMU case, since they are all part of the same process (VM and QEMU), where is the data copy? Apologies if these are basic questions, trying to understand the path here. Thanks in advance. MW On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam venkateswararao.nandi...@citrix.com wrote: I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is realtively zero copy. I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided. Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space. -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Zhang Haoyu Sent: Tuesday, June 03, 2014 4:30 PM To: Venkateswara Rao Nandigam; Marcus White Cc: kvm Subject: Re: RE: Some more basic questions.. A few additional questions:) 1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also. An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively. I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA-HPA and GPA-HPA, or macvtap over SRIOV-VF is used). Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right? -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Some more basic questions..
A few additional questions:) 1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also. 2. A related question to (1), is the QEMU process separate from the VM process or do they all run on the same process as different threads? I did read about IO threads and it seemed like they are all part of the same process, wanted to confirm. Thank you:) On Thu, May 29, 2014 at 11:48 PM, Zhang Haoyu zhan...@sangfor.com wrote: Thanks Zhang and Venkateshwara, some more follow up questions below:) 1. Does -realtime mlock=on allocate all the memory upfront and keep it for the VM, or does it just make sure the memory that is allocated within the guest is not swapped out under host memory pressure? “-realtime mlock=on” will mlockall(MCL_CURRENT | MCL_FUTURE) QEMU's ALL memory, because VM's memory is part of QEMU's, so this option will keep VM's memory unswapped. 2. I notice on a 4G guest on an 8G host, guest allocates only about 1G initially, and the rest later as I start applications. Is there a way for me to reserve ALL memory (4G in this case) upfront somehow without changes to guest and without allocating it? It will have to be something the host OS or some component within the host OS. Isnt there something to that effect? It seems odd that there isnt. On linux, user-process's memory is allocating on demand, the physical memory does not allocate until the virtual memory is touched. Because VM's memory is part of QEMU's, so ... I guess the VM you said above is linux guest. Windows guest will memset its memory during booting period. You can use -realtime mlock=on to reserve VM's ALL memory upfront. Thank you in advance. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Some more basic questions..
On Mon, Jun 2, 2014 at 8:43 PM, Zhang Haoyu zhan...@sangfor.com wrote: A few additional questions:) 1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also. I'm not sure, it perhaps depend on which IO mode you chose, qemu pthreads IO, or linux AIO? Somebody else do a favour, please. That would be definitely good to know:) I didnt know about the modes as such. Can you clarify? Are you talking about vhost-LIO vs Qemu? The LIO part, I do believe, is zero copy.. I have seen pretty high memory usage on QEMU thread, not sure why. 2. A related question to (1), is the QEMU process separate from the VM process or do they all run on the same process as different threads? I did read about IO threads and it seemed like they are all part of the same process, wanted to confirm. Yes, vcpu threads and qemu backend IO threads belong to the same process(qemu). Thank you:) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Some more basic questions..
Thanks Zhang and Venkateshwara, some more follow up questions below:) 1. Does -realtime mlock=on allocate all the memory upfront and keep it for the VM, or does it just make sure the memory that is allocated within the guest is not swapped out under host memory pressure? 2. I notice on a 4G guest on an 8G host, guest allocates only about 1G initially, and the rest later as I start applications. Is there a way for me to reserve ALL memory (4G in this case) upfront somehow without changes to guest and without allocating it? It will have to be something the host OS or some component within the host OS. Isnt there something to that effect? It seems odd that there isnt. Thank you in advance. On Wed, May 28, 2014 at 11:45 PM, Zhang Haoyu zhan...@sangfor.com wrote: Hello, Some more basic questions.. 1. How can I ensure that memory for the a guest is available and reserved? In other words, I bring up a Linux VM which has 4G allocated, I want to make sure it has all the 4G available right away. I saw references to balloon driver, it seemed like that was more for dynamic memory exchange between host and guest. In my case, it is a Linux guest with a Linux VM. Try -realtime mlock=on, this option ensure that all the memory allocated for VM is available right away. 2. Does the host reclaim pages from guest if it needs it without a balloon driver? Yes, just consider the memory of VM as that of QEMU(user-process). 3. This might be a very basic question, please bear with me:) If I use virtio for say network and block, does network and block traffic still go through QEMU? Is the host part of virtio basically QEMU or is it something that runs in the host kernel. If QEMU, does every IO still pass through it? Found some conflicting information, so not a 100% sure. Found this, not sure if it is 100% accurate? Trying to understand the flow through different layers, and what the layers are. http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf It depends on whether you enable the vhost or something. Thank you in Advance:) -- -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Some more basic questions..
Hello, Some more basic questions.. 1. How can I ensure that memory for the a guest is available and reserved? In other words, I bring up a Linux VM which has 4G allocated, I want to make sure it has all the 4G available right away. I saw references to balloon driver, it seemed like that was more for dynamic memory exchange between host and guest. In my case, it is a Linux guest with a Linux VM. 2. Does the host reclaim pages from guest if it needs it without a balloon driver? 3. This might be a very basic question, please bear with me:) If I use virtio for say network and block, does network and block traffic still go through QEMU? Is the host part of virtio basically QEMU or is it something that runs in the host kernel. If QEMU, does every IO still pass through it? Found some conflicting information, so not a 100% sure. Found this, not sure if it is 100% accurate? Trying to understand the flow through different layers, and what the layers are. http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf Thank you in Advance:) -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re:
Hello, I had some basic questions regarding KVM, and would appreciate any help:) I have been reading about the KVM architecture, and as I understand it, the guest shows up as a regular process in the host itself.. I had some questions around that.. 1. Are the guest processes implemented as a control group within the overall VM process itself? Is the VM a kernel process or a user process? User process. 2. Is there a way for me to force some specific CPU/s to a guest, and those CPUs to be not used for any work on the host itself? Pinning is just making sure the vCPU runs on the same physical CPU always, I am looking for something more than that.. Control groups. Do control groups prevent the host from using those CPUs? I want only the VM to use the CPUs, and dont want any host user or kernel threads to run on that physical CPU. I looked up control groups, maybe I missed something there. I will go back and take a look. If you can clarify, I would appreciate it:) 3. If the host is compiled as a non pre-emptible kernel, kernel process run to completion until they give up the CPU themselves. In the context of a guest, I am trying to understand what that would mean in the context of KVM and guest VMs. If the VM is a user process, it means nothing, I wasnt sure as per (1). What problem are you trying to solve? Its more of an investigation at this point to understand what can happen.. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re:
Hello, A friendly bump to see if anyone has any ideas:-) Cheers! On Sun, Apr 13, 2014 at 2:01 PM, Marcus White roastedseawee...@gmail.com wrote: Hello, I had some basic questions regarding KVM, and would appreciate any help:) I have been reading about the KVM architecture, and as I understand it, the guest shows up as a regular process in the host itself.. I had some questions around that.. 1. Are the guest processes implemented as a control group within the overall VM process itself? Is the VM a kernel process or a user process? 2. Is there a way for me to force some specific CPU/s to a guest, and those CPUs to be not used for any work on the host itself? Pinning is just making sure the vCPU runs on the same physical CPU always, I am looking for something more than that.. 3. If the host is compiled as a non pre-emptible kernel, kernel process run to completion until they give up the CPU themselves. In the context of a guest, I am trying to understand what that would mean in the context of KVM and guest VMs. If the VM is a user process, it means nothing, I wasnt sure as per (1). Cheers! M -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[no subject]
Hello, I had some basic questions regarding KVM, and would appreciate any help:) I have been reading about the KVM architecture, and as I understand it, the guest shows up as a regular process in the host itself.. I had some questions around that.. 1. Are the guest processes implemented as a control group within the overall VM process itself? Is the VM a kernel process or a user process? 2. Is there a way for me to force some specific CPU/s to a guest, and those CPUs to be not used for any work on the host itself? Pinning is just making sure the vCPU runs on the same physical CPU always, I am looking for something more than that.. 3. If the host is compiled as a non pre-emptible kernel, kernel process run to completion until they give up the CPU themselves. In the context of a guest, I am trying to understand what that would mean in the context of KVM and guest VMs. If the VM is a user process, it means nothing, I wasnt sure as per (1). Cheers! M -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
