[virtio-dev] Re: [Qemu-devel] [virtio-dev] Re: [virtio-dev] Re: [PATCH v2 00/16] Vhost-pci for inter-VM communication
On 05/23/2017 10:08 AM, Jason Wang wrote: On 2017年05月22日 19:46, Wang, Wei W wrote: On Monday, May 22, 2017 10:28 AM, Jason Wang wrote: On 2017年05月19日 23:33, Stefan Hajnoczi wrote: On Fri, May 19, 2017 at 11:10:33AM +0800, Jason Wang wrote: On 2017年05月18日 11:03, Wei Wang wrote: On 05/17/2017 02:22 PM, Jason Wang wrote: On 2017年05月17日 14:16, Jason Wang wrote: On 2017年05月16日 15:12, Wei Wang wrote: Hi: Care to post the driver codes too? OK. It may take some time to clean up the driver code before post it out. You can first have a check of the draft at the repo here: https://github.com/wei-w-wang/vhost-pci-driver Best, Wei Interesting, looks like there's one copy on tx side. We used to have zerocopy support for tun for VM2VM traffic. Could you please try to compare it with your vhost-pci-net by: We can analyze from the whole data path - from VM1's network stack to send packets -> VM2's network stack to receive packets. The number of copies are actually the same for both. That's why I'm asking you to compare the performance. The only reason for vhost-pci is performance. You should prove it. There is another reason for vhost-pci besides maximum performance: vhost-pci makes it possible for end-users to run networking or storage appliances in compute clouds. Cloud providers do not allow end-users to run custom vhost-user processes on the host so you need vhost-pci. Stefan Then it has non NFV use cases and the question goes back to the performance comparing between vhost-pci and zerocopy vhost_net. If it does not perform better, it was less interesting at least in this case. Probably I can share what we got about vhost-pci and vhost-user: https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost_pci_vs_vhost_user.pdf Right now, I don’t have the environment to add the vhost_net test. Thanks, the number looks good. But I have some questions: - Is the number measured through your vhost-pci kernel driver code? Yes, the kernel driver code. - Have you tested packet size other than 64B? Not yet. - Is zerocopy supported in OVS-dpdk? If yes, is it enabled in your test? zerocopy is not used in the test, but I don't think zerocopy can increase the throughput to 2x. On the other side, we haven't put effort to optimize the draft kernel driver yet. Best, Wei - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
Re: [virtio-dev] Re: [PATCH RFC] virtio-net: enable configurable tx queue size
On 05/23/2017 10:04 AM, Jason Wang wrote: On 2017年05月22日 19:52, Wei Wang wrote: On 05/20/2017 04:42 AM, Michael S. Tsirkin wrote: On Fri, May 19, 2017 at 10:32:19AM +0800, Wei Wang wrote: This patch enables the virtio-net tx queue size to be configurable between 256 (the default queue size) and 1024 by the user. The queue size specified by the user should be power of 2. Setting the tx queue size to be 1024 requires the guest driver to support the VIRTIO_NET_F_MAX_CHAIN_SIZE feature. This should be a generic ring feature, not one specific to virtio net. OK. How about making two more changes below: 1) make the default tx queue size = 1024 (instead of 256). As has been pointed out, you need compat the default value too in this case. The driver gets the size info from the device, then would it cause any compatibility issue if we change the default ring size to 1024 in the vhost case? In other words, is there any software (i.e. any virtio-net driver) functions based on the assumption of 256 queue size? For live migration, the queue size that is being used will also be transferred to the destination. We can reduce the size (to 256) if the MAX_CHAIN_SIZE feature is not supported by the guest. In this way, people who apply the QEMU patch can directly use the largest queue size(1024) without adding the booting command line. 2) The vhost backend does not use writev, so I think when the vhost backed is used, using 1024 queue size should not depend on the MAX_CHAIN_SIZE feature. But do we need to consider even larger queue size now? Need Michael's feedback on this. Meanwhile, I'll get the next version of code ready and check if larger queue size would cause any corner case. Btw, I think it's better to draft a spec patch. I think it should be easier to draft the spec patch when the code is almost done. Best, Wei - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [Qemu-devel] [virtio-dev] Re: [virtio-dev] Re: [PATCH v2 00/16] Vhost-pci for inter-VM communication
On 2017年05月22日 19:46, Wang, Wei W wrote: On Monday, May 22, 2017 10:28 AM, Jason Wang wrote: On 2017年05月19日 23:33, Stefan Hajnoczi wrote: On Fri, May 19, 2017 at 11:10:33AM +0800, Jason Wang wrote: On 2017年05月18日 11:03, Wei Wang wrote: On 05/17/2017 02:22 PM, Jason Wang wrote: On 2017年05月17日 14:16, Jason Wang wrote: On 2017年05月16日 15:12, Wei Wang wrote: Hi: Care to post the driver codes too? OK. It may take some time to clean up the driver code before post it out. You can first have a check of the draft at the repo here: https://github.com/wei-w-wang/vhost-pci-driver Best, Wei Interesting, looks like there's one copy on tx side. We used to have zerocopy support for tun for VM2VM traffic. Could you please try to compare it with your vhost-pci-net by: We can analyze from the whole data path - from VM1's network stack to send packets -> VM2's network stack to receive packets. The number of copies are actually the same for both. That's why I'm asking you to compare the performance. The only reason for vhost-pci is performance. You should prove it. There is another reason for vhost-pci besides maximum performance: vhost-pci makes it possible for end-users to run networking or storage appliances in compute clouds. Cloud providers do not allow end-users to run custom vhost-user processes on the host so you need vhost-pci. Stefan Then it has non NFV use cases and the question goes back to the performance comparing between vhost-pci and zerocopy vhost_net. If it does not perform better, it was less interesting at least in this case. Probably I can share what we got about vhost-pci and vhost-user: https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost_pci_vs_vhost_user.pdf Right now, I don’t have the environment to add the vhost_net test. Thanks, the number looks good. But I have some questions: - Is the number measured through your vhost-pci kernel driver code? - Have you tested packet size other than 64B? - Is zerocopy supported in OVS-dpdk? If yes, is it enabled in your test? Btw, do you have data about vhost_net v.s. vhost_user? I haven't. Thanks Best, Wei - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
Re: [virtio-dev] Re: [PATCH RFC] virtio-net: enable configurable tx queue size
On 2017年05月22日 19:52, Wei Wang wrote: On 05/20/2017 04:42 AM, Michael S. Tsirkin wrote: On Fri, May 19, 2017 at 10:32:19AM +0800, Wei Wang wrote: This patch enables the virtio-net tx queue size to be configurable between 256 (the default queue size) and 1024 by the user. The queue size specified by the user should be power of 2. Setting the tx queue size to be 1024 requires the guest driver to support the VIRTIO_NET_F_MAX_CHAIN_SIZE feature. This should be a generic ring feature, not one specific to virtio net. OK. How about making two more changes below: 1) make the default tx queue size = 1024 (instead of 256). As has been pointed out, you need compat the default value too in this case. We can reduce the size (to 256) if the MAX_CHAIN_SIZE feature is not supported by the guest. In this way, people who apply the QEMU patch can directly use the largest queue size(1024) without adding the booting command line. 2) The vhost backend does not use writev, so I think when the vhost backed is used, using 1024 queue size should not depend on the MAX_CHAIN_SIZE feature. But do we need to consider even larger queue size now? Btw, I think it's better to draft a spec patch. Thanks Best, Wei - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [RFC PATCH kvmtool 00/15] Add virtio-iommu
Hi Bharat, On 22/05/17 09:26, Bharat Bhushan wrote: > Hi Jean, > > I am trying to run and review on my side but I see Linux patches are not with > latest kernel version. > Will it be possible for you to share your Linux and kvmtool git repository > reference? Please find linux and kvmtool patches at the following repos: git://linux-arm.org/kvmtool-jpb.git virtio-iommu/base git://linux-arm.org/linux-jpb.git virtio-iommu/base Note that these branches are unstable, subject to fixes and rebase. I'll try to keep them in sync with upstream. Thanks, Jean - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] RE: [Qemu-devel] [virtio-dev] Re: [virtio-dev] Re: [PATCH v2 00/16] Vhost-pci for inter-VM communication
On Monday, May 22, 2017 10:28 AM, Jason Wang wrote: > On 2017年05月19日 23:33, Stefan Hajnoczi wrote: > > On Fri, May 19, 2017 at 11:10:33AM +0800, Jason Wang wrote: > >> On 2017年05月18日 11:03, Wei Wang wrote: > >>> On 05/17/2017 02:22 PM, Jason Wang wrote: > On 2017年05月17日 14:16, Jason Wang wrote: > > On 2017年05月16日 15:12, Wei Wang wrote: > >>> Hi: > >>> > >>> Care to post the driver codes too? > >>> > >> OK. It may take some time to clean up the driver code before post > >> it out. You can first have a check of the draft at the repo here: > >> https://github.com/wei-w-wang/vhost-pci-driver > >> > >> Best, > >> Wei > > Interesting, looks like there's one copy on tx side. We used to > > have zerocopy support for tun for VM2VM traffic. Could you please > > try to compare it with your vhost-pci-net by: > > > >>> We can analyze from the whole data path - from VM1's network stack > >>> to send packets -> VM2's network stack to receive packets. The > >>> number of copies are actually the same for both. > >> That's why I'm asking you to compare the performance. The only reason > >> for vhost-pci is performance. You should prove it. > > There is another reason for vhost-pci besides maximum performance: > > > > vhost-pci makes it possible for end-users to run networking or storage > > appliances in compute clouds. Cloud providers do not allow end-users > > to run custom vhost-user processes on the host so you need vhost-pci. > > > > Stefan > > Then it has non NFV use cases and the question goes back to the performance > comparing between vhost-pci and zerocopy vhost_net. If it does not perform > better, it was less interesting at least in this case. > Probably I can share what we got about vhost-pci and vhost-user: https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost_pci_vs_vhost_user.pdf Right now, I don’t have the environment to add the vhost_net test. Btw, do you have data about vhost_net v.s. vhost_user? Best, Wei
[virtio-dev] RE: [RFC PATCH kvmtool 00/15] Add virtio-iommu
Hi Jean, I am trying to run and review on my side but I see Linux patches are not with latest kernel version. Will it be possible for you to share your Linux and kvmtool git repository reference? Thanks -Bharat > -Original Message- > From: virtualization-boun...@lists.linux-foundation.org > [mailto:virtualization-boun...@lists.linux-foundation.org] On Behalf Of Jean- > Philippe Brucker > Sent: Saturday, April 08, 2017 12:55 AM > To: io...@lists.linux-foundation.org; k...@vger.kernel.org; > virtualizat...@lists.linux-foundation.org; virtio-dev@lists.oasis-open.org > Cc: cd...@linaro.org; lorenzo.pieral...@arm.com; m...@redhat.com; > marc.zyng...@arm.com; j...@8bytes.org; will.dea...@arm.com; > robin.mur...@arm.com > Subject: [RFC PATCH kvmtool 00/15] Add virtio-iommu > > Implement a virtio-iommu device and translate DMA traffic from vfio and > virtio devices. Virtio needed some rework to support scatter-gather accesses > to vring and buffers at page granularity. Patch 3 implements the actual > virtio- > iommu device. > > Adding --viommu on the command-line now inserts a virtual IOMMU in front > of all virtio and vfio devices: > > $ lkvm run -k Image --console virtio -p console=hvc0 \ > --viommu --vfio 0 --vfio 4 --irqchip gicv3-its > ... > [2.998949] virtio_iommu virtio0: probe successful > [3.007739] virtio_iommu virtio1: probe successful > ... > [3.165023] iommu: Adding device :00:00.0 to group 0 > [3.536480] iommu: Adding device 10200.virtio to group 1 > [3.553643] iommu: Adding device 10600.virtio to group 2 > [3.570687] iommu: Adding device 10800.virtio to group 3 > [3.627425] iommu: Adding device 10a00.virtio to group 4 > [7.823689] iommu: Adding device :00:01.0 to group 5 > ... > > Patches 13 and 14 add debug facilities. Some statistics are gathered for each > address space and can be queried via the debug builtin: > > $ lkvm debug -n guest-1210 --iommu stats > iommu 0 "viommu-vfio" > kicks 1255 > requests 1256 > ioas 1 > maps7 > unmaps 4 > resident2101248 > ioas 6 > maps623 > unmaps 620 > resident16384 > iommu 1 "viommu-virtio" > kicks 11426 > requests 11431 > ioas 2 > maps2836 > unmaps 2835 > resident8192 > accesses2836 > ... > > This is based on the VFIO patchset[1], itself based on Andre's ITS work. > The VFIO bits have only been tested on a software model and are unlikely to > work on actual hardware, but I also tested virtio on an ARM Juno. > > [1] http://www.spinics.net/lists/kvm/msg147624.html > > Jean-Philippe Brucker (15): > virtio: synchronize virtio-iommu headers with Linux > FDT: (re)introduce a dynamic phandle allocator > virtio: add virtio-iommu > Add a simple IOMMU > iommu: describe IOMMU topology in device-trees > irq: register MSI doorbell addresses > virtio: factor virtqueue initialization > virtio: add vIOMMU instance for virtio devices > virtio: access vring and buffers through IOMMU mappings > virtio-pci: translate MSIs with the virtual IOMMU > virtio: set VIRTIO_F_IOMMU_PLATFORM when necessary > vfio: add support for virtual IOMMU > virtio-iommu: debug via IPC > virtio-iommu: implement basic debug commands > virtio: use virtio-iommu when available > > Makefile | 3 + > arm/gic.c | 4 + > arm/include/arm-common/fdt-arch.h | 2 +- > arm/pci.c | 49 ++- > builtin-debug.c | 8 +- > builtin-run.c | 2 + > fdt.c | 35 ++ > include/kvm/builtin-debug.h | 6 + > include/kvm/devices.h | 4 + > include/kvm/fdt.h | 20 + > include/kvm/iommu.h | 105 + > include/kvm/irq.h | 3 + > include/kvm/kvm-config.h | 1 + > include/kvm/vfio.h| 2 + > include/kvm/virtio-iommu.h| 15 + > include/kvm/virtio-mmio.h | 1 + > include/kvm/virtio-pci.h | 2 + > include/kvm/virtio.h | 137 +- > include/linux/virtio_config.h | 74 > include/linux/virtio_ids.h| 4 + > include/linux/virtio_iommu.h | 142 ++ > iommu.c | 240 ++ > irq.c | 35 ++ > kvm-ipc.c | 43 +- > mips/include/kvm/fdt-arch.h | 2 +- > powerpc/include/kvm/fdt-arch.h| 2 +- > vfio.c| 281 +++- > virtio/9p.c | 7 +- >
