Re: [RFC] Discuss about an new idea "Vsock over Virtio-net"
On 2018/12/4 12:08, Michael S. Tsirkin wrote: > On Tue, Dec 04, 2018 at 10:21:40AM +0800, jiangyiwen wrote: >> On 2018/12/4 9:31, Michael S. Tsirkin wrote: >>> On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: On 2018/11/30 21:40, Michael S. Tsirkin wrote: > On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: >> >> On 2018/11/30 下午8:55, Jason Wang wrote: >>> >>> On 2018/11/30 下午8:52, Michael S. Tsirkin wrote: >>If you want to compare it with >> something that would be TCP or QUIC. The fundamental >> difference between >> virtio-vsock and e.g. TCP is that TCP operates in a packet >> loss environment. >> So they are using timers for reliability, and receiver is >> always free to >> discard any unacked data. > Virtio-net knows nothing above L2, so they are totally > transparent to device > itself. I still don't get why not using virtio-net instead. > > > Thanks Is your question why is virtio-vsock used instead of TCP on top of IP on top of virtio-net? >>> >>> No, my question is why not do vsock through virtio-net. >>> >>> Thanks >>> >> >> Just to clarify, it's not about vosck over ethernet, and it's not about >> inventing new features or APIs. It's probably something like: >> >> - Let virtio-net driver probe vsock device and do vosck specific things >> if >> needed to share as much codes. >> >> - A new kind of sockfd (which is vsock based) for vhost-net for it to do >> vsock specific things (hopefully it can be transparent). >> >> The change should be totally transparent to userspace applications. >> >> Thanks > > Which code is duplicated between virtio vsock and virtio net right now? > Hi Michael, AFAIK, there is almost no duplicate code between virtio vsock and virtio net now. But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, it has some duplicate codes from virtio net. Based on it, we both think vsock may use virtio net as a transport channel, in this way, vsock can use some of virtio net great features. Thanks, Yiwen. >>> >>> What I would do is just copy some code and show a performance >>> benefit. If that works out it will be clearer which code >>> should be shared. >>> >> >> Hi Michael, >> >> I have already sent a series of patches (VSOCK: support mergeable rx buffer >> in vhost-vsock) >> a month ago, and the performance as follows: >> >> I write a tool to test the vhost-vsock performance, mainly send big >> packet(64K) included guest->Host and Host->Guest. The result as >> follows: >> >> Before performance: >> Single socketMultiple sockets(Max Bandwidth) >> Guest->Host ~400MB/s ~480MB/s >> Host->Guest ~1450MB/s~1600MB/s >> >> After performance: >> Single socketMultiple sockets(Max Bandwidth) >> Guest->Host ~1700MB/s~2900MB/s >> Host->Guest ~1700MB/s~2900MB/s >> >> >From the test results, the performance is improved obviously, and guest >> memory will not be wasted. > > Oh I didn't see that one. Pls CC me in the future. > > Looking at it I agree zero page allocation looks like an issue > but besides that, I think we can merge something similar > and look at refactoring and future extensions later. > > However, any interface change (e.g. a new feature) must be CC'd to one of > virtio lists (subscriber-only). > > Okay, previously I send Virtio-vsock patch only CC stefan and mailing lists based on MAINTAINERS, because it only be related to Virtio-vsock. Then, first I send v2 patch based on Jason's suggestions, and then let's see how to combine with virtio-vsock and virtio-net. What do you think? Thanks, Yiwen. > >> In addition, multiqueue feature I have not implemented it yet. >> >> Thanks, >> Yiwen. >> > > > > > . > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC] Discuss about an new idea "Vsock over Virtio-net"
On Tue, Dec 04, 2018 at 10:21:40AM +0800, jiangyiwen wrote: > On 2018/12/4 9:31, Michael S. Tsirkin wrote: > > On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: > >> On 2018/11/30 21:40, Michael S. Tsirkin wrote: > >>> On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: > > On 2018/11/30 下午8:55, Jason Wang wrote: > > > > On 2018/11/30 下午8:52, Michael S. Tsirkin wrote: > If you want to compare it with > something that would be TCP or QUIC. The fundamental > difference between > virtio-vsock and e.g. TCP is that TCP operates in a packet > loss environment. > So they are using timers for reliability, and receiver is > always free to > discard any unacked data. > >>> Virtio-net knows nothing above L2, so they are totally > >>> transparent to device > >>> itself. I still don't get why not using virtio-net instead. > >>> > >>> > >>> Thanks > >> Is your question why is virtio-vsock used instead of TCP on top of IP > >> on top of virtio-net? > >> > >> > > > > No, my question is why not do vsock through virtio-net. > > > > Thanks > > > > Just to clarify, it's not about vosck over ethernet, and it's not about > inventing new features or APIs. It's probably something like: > > - Let virtio-net driver probe vsock device and do vosck specific things > if > needed to share as much codes. > > - A new kind of sockfd (which is vsock based) for vhost-net for it to do > vsock specific things (hopefully it can be transparent). > > The change should be totally transparent to userspace applications. > > Thanks > >>> > >>> Which code is duplicated between virtio vsock and virtio net right now? > >>> > >> > >> Hi Michael, > >> > >> AFAIK, there is almost no duplicate code between virtio vsock and virtio > >> net now. > >> > >> But, if virtio vsock wants to support mergeable rx buffer and multiqueue > >> feature, > >> it has some duplicate codes from virtio net. Based on it, we both think > >> vsock > >> may use virtio net as a transport channel, in this way, vsock can use some > >> of > >> virtio net great features. > >> > >> Thanks, > >> Yiwen. > > > > What I would do is just copy some code and show a performance > > benefit. If that works out it will be clearer which code > > should be shared. > > > > Hi Michael, > > I have already sent a series of patches (VSOCK: support mergeable rx buffer > in vhost-vsock) > a month ago, and the performance as follows: > > I write a tool to test the vhost-vsock performance, mainly send big > packet(64K) included guest->Host and Host->Guest. The result as > follows: > > Before performance: > Single socketMultiple sockets(Max Bandwidth) > Guest->Host ~400MB/s ~480MB/s > Host->Guest ~1450MB/s~1600MB/s > > After performance: > Single socketMultiple sockets(Max Bandwidth) > Guest->Host ~1700MB/s~2900MB/s > Host->Guest ~1700MB/s~2900MB/s > > >From the test results, the performance is improved obviously, and guest > memory will not be wasted. Oh I didn't see that one. Pls CC me in the future. Looking at it I agree zero page allocation looks like an issue but besides that, I think we can merge something similar and look at refactoring and future extensions later. However, any interface change (e.g. a new feature) must be CC'd to one of virtio lists (subscriber-only). > In addition, multiqueue feature I have not implemented it yet. > > Thanks, > Yiwen. > ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC] Discuss about an new idea "Vsock over Virtio-net"
On 2018/12/4 9:31, Michael S. Tsirkin wrote: > On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: >> On 2018/11/30 21:40, Michael S. Tsirkin wrote: >>> On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: On 2018/11/30 下午8:55, Jason Wang wrote: > > On 2018/11/30 下午8:52, Michael S. Tsirkin wrote: If you want to compare it with something that would be TCP or QUIC. The fundamental difference between virtio-vsock and e.g. TCP is that TCP operates in a packet loss environment. So they are using timers for reliability, and receiver is always free to discard any unacked data. >>> Virtio-net knows nothing above L2, so they are totally >>> transparent to device >>> itself. I still don't get why not using virtio-net instead. >>> >>> >>> Thanks >> Is your question why is virtio-vsock used instead of TCP on top of IP >> on top of virtio-net? >> >> > > No, my question is why not do vsock through virtio-net. > > Thanks > Just to clarify, it's not about vosck over ethernet, and it's not about inventing new features or APIs. It's probably something like: - Let virtio-net driver probe vsock device and do vosck specific things if needed to share as much codes. - A new kind of sockfd (which is vsock based) for vhost-net for it to do vsock specific things (hopefully it can be transparent). The change should be totally transparent to userspace applications. Thanks >>> >>> Which code is duplicated between virtio vsock and virtio net right now? >>> >> >> Hi Michael, >> >> AFAIK, there is almost no duplicate code between virtio vsock and virtio net >> now. >> >> But, if virtio vsock wants to support mergeable rx buffer and multiqueue >> feature, >> it has some duplicate codes from virtio net. Based on it, we both think vsock >> may use virtio net as a transport channel, in this way, vsock can use some of >> virtio net great features. >> >> Thanks, >> Yiwen. > > What I would do is just copy some code and show a performance > benefit. If that works out it will be clearer which code > should be shared. > Hi Michael, I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows: I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows: Before performance: Single socketMultiple sockets(Max Bandwidth) Guest->Host ~400MB/s ~480MB/s Host->Guest ~1450MB/s~1600MB/s After performance: Single socketMultiple sockets(Max Bandwidth) Guest->Host ~1700MB/s~2900MB/s Host->Guest ~1700MB/s~2900MB/s From the test results, the performance is improved obviously, and guest memory will not be wasted. In addition, multiqueue feature I have not implemented it yet. Thanks, Yiwen. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFC] Discuss about an new idea "Vsock over Virtio-net"
On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: > On 2018/11/30 21:40, Michael S. Tsirkin wrote: > > On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: > >> > >> On 2018/11/30 下午8:55, Jason Wang wrote: > >>> > >>> On 2018/11/30 下午8:52, Michael S. Tsirkin wrote: > >>If you want to compare it with > >> something that would be TCP or QUIC. The fundamental > >> difference between > >> virtio-vsock and e.g. TCP is that TCP operates in a packet > >> loss environment. > >> So they are using timers for reliability, and receiver is > >> always free to > >> discard any unacked data. > > Virtio-net knows nothing above L2, so they are totally > > transparent to device > > itself. I still don't get why not using virtio-net instead. > > > > > > Thanks > Is your question why is virtio-vsock used instead of TCP on top of IP > on top of virtio-net? > > > >>> > >>> No, my question is why not do vsock through virtio-net. > >>> > >>> Thanks > >>> > >> > >> Just to clarify, it's not about vosck over ethernet, and it's not about > >> inventing new features or APIs. It's probably something like: > >> > >> - Let virtio-net driver probe vsock device and do vosck specific things if > >> needed to share as much codes. > >> > >> - A new kind of sockfd (which is vsock based) for vhost-net for it to do > >> vsock specific things (hopefully it can be transparent). > >> > >> The change should be totally transparent to userspace applications. > >> > >> Thanks > > > > Which code is duplicated between virtio vsock and virtio net right now? > > > > Hi Michael, > > AFAIK, there is almost no duplicate code between virtio vsock and virtio net > now. > > But, if virtio vsock wants to support mergeable rx buffer and multiqueue > feature, > it has some duplicate codes from virtio net. Based on it, we both think vsock > may use virtio net as a transport channel, in this way, vsock can use some of > virtio net great features. > > Thanks, > Yiwen. What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared. -- MST ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v2] vhost: fix IOTLB locking
From: Jean-Philippe Brucker
Date: Fri, 30 Nov 2018 16:05:53 +
> Commit 78139c94dc8c ("net: vhost: lock the vqs one by one") moved the vq
> lock to improve scalability, but introduced a possible deadlock in
> vhost-iotlb. vhost_iotlb_notify_vq() now takes vq->mutex while holding
> the device's IOTLB spinlock. And on the vhost_iotlb_miss() path, the
> spinlock is taken while holding vq->mutex.
>
> Since calling vhost_poll_queue() doesn't require any lock, avoid the
> deadlock by not taking vq->mutex.
>
> Fixes: 78139c94dc8c ("net: vhost: lock the vqs one by one")
> Acked-by: Jason Wang
> Acked-by: Michael S. Tsirkin
> Signed-off-by: Jean-Philippe Brucker
Applied, thank you.
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v5 0/7] Add virtio-iommu driver
Hi Michael, On 11/27/18 6:16 PM, Michael S. Tsirkin wrote: > On Tue, Nov 27, 2018 at 06:09:25PM +0100, Auger Eric wrote: >> Hi Michael, >> >> On 11/27/18 5:53 PM, Michael S. Tsirkin wrote: >>> On Thu, Nov 22, 2018 at 07:37:54PM +, Jean-Philippe Brucker wrote: Implement the virtio-iommu driver, following specification v0.9 [1]. Since v4 [2] I fixed the issues reported by Eric, and added Reviewed-by from Eric and Rob. Thanks! I changed the specification to fix one inconsistency discussed in v4. That the device fills the probe buffer with zeroes is now a "SHOULD" instead of a "MAY", since it's the only way for the driver to know if the device wrote the status. Existing devices already do this. In addition the device now needs to fill the three padding bytes at the tail with zeroes. You can find Linux driver and kvmtool device on branches virtio-iommu/v0.9 [3]. I also lightly tested with Eric's latest QEMU device [4]. >>> >>> I tried to get this to work on my x86 box but without >>> success. Any hints? Does this have to do with the IORT table? >>> I think we really should just reserve our own table ID >>> and avoid the pain of trying to add things to the IORT spec. >>> I'm reluctant to merge lots of code that I can't easily test. >>> Again, if we found a way to push more configuration into >>> virtio config space the problem space would be smaller. >> >> You can at least test it with QEMU ARM virt in TCG mode. > > It's slow enough that I generally just focus on KVM. > >> Then I have >> worked on the IORT integration in PC/Q35 but this is not yet functional. >> I need to debug what's wrong on guest ACPI probing. I plan to work on >> this this week. >> >> Thanks >> >> Eric > > Sounds good. Did you need to make changes to IORT? I don't remember. If > yes it would be very easy to just have a virtio specific ACPI table - > I am assuming ARM guys will be just as hostile to virt changes > to IORT as they were to minor changes to ARM vIOMMU. I eventually succeeded to prototype the IORT integration on x86. Please find below the branches to use for testing: - QEMU: https://github.com/eauger/qemu.git v3.1.0-rc3-virtio-iommu-v0.9-x86 On top of "[RFC v9 00/17] VIRTIO-IOMMU device", I added the IORT build for pc machine - LINUX GUEST: https://github.com/eauger/linux.git virtio-iommu-v0.9-iort-x86 Jean's virtio-iommu/devel branch + 2 hacks (to be discussed separately) Make sure the CONFIG_VIRTIO_IOMMU config is set Used qemu cmd line featuring -device virtio-iommu-pci,addr=0xa: ./x86_64-softmmu/qemu-system-x86_64 -M q35,accel=kvm,usb=off,sata=off,dump-guest-core=off -cpu Haswell,-hle,-rtm -smp 4,sockets=4,cores=1,threads=1 -m 4G -display none --enable-kvm -serial tcp:localhost:,server -device virtio-iommu-pci,addr=0xa -trace events=/home/augere/UPSTREAM/qemu/hw-virtio-iommu -qmp unix:/home/augere/TEST/QEMU/qmp-sock,server,nowait -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -realtime mlock=off -no-hpet -no-shutdown -device virtio-blk-pci,scsi=off,drive=drv0,id=virtio-disk0,bootindex=1,iommu_platform,disable-modern=off,disable-legacy=on -drive file=/home/augere/VM/IMAGES/vm0.qcow2,format=qcow2,if=none,id=drv0 -device virtio-net-pci,bus=pcie.0,netdev=nic0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on -netdev tap,id=nic0,script=/home/augere/TEST/SCRIPTS/qemu-ifup,downscript=/home/augere/TEST/SCRIPTS/qemu-ifdown -kernel /home/augere/VM/BOOT/vmlinuz-4.20.0-rc3-guest-defconfig-virtio-iommu-0.9-x86+ -initrd /home/augere/VM/BOOT/initrd.img-4.20.0-rc3-guest-defconfig-virtio-iommu-0.9-x86+ -append 'root=/dev/mapper/rhel_dhcp156--238-root ro crashkernel=auto rd.lvm.lv=rhel_dhcp156-238/root rd.lvm.lv=rhel_dhcp156-238/swap rw ignore_loglevel console=tty0 console=ttyS0,115200n8 serial acpi=force' -net none -d guest_errors I put sata=off otherwise I have early untranslated transactions that prevent the guest from booting. Please let me know if you encounter any issue reproducing the environment on your side. Thanks Eric > > >>> [1] Virtio-iommu specification v0.9, sources, pdf and diff from v0.8 git://linux-arm.org/virtio-iommu.git virtio-iommu/v0.9 http://jpbrucker.net/virtio-iommu/spec/v0.9/virtio-iommu-v0.9.pdf http://jpbrucker.net/virtio-iommu/spec/diffs/virtio-iommu-pdf-diff-v0.8-v0.9.pdf [2] [PATCH v4 0/7] Add virtio-iommu driver https://lists.linuxfoundation.org/pipermail/iommu/2018-November/031074.html [3] git://linux-arm.org/linux-jpb.git virtio-iommu/v0.9 git://linux-arm.org/kvmtool-jpb.git virtio-iommu/v0.9 [4] [RFC v9 00/17] VIRTIO-IOMMU device https://www.mail-archive.com/qemu-devel@nongnu.org/msg575578.html Jean-Philippe Brucker (7): dt-bindings: virtio-mmio: Add IOMMU description dt-bindings: virtio: Add virtio-pci-iommu node of: Allow the
