Hi On Wed, Apr 19, 2017 at 10:38 AM Wang, Wei W <wei.w.w...@intel.com> wrote:
> Hi, > > We made some design changes to the original vhost-pci design, and want to > open > a discussion about the latest design (labelled 2.0) and its extension > (2.1). > 2.0 design: One VM shares the entire memory of another VM > 2.1 design: One VM uses an intermediate memory shared with another VM for > packet transmission. > > For the convenience of discussion, I have some pictures presented at this > link: > > *https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost-pci-rfc2.0.pdf* > <https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost-pci-rfc2.0.pdf> > > Fig. 1 shows the common driver frame that we want use to build the 2.0 and > 2.1 > design. A TX/RX engine consists of a local ring and an exotic ring. > Isn't "external" (or "remote") more appropriate than "exotic" ? Local ring: > 1) allocated by the driver itself; > 2) registered with the device (i.e. virtio_add_queue()) > Exotic ring: > 1) ring memory comes from the outside (of the driver), and exposed to the > driver > via a BAR MMIO; > 2) does not have a registration in the device, so no ioeventfd/irqfd, > configuration > registers allocated in the device > > Fig. 2 shows how the driver frame is used to build the 2.0 design. > 1) Asymmetric: vhost-pci-net <-> virtio-net > 2) VM1 shares the entire memory of VM2, and the exotic rings are the rings > from VM2. > 3) Performance (in terms of copies between VMs): > TX: 0-copy (packets are put to VM2’s RX ring directly) > RX: 1-copy (the green arrow line in the VM1’s RX engine) > Why is the copy necessary? > Fig. 3 shows how the driver frame is used to build the 2.1 design. > 1) Symmetric: vhost-pci-net <-> vhost-pci-net > 2) Share an intermediate memory, allocated by VM1’s vhost-pci device, > for data exchange, and the exotic rings are built on the shared memory > 3) Performance: > TX: 1-copy > RX: 1-copy > > Fig. 4 shows the inter-VM notification path for 2.0 (2.1 is similar). > The four eventfds are allocated by virtio-net, and shared with > vhost-pci-net: > Uses virtio-net’s TX/RX kickfd as the vhost-pci-net’s RX/TX callfd > Uses virtio-net’s TX/RX callfd as the vhost-pci-net’s RX/TX kickfd > Example of how it works: > After packets are put into vhost-pci-net’s TX, the driver kicks TX, which > causes the an interrupt associated with fd3 to be injected to virtio-net > > The draft code of the 2.0 design is ready, and can be found here: > Qemu: *https://github.com/wei-w-wang/vhost-pci-device* > <https://github.com/wei-w-wang/vhost-pci-device> > The repository contains a single big commit ( https://github.com/wei-w-wang/vhost-pci-device/commit/fa01ec5e41de176197dae505c05b659f5483187f). Please try to provide a seperate patch or series of patch from an upstream commit/release point. > Guest driver: *https://github.com/wei-w-wang/vhost-pci-driver* > <https://github.com/wei-w-wang/vhost-pci-driver> > > We tested the 2.0 implementation using the Spirent packet > generator to transmit 64B packets, the results show that the > throughput of vhost-pci reaches around 1.8Mpps, which is around > two times larger than the legacy OVS+DPDK. Also, vhost-pci shows > better scalability than OVS+DPDK. > > > Best, > Wei > > > > > > > > > > > > > > > > > > > > > > -- Marc-André Lureau
