Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-10 Thread Shirley Ma
Hello Avi, On Mon, 2010-08-02 at 19:32 +0300, Avi Kivity wrote: When are you seeing gup_fast() fall back to gup()? It should be at most once per page (when a guest starts up none of its pages are mapped, it faults them in on demand). netperf/netserver latency results are pretty good for

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-05 Thread Xin, Xiaohui
: Thursday, July 29, 2010 7:15 PM To: net...@vger.kernel.org; kvm@vger.kernel.org; linux-ker...@vger.kernel.org; m...@redhat.com; mi...@elte.hu; da...@davemloft.net; herb...@gondor.apana.org.au; jd...@linux.intel.com Subject: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net. We

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-04 Thread Arnd Bergmann
On Wednesday 04 August 2010, Dong, Eddie wrote: Arnd Bergmann wrote: On Friday 30 July 2010 17:51:52 Shirley Ma wrote: I think it should be less duplicated code in the kernel if we use macvtap to support what media passthrough driver here. Since macvtap has support virtio_net head and

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-04 Thread Shirley Ma
Hello Eddie, On Wed, 2010-08-04 at 10:06 +0800, Dong, Eddie wrote: But zero-copy is a Linux generic feature that can be used by other VMMs as well if the BE service drivers want to incorporate. If we can make mp device VMM-agnostic (it may be not yet in current patch), that will help Linux

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-03 Thread Xin, Xiaohui
...@linux.intel.com Subject: Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net. Hello Xiaohui, On Thu, 2010-07-29 at 19:14 +0800, xiaohui@intel.com wrote: The idea is simple, just to pin the guest VM user space and then let host NIC driver has the chance to directly DMA

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-03 Thread Shirley Ma
Hello Xiaohui, On Tue, 2010-08-03 at 16:48 +0800, Xin, Xiaohui wrote: May you share me with your performance results (including BW and latency)on vhost-net and how you get them(your configuration and especially with the affinity settings)? My macvtap zero copy is incomplete, I am testing

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-03 Thread Dong, Eddie
Arnd Bergmann wrote: On Friday 30 July 2010 17:51:52 Shirley Ma wrote: On Fri, 2010-07-30 at 16:53 +0800, Xin, Xiaohui wrote: Since vhost-net already supports macvtap/tun backends, do you think whether it's better to implement zero copy in macvtap/tun than inducing a new media passthrough

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Shirley Ma
Hello Avi, On Sun, 2010-08-01 at 11:18 +0300, Avi Kivity wrote: I don't understand. Under what conditions do you use get_user_pages() instead of get_user_pages_fast()? Why? The code always calls get_user_pages_fast, however, the page will be unpinned in skb_free if the same page is not

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Shirley Ma
Hello Michael, On Sun, 2010-08-01 at 11:31 +0300, Michael S. Tsirkin wrote: I think we should explore the idea for the driver to fall back on data copy for small message sizes. The benefit of zero copy would then be CPU utilization on large messages. Yes, we used to have 128 bytes for small

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Shirley Ma
Hello Michael, On Sun, 2010-08-01 at 11:31 +0300, Michael S. Tsirkin wrote: Could you provide an example of a good setup? Specifically, is it a good idea for the vhost thread to inherit CPU affinities from qemu? I need to retest my set up with multi-threads vhost. My previous set up applies

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Avi Kivity
On 08/02/2010 07:01 PM, Shirley Ma wrote: Hello Avi, On Sun, 2010-08-01 at 11:18 +0300, Avi Kivity wrote: I don't understand. Under what conditions do you use get_user_pages() instead of get_user_pages_fast()? Why? The code always calls get_user_pages_fast, however, the page will be

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Shirley Ma
On Mon, 2010-08-02 at 19:11 +0300, Avi Kivity wrote: I don't understand this. gup_fast() only calls gup() if the page is swapped out or read-only. Oh, I used the page as read-only on xmit path. Should I use write instead? Thanks Shirley -- To unsubscribe from this list: send the line

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-02 Thread Avi Kivity
On 08/02/2010 07:25 PM, Shirley Ma wrote: On Mon, 2010-08-02 at 19:11 +0300, Avi Kivity wrote: I don't understand this. gup_fast() only calls gup() if the page is swapped out or read-only. Oh, I used the page as read-only on xmit path. Should I use write instead? No, for xmit getting the

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-01 Thread Avi Kivity
On 07/30/2010 06:46 PM, Shirley Ma wrote: Hello Avi, On Fri, 2010-07-30 at 08:02 +0300, Avi Kivity wrote: get_user_pages() is indeed slow. But what about get_user_pages_fast()? Note that when the page is first touched, get_user_pages_fast() falls back to get_user_pages(), so the latency

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-08-01 Thread Michael S. Tsirkin
On Thu, Jul 29, 2010 at 03:31:22PM -0700, Shirley Ma wrote: I did some vhost performance measurement over 10Gb ixgbe, and found that in order to get consistent BW results, netperf/netserver, qemu, vhost threads smp affinities are required. Could you provide an example of a good setup?

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-31 Thread Arnd Bergmann
On Friday 30 July 2010 17:51:52 Shirley Ma wrote: On Fri, 2010-07-30 at 16:53 +0800, Xin, Xiaohui wrote: Since vhost-net already supports macvtap/tun backends, do you think whether it's better to implement zero copy in macvtap/tun than inducing a new media passthrough device here?

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-30 Thread Xin, Xiaohui
Hello Xiaohui, On Thu, 2010-07-29 at 19:14 +0800, xiaohui@intel.com wrote: The idea is simple, just to pin the guest VM user space and then let host NIC driver has the chance to directly DMA to it. The patches are based on vhost-net backend driver. We add a device which provides proto_ops

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-30 Thread Shirley Ma
Hello Avi, On Fri, 2010-07-30 at 08:02 +0300, Avi Kivity wrote: get_user_pages() is indeed slow. But what about get_user_pages_fast()? Note that when the page is first touched, get_user_pages_fast() falls back to get_user_pages(), so the latency needs to be measured after quite a bit of

RE: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-30 Thread Shirley Ma
Hello Xiaohui, On Fri, 2010-07-30 at 16:53 +0800, Xin, Xiaohui wrote: Since vhost-net already supports macvtap/tun backends, do you think whether it's better to implement zero copy in macvtap/tun than inducing a new media passthrough device here? I'm not sure if there will be more

[RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-29 Thread xiaohui . xin
We provide an zero-copy method which driver side may get external buffers to DMA. Here external means driver don't use kernel space to allocate skb buffers. Currently the external buffer can be from guest virtio-net driver. The idea is simple, just to pin the guest VM user space and then let host

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-29 Thread Shirley Ma
Hello Xiaohui, On Thu, 2010-07-29 at 19:14 +0800, xiaohui@intel.com wrote: The idea is simple, just to pin the guest VM user space and then let host NIC driver has the chance to directly DMA to it. The patches are based on vhost-net backend driver. We add a device which provides

Re: [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.

2010-07-29 Thread Avi Kivity
On 07/30/2010 01:31 AM, Shirley Ma wrote: Our goal is to improve the bandwidth and reduce the CPU usage. Exact performance data will be provided later. I did some vhost performance measurement over 10Gb ixgbe, and found that in order to get consistent BW results, netperf/netserver, qemu,