Re: TODO list for qemu+KVM networking performance v2

2009-06-04 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Thu, Jun 04, 2009 at 01:16:05PM -0400, Gregory Haskins wrote: > >> Michael S. Tsirkin wrote: >> >>> As I'm new to qemu/kvm, to figure out how networking performance can be >>> improved, I >>> went over the co

Re: TODO list for qemu+KVM networking performance v2

2009-06-04 Thread Gregory Haskins
Michael S. Tsirkin wrote: > As I'm new to qemu/kvm, to figure out how networking performance can be > improved, I > went over the code and took some notes. As I did this, I tried to record > ideas > from recent discussions and ideas that came up on improving performance. Thus > this list. > > Th

Re: [PATCHv2 2/2] vhost_net: a kernel-level virtio server

2009-08-11 Thread Gregory Haskins
Michael S. Tsirkin wrote: > What it is: vhost net is a character device that can be used to reduce > the number of system calls involved in virtio networking. > Existing virtio net code is used in the guest without modification. > > There's similarity with vringfd, with some differences and reduce

Re: [PATCHv2 0/2] vhost: a kernel-level virtio server

2009-08-11 Thread Gregory Haskins
Michael S. Tsirkin wrote: > This implements vhost: a kernel-level backend for virtio, > The main motivation for this work is to reduce virtualization > overhead for virtio by removing system calls on data path, > without guest changes. For virtio-net, this removes up to > 4 system calls per packet:

Re: [PATCHv2 0/2] vhost: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Tue, Aug 11, 2009 at 07:49:37PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> This implements vhost: a kernel-level backend for virtio, >>> The main motivation for this work is to reduce virtualization >>> overhead

Re: [PATCHv2 0/2] vhost: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 07:56:05AM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> >>> 1. use a dedicated network interface with SRIOV, program mac to match >>>that of guest (for testing, you can set prom

Re: [PATCHv2 2/2] vhost_net: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Tue, Aug 11, 2009 at 08:06:02PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> What it is: vhost net is a character device that can be used to reduce >>> the number of system calls involved in virtio networking. >>>

Re: [PATCHv2 2/2] vhost_net: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 09:01:35AM -0400, Gregory Haskins wrote: >> I think I understand what your comment above meant: You don't need to >> do synchronize_rcu() because you can flush the workqueue instead to >> ensure that all reader

Re: [PATCHv2 0/2] vhost: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Arnd Bergmann wrote: > On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>> If I understand it correctly, you can at least connect a veth pair >>> to a bridge, right? Something like >>> >>>veth0 - veth1 - vhost - guest 1 >>> eth0 - br0-| >>>veth2 - veth3 - vhost - gue

Re: [PATCHv2 0/2] vhost: a kernel-level virtio server

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 09:51:45AM -0400, Gregory Haskins wrote: >> Arnd Bergmann wrote: >>> On Wednesday 12 August 2009, Michael S. Tsirkin wrote: >>>>> If I understand it correctly, you can at least connect a veth pair >>

Re: [PATCH 0/3] qemu-kvm: vhost net support

2009-08-12 Thread Gregory Haskins
Michael S. Tsirkin wrote: > This adds support for vhost-net virtio kernel backend. > > This is RFC, but works without issues for me. > > Still needs to be split up, tested and benchmarked properly, > but posting it here in case people want to test drive > the kernel bits I posted. This has a lar

Re: [PATCH 0/3] qemu-kvm: vhost net support

2009-08-13 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Aug 12, 2009 at 04:27:44PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> This adds support for vhost-net virtio kernel backend. >>> >>> This is RFC, but works without issues for me. >>> >>&

Re: [kvm-devel] [PATCH 4/5] KVM: Add hypercall queue for paravirt_ops implementation

2007-06-18 Thread Gregory Haskins
On Mon, 2007-06-18 at 15:50 +0300, Avi Kivity wrote: > > These numbers are pretty bad. I'd like to improve them, even without PV. > There's a 20% speedup just waiting to be checked in in the lapic branch ;) (This gain was observed using "make mrproper; make defconfig; time make" on Intel with

Re: [kvm-devel] [PATCH 4/5] KVM: Add hypercall queue for paravirt_ops implementation

2007-06-18 Thread Gregory Haskins
On Mon, 2007-06-18 at 08:19 -0500, Anthony Liguori wrote: > Gregory Haskins wrote: > > On Mon, 2007-06-18 at 15:50 +0300, Avi Kivity wrote: > > > > > >> These numbers are pretty bad. I'd like to improve them, even without PV. > >> > >>

Re: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-16 Thread Gregory Haskins
Hi Rusty, Comments inline... On Fri, 2007-08-17 at 11:25 +1000, Rusty Russell wrote: > > Transport has several parts. What the hypervisor knows about (usually > shared memory and some interrupt mechanism and possibly "DMA") and what > is convention between users (eg. ringbuffer layouts). Whet

Re: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-17 Thread Gregory Haskins
On Fri, 2007-08-17 at 17:43 +1000, Rusty Russell wrote: > Sure, these discussions can get pretty esoteric. The question is > whether you want a point-to-point transport (as we discuss here), or an > N-way. Lguest has N-way, but I'm not convinced it's worthwhile, as > there's some overhead

Re: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-20 Thread Gregory Haskins
On Sun, 2007-08-19 at 12:24 +0300, Avi Kivity wrote: > Rusty Russell wrote: > > 2) We either need huge descriptors or some chaining mechanism to > > handle scatter-gather. > > > > Or, my preference, have a small sglist in the descriptor; Define "small" ;) There a certainly pa

RE: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-20 Thread Gregory Haskins
On Mon, 2007-08-20 at 07:03 -0700, Dor Laor wrote: > >> > 2) We either need huge descriptors or some chaining > >mechanism to > >> > handle scatter-gather. > >> > > >> > >> Or, my preference, have a small sglist in the descriptor; > > > > > >Define "small" ;) > > > >There a certainl

RE: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-21 Thread Gregory Haskins
On Tue, 2007-08-21 at 17:58 +1000, Rusty Russell wrote: > Partly the horror of the code, but mainly because it is an in-order > ring. You'll note that we use a reply ring, so we don't need to know > how much the other side has consumed (and it needn't do so in order). > I have certainly been kn

Re: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-21 Thread Gregory Haskins
On Tue, 2007-08-21 at 15:25 +0300, Avi Kivity wrote: > Gregory Haskins wrote: > > On Tue, 2007-08-21 at 17:58 +1000, Rusty Russell wrote: > > > > > >> Partly the horror of the code, but mainly because it is an in-order > >> ring. You'll note that w

RE: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-21 Thread Gregory Haskins
On Tue, 2007-08-21 at 23:47 +1000, Rusty Russell wrote: > Hi Gregory, > > The main current use is disk drivers: they process out-of-order. Maybe for you ;) I am working on the networking/IVMC side. > > > I think the use of rings for the tx-path in of > > itself is questionable unless

RE: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-21 Thread Gregory Haskins
On Tue, 2007-08-21 at 10:06 -0400, Gregory Haskins wrote: > On Tue, 2007-08-21 at 23:47 +1000, Rusty Russell wrote: > > > > In the guest -> host direction, an interface like virtio is designed > > for batching, with the explicit distinction between add_buf &

Re: [kvm-devel] [PATCH 00/10] PV-IO v3

2007-08-21 Thread Gregory Haskins
On Tue, 2007-08-21 at 20:12 +0300, Avi Kivity wrote: > No, sync() means "make the other side aware that there's work to be done". > Ok, but still the important thing isn't the kick per se, but the resulting completetion. Can we do interrupt driven reclamation? Some of those virtio_net emails I

Re: Use of virtio device IDs

2007-11-07 Thread Gregory Haskins
Anthony Liguori wrote: > > Right now, we would have to have every PCI vendor/device ID pair in the > virtio PCI driver ID table for every virtio device. I realize you guys are probably far down this road in the design process, but FWIW: This is a major motivation for the reason that the IOQ stuf

Re: Use of virtio device IDs

2007-11-07 Thread Gregory Haskins
Avi Kivity wrote: > > I dislike strings. They make it look as if you have a nice extensible > interface, where in reality you have a poorly documented interface which > leads to poor interoperability. Its not really a full fledged interface, but rather just a simple id mechanism. A decentralize

Re: Use of virtio device IDs

2007-11-13 Thread Gregory Haskins
Avi Kivity wrote: > Gregory Haskins wrote: >>> PCI means that you can reuse all of the platform's infrastructure for >>> irq allocation, discovery, device hotplug, and management. >>> >> Its tempting to use, yes. However, most of that infrastructur

Re: [PATCHv5 1/3] mm: export use_mm/unuse_mm to modules

2009-08-28 Thread Gregory Haskins
Michael S. Tsirkin wrote: > vhost net module wants to do copy to/from user from a kernel thread, > which needs use_mm (like what fs/aio has). Move that into mm/ and > export to modules. Michael, Andrew, I am just curious: Is there any technical reason why a kthread cannot have a long-term use_m

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-11 Thread Gregory Haskins
Gregory Haskins wrote: [snip] > > FWIW: VBUS handles this situation via the "memctx" abstraction. IOW, > the memory is not assumed to be a userspace address. Rather, it is a > memctx-specific address, which can be userspace, or any other type > (including hardware, d

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-11 Thread Gregory Haskins
Ira W. Snyder wrote: > On Mon, Sep 07, 2009 at 01:15:37PM +0300, Michael S. Tsirkin wrote: >> On Thu, Sep 03, 2009 at 11:39:45AM -0700, Ira W. Snyder wrote: >>> On Thu, Aug 27, 2009 at 07:07:50PM +0300, Michael S. Tsirkin wrote: What it is: vhost net is a character device that can be used to r

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-14 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Fri, Sep 11, 2009 at 12:00:21PM -0400, Gregory Haskins wrote: >> FWIW: VBUS handles this situation via the "memctx" abstraction. IOW, >> the memory is not assumed to be a userspace address. Rather, it is a >> memctx-specific address,

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-14 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Mon, Sep 14, 2009 at 12:08:55PM -0400, Gregory Haskins wrote: >> For Ira's example, the addresses would represent a physical address on >> the PCI boards, and would follow any kind of relevant rules for >> converting a "GPA" t

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-14 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Mon, Sep 14, 2009 at 12:08:55PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> On Fri, Sep 11, 2009 at 12:00:21PM -0400, Gregory Haskins wrote: >>>> FWIW: VBUS handles this situation via the "memctx" abstracti

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Avi Kivity wrote: > On 09/14/2009 10:14 PM, Gregory Haskins wrote: >> To reiterate, as long as the model is such that the ppc boards are >> considered the "owner" (direct access, no translation needed) I believe >> it will work. If the pointers are expected to b

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Avi Kivity wrote: > On 09/15/2009 04:03 PM, Gregory Haskins wrote: >> >>> In this case the x86 is the owner and the ppc boards use translated >>> access. Just switch drivers and device and it falls into place. >>> >>> >> You could switch

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Avi Kivity wrote: > On 09/15/2009 04:50 PM, Gregory Haskins wrote: >>> Why? vhost will call get_user_pages() or copy_*_user() which ought to >>> do the right thing. >>> >> I was speaking generally, not specifically to Ira's architecture. What &g

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Tue, Sep 15, 2009 at 04:08:23PM -0400, Gregory Haskins wrote: >> No, what I mean is how do you surface multiple ethernet and consoles to >> the guests? For Ira's case, I think he needs at minimum at least one of >> each, and he mentioned

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Tue, Sep 15, 2009 at 04:43:58PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> On Tue, Sep 15, 2009 at 04:08:23PM -0400, Gregory Haskins wrote: >>>> No, what I mean is how do you surface multiple ethernet and consoles

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-15 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Tue, Sep 15, 2009 at 05:39:27PM -0400, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> On Tue, Sep 15, 2009 at 04:43:58PM -0400, Gregory Haskins wrote: >>>> Michael S. Tsirkin wrote: >>>>> On Tue, Sep 15, 20

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-16 Thread Gregory Haskins
Avi Kivity wrote: > On 09/15/2009 11:08 PM, Gregory Haskins wrote: >> >>> There's virtio-console, virtio-blk etc. None of these have kernel-mode >>> servers, but these could be implemented if/when needed. >>> >> IIUC, Ira already

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-16 Thread Gregory Haskins
Avi Kivity wrote: > On 09/16/2009 02:44 PM, Gregory Haskins wrote: >> The problem isn't where to find the models...the problem is how to >> aggregate multiple models to the guest. >> > > You mean configuration? > >>> You instantiate multi

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-16 Thread Gregory Haskins
Avi Kivity wrote: > On 09/16/2009 05:10 PM, Gregory Haskins wrote: >> >>> If kvm can do it, others can. >>> >> The problem is that you seem to either hand-wave over details like this, >> or you give details that are pretty much exactly what vbus d

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-16 Thread Gregory Haskins
Avi Kivity wrote: > On 09/16/2009 10:22 PM, Gregory Haskins wrote: >> Avi Kivity wrote: >> >>> On 09/16/2009 05:10 PM, Gregory Haskins wrote: >>> >>>>> If kvm can do it, others can. >>>>> >>>>> >

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-16 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Sep 16, 2009 at 10:10:55AM -0400, Gregory Haskins wrote: >>> There is no role reversal. >> So if I have virtio-blk driver running on the x86 and vhost-blk device >> running on the ppc board, I can use the ppc board as a block-device. &g

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-23 Thread Gregory Haskins
Avi Kivity wrote: > On 09/22/2009 12:43 AM, Ira W. Snyder wrote: >> >>> Sure, virtio-ira and he is on his own to make a bus-model under that, or >>> virtio-vbus + vbus-ira-connector to use the vbus framework. Either >>> model can work, I agree. >>> >>> >> Yes, I'm having to create my own bus

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-23 Thread Gregory Haskins
Avi Kivity wrote: > On 09/23/2009 05:26 PM, Gregory Haskins wrote: >> >> >>>> Yes, I'm having to create my own bus model, a-la lguest, virtio-pci, >>>> and >>>> virtio-s390. It isn't especially easy. I can steal lots of code fro

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-23 Thread Gregory Haskins
Gregory Haskins wrote: > Avi Kivity wrote: >> On 09/23/2009 05:26 PM, Gregory Haskins wrote: >>> >>>>> Yes, I'm having to create my own bus model, a-la lguest, virtio-pci, >>>>> and >>>>> virtio-s390. It isn't especially

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-23 Thread Gregory Haskins
Avi Kivity wrote: > On 09/23/2009 08:58 PM, Gregory Haskins wrote: >>> >>>> It also pulls parts of the device model into the host kernel. >>>> >>> That is the point. Most of it needs to be there for performance. >>> >>

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-24 Thread Gregory Haskins
Avi Kivity wrote: > On 09/24/2009 12:15 AM, Gregory Haskins wrote: >> >>>> There are various aspects about designing high-performance virtual >>>> devices such as providing the shortest paths possible between the >>>> physical resources and the consum

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-24 Thread Gregory Haskins
Avi Kivity wrote: > On 09/23/2009 10:37 PM, Avi Kivity wrote: >> >> Example: feature negotiation. If it happens in userspace, it's easy >> to limit what features we expose to the guest. If it happens in the >> kernel, we need to add an interface to let the kernel know which >> features it should

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-25 Thread Gregory Haskins
Avi Kivity wrote: > On 09/24/2009 09:03 PM, Gregory Haskins wrote: >> >>> I don't really see how vhost and vbus are different here. vhost expects >>> signalling to happen through a couple of eventfds and requires someone >>> to supply them and implement k

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-09-30 Thread Gregory Haskins
Avi Kivity wrote: > On 09/26/2009 12:32 AM, Gregory Haskins wrote: >>>> >>>> I realize in retrospect that my choice of words above implies vbus _is_ >>>> complete, but this is not what I was saying. What I was trying to >>>> convey is that vbus

Re: [PATCHv5 3/3] vhost_net: a kernel-level virtio server

2009-10-01 Thread Gregory Haskins
Avi Kivity wrote: > On 09/30/2009 10:04 PM, Gregory Haskins wrote: > > >>> A 2.6.27 guest, or Windows guest with the existing virtio drivers, >>> won't work >>> over vbus. >>> >> Binary compatibility with existing virtio drivers,

Re: [PATCHv7 3/3] vhost_net: a kernel-level virtio server

2009-11-03 Thread Gregory Haskins
Gregory Haskins wrote: > Eric Dumazet wrote: >> Michael S. Tsirkin a écrit : >>> +static void handle_tx(struct vhost_net *net) >>> +{ >>> + struct vhost_virtqueue *vq = &net->dev.vqs[VHOST_NET_VQ_TX]; >>> + unsigned head, out, in, s; >

Re: [PATCHv7 3/3] vhost_net: a kernel-level virtio server

2009-11-03 Thread Gregory Haskins
Eric Dumazet wrote: > Michael S. Tsirkin a écrit : >> +static void handle_tx(struct vhost_net *net) >> +{ >> +struct vhost_virtqueue *vq = &net->dev.vqs[VHOST_NET_VQ_TX]; >> +unsigned head, out, in, s; >> +struct msghdr msg = { >> +.msg_name = NULL, >> +.msg_name

Re: [PATCHv7 2/3] mm: export use_mm/unuse_mm to modules

2009-11-03 Thread Gregory Haskins
Michael S. Tsirkin wrote: > vhost net module wants to do copy to/from user from a kernel thread, > which needs use_mm. Export it to modules. > > Acked-by: Andrea Arcangeli > Signed-off-by: Michael S. Tsirkin I need this too: Acked-by: Gregory Haskins > --- > mm

Re: [PATCHv7 3/3] vhost_net: a kernel-level virtio server

2009-11-03 Thread Gregory Haskins
Eric Dumazet wrote: > Gregory Haskins a écrit : >> Gregory Haskins wrote: >>> Eric Dumazet wrote: >>>> Michael S. Tsirkin a écrit : >>>> using rcu_dereference() and mutex_lock() at the same time seems wrong, I >>>> suspect >>>>

Re: [PATCHv8 0/3] vhost: a kernel-level virtio server

2009-11-04 Thread Gregory Haskins
Michael S. Tsirkin wrote: > Ok, I think I've addressed all comments so far here. > Rusty, I'd like this to go into linux-next, through your tree, and > hopefully 2.6.33. What do you think? I think the benchmark data is a prerequisite for merge consideration, IMO. Do you have anything for us to l

Re: [PATCHv7 3/3] vhost_net: a kernel-level virtio server

2009-11-04 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Nov 04, 2009 at 09:25:42AM -0800, Paul E. McKenney wrote: >> (Sorry, but, as always, I could not resist!) >> >> Thanx, Paul > > Thanks Paul! > Jonathan: are you reading this? > Another one for your quotes of t

Re: [PATCHv8 0/3] vhost: a kernel-level virtio server

2009-11-04 Thread Gregory Haskins
Michael S. Tsirkin wrote: > On Wed, Nov 04, 2009 at 11:02:15AM -0500, Gregory Haskins wrote: >> Michael S. Tsirkin wrote: >>> Ok, I think I've addressed all comments so far here. >>> Rusty, I'd like this to go into linux-next, through your tree, and >>&