Rusty Russell wrote: > On Wed, 2007-04-11 at 07:26 +0300, Avi Kivity wrote: > >> Nope. Being async is critical for copyless networking: >> >> - in the transmit path, so need to stop the sender (guest) from touching >> the memory until it's on the wire. This means 100% of packets sent will >> be blocked. >> > > Hi Avi, > > You keep saying stuff like this, and I keep ignoring it. OK, I'll > bite: > > Why would we try to prevent the sender from altering the packets? > >
To avoid data corruption. The guest wants to send a packet. It calls write(), which causes an skb to be allocated, data to be copied into it, the entire networking stack gets into gear, and the guest-side driver instructs the "device" to send the packet. With async operations, the saga continues like this: the host-side driver allocates an skb, get_page()s and attaches the data to the new skb, this skb crosses the bridge, trickles into the real ethernet device, gets queued there, sent, interrupts fire, triggering async completion. On this completion, we send a virtual interrupt to the guest, which tells it to destroy the skb and reclaim the pages attached to it. Without async operations, we don't have a hook to notify the guest when to reclaim the skb. If we do it too soon, the skb can be reclaimed and the memory reused before the real device gets to see it, so we end up sending data that we did not intend. The only way to avoid it is to copy the data somewhere safe, but that is exactly what we don't want to do. >> - multiple packets per operation (for interrupt mitigation) (like >> lio_listio) >> > > The benefits for interrupt mitigation are less clear to me in a virtual > environment (scheduling tends to make it happen anyway); I'd want to > benchmark it. > > Yes, the guest will probably submit multiple packets in one hypercall. It would be nice for the userspace driver to be able to submit them to the host kernel in one syscall. > Some kind of batching to reduce syscall overhead, perhaps, but TSO would > go a fair way towards that anyway (probably not enough). > > For some workloads, sure. >> - scatter/gather packets (iovecs) >> > > Yes, and this is already present in the tap device. Anthony suggested a > slightly nasty hack for multiple sg packets in one writev()/readv, which > could also give us batching. > > No need for hacks if we get list aio support one day. >> - configurable wakeup (by packet count/timeout) for queue management >> > > I'm not convinced that this is a showstopper, though. > It probably isn't. It's free with aio though. > >> - hacks (tso) >> > > I'd usually go for a batch interface over TSO, but if the card we're > sending to actually does TSO then TSO will probably win. > Sure, if tso helps a regular host then it should help one that happens to be running a virtual machine. > >> Most of these can be provided by a combination of the pending aio work, >> the pending aio/fd integration, and the not-so-pending tap aio work. As >> the first two are available as patches and the third is limited to the >> tap device, it is not unreasonable to try it out. Maybe it will turn >> out not to be as difficult as I predicted just a few lines above. >> > > Indeed, I don't think we're asking for a revolution a-la VJ-style > channels. But I'm still itching to get back to that, and this might yet > provide an excuse 8) > I'll be happy if this can be made to work. It will make the paravirt guest-side driver work in kvm-less setups, which are useful for testing, and of course reduction in kernel code is beneficial. It will be slower that in-kernel, but if we get the batching right, perhaps not significantly slower. I'm mostly concerned that this depends on code that has eluded merging for such a long time. -- error compiling committee.c: too many arguments to function ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
