Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
On Friday 21 March 2008 01:11:35 Anthony Liguori wrote: > Rusty Russell wrote: > >There are three possible solutions: > > 1) Just offer the lowest common denominator to both sides (ie. no > > features). This is what I do with lguest in these patches. > > 2) Offer something and handle the case where one Guest accepts and > > another doesn't by emulating it. ie. de-TSO the packets manually. > > 3) "Hot unplug" the device from the guest which asks for the greater > > features, then re-add it offering less features. Requires hotplug in the > > guest OS. > > 4) Add a feature negotiation feature. The feature that gets set is the > "feature negotiate" feature. If a guest doesn't support feature > negotiation, you end up with the least-common denominator (no > features). If both guests support feature negotiation, you can then add > something new to determine the true common subset. Hmm, I discarded that out of hand as too icky, but we might end up there. Analyse features like normal, accept feature negotiation, set DRIVER_OK, wait for config change, if feature negotiation is still set then go around again (presumably some features have been removed). I'll prototype it and see how we go. Thanks, Rusty. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
On Thursday 20 March 2008 17:54:45 Avi Kivity wrote: > Rusty Russell wrote: > > Hi all, > > > >Just finished my prototype of inter-guest virtio, using networking as > > an example. Each guest mmaps the other's address space and uses a FIFO > > for notifications. > > Isn't that a security hole (hole? chasm)? If the two guests can access > each other's memory, they might as well be just one guest, and > communicate internally. Sorry, sloppy language on my part. Each launcher process maps the other guest's memory as well: ie. copying occurs in the host. > My feeling is that the host needs to copy the data, using dma if > available. Another option is to have one guest map the other's memory > for read and write, while the other guest is unprivileged. This allows > one privileged guest to provide services for other, unprivileged guests, > like domain 0 or driver domains in Xen. One having privilege is possible, even trivial with the current patch (it's actually doing a completely generic inter-virtio-ring shuffle). I chose the symmetrical approach for this demo for no particularly good reason. Cheers, Rusty. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Avi Kivity wrote: > > I disagree. A driver domain is shared between multiple guests, and if > one of the guests manages to break into qemu then it can see other > guest's data. You still don't strictly need to do things in the kernel if this is your concern. You can have another process map both guest's address spaces and do the copying on behalf of each guest if you're paranoid about escaping into QEMU. > [Driver domains are a horrible idea IMO, but that's another story] I don't disagree :-) Regards, Anthony Liguori - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Anthony Liguori wrote: > > You can have the file descriptor be opened O_RDONLY so trust isn't an > issue. > Reading is just as bad as writing. >> This implies trusting the other userspace, which is not a good >> thing. Let the kernel copy, we already trust it, and it has more >> resources to do the copy. >> > > You're going to end up with the same trust issues no matter what > unless you let the kernel look directly at the virtio ring queue. > That's the only way to arbitrate what memory gets copied. That's what we need, then. > There may be a generic API here for fast interprocess IO, I don't > know. splice() is a little awkward though for this because you really > don't want to sit in a splice() loop. What you want is for both sides > to be kick'ing the kernel and the kernel to raise an event via > eventfd() or something. > > Absent whatever this kernel API is (which is really just helpful with > a DMA engine), I think the current userspace approach is pretty > reasonable. Not just for interguest IO but also for driver domains > which I think is a logical extension. I disagree. A driver domain is shared between multiple guests, and if one of the guests manages to break into qemu then it can see other guest's data. [Driver domains are a horrible idea IMO, but that's another story] -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Avi Kivity wrote: > Anthony Liguori wrote: >> Avi Kivity wrote: >>> Anthony Liguori wrote: Avi Kivity wrote: Each guest's host userspace mmaps the other guest's address space. The userspace then does a copy on both the tx and rx paths. >>> >>> Well, that's better security-wise (I'd still prefer to avoid it, so >>> we can run each guest under a separate uid), but then we lose >>> performance wise. >> >> What performance win? I'm not sure the copies can be eliminated in >> the case of interguest IO. >> > > I guess not. But at least you can dma instead of busy-copying. > >> Fast interguest IO means mmap()'ing the other guest's address space >> read-only. You can have the file descriptor be opened O_RDONLY so trust isn't an issue. > This implies trusting the other userspace, which is not a good thing. > Let the kernel copy, we already trust it, and it has more resources to > do the copy. > You're going to end up with the same trust issues no matter what unless you let the kernel look directly at the virtio ring queue. That's the only way to arbitrate what memory gets copied. There may be a generic API here for fast interprocess IO, I don't know. splice() is a little awkward though for this because you really don't want to sit in a splice() loop. What you want is for both sides to be kick'ing the kernel and the kernel to raise an event via eventfd() or something. Absent whatever this kernel API is (which is really just helpful with a DMA engine), I think the current userspace approach is pretty reasonable. Not just for interguest IO but also for driver domains which I think is a logical extension. Regards, Anthony Liguori - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Anthony Liguori wrote: > Avi Kivity wrote: >> Anthony Liguori wrote: >>> Avi Kivity wrote: >>> >>> Each guest's host userspace mmaps the other guest's address space. >>> The userspace then does a copy on both the tx and rx paths. >>> >> >> Well, that's better security-wise (I'd still prefer to avoid it, so >> we can run each guest under a separate uid), but then we lose >> performance wise. > > What performance win? I'm not sure the copies can be eliminated in > the case of interguest IO. > I guess not. But at least you can dma instead of busy-copying. > Fast interguest IO means mmap()'ing the other guest's address space > read-only. This implies trusting the other userspace, which is not a good thing. Let the kernel copy, we already trust it, and it has more resources to do the copy. > If you had a pv dma registration api you could conceivably only allow > the active dma entries to be mapped but my fear would be that the > zap'ing on unregister would hurt performance. > Yes, mmu games are costly. They also only work on page granularity which isn't always possible to guarantee. >>> Conceivably, this could be done as a read-only mapping so that each >>> guest userspace copies only the rx packets. That's about as secure >>> as you're going to get with this approach I think. >>> >> >> Maybe we can terminate the virtio queue in the host kernel as a pipe, >> and splice pipes together. >> >> That gives us guest-guest and guest-process communications, and if >> you use aio the kernel can use a dma engine for the copy. > > Ah, so you're looking to use a DMA engine for accelerated copy. > Perhaps the answer is to expose the DMA engine via a userspace API? That's one option, but it still involves sharing all of memory. Splicing pipes might be better. -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Avi Kivity wrote: > Anthony Liguori wrote: >> Avi Kivity wrote: >> >> Each guest's host userspace mmaps the other guest's address space. >> The userspace then does a copy on both the tx and rx paths. >> > > Well, that's better security-wise (I'd still prefer to avoid it, so we > can run each guest under a separate uid), but then we lose performance > wise. What performance win? I'm not sure the copies can be eliminated in the case of interguest IO. Fast interguest IO means mmap()'ing the other guest's address space read-only. If you had a pv dma registration api you could conceivably only allow the active dma entries to be mapped but my fear would be that the zap'ing on unregister would hurt performance. >> Conceivably, this could be done as a read-only mapping so that each >> guest userspace copies only the rx packets. That's about as secure >> as you're going to get with this approach I think. >> > > Maybe we can terminate the virtio queue in the host kernel as a pipe, > and splice pipes together. > > That gives us guest-guest and guest-process communications, and if you > use aio the kernel can use a dma engine for the copy. Ah, so you're looking to use a DMA engine for accelerated copy. Perhaps the answer is to expose the DMA engine via a userspace API? Regards, Anthony Liguori - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Anthony Liguori wrote: > Avi Kivity wrote: >> Rusty Russell wrote: >> >>> Hi all, >>> >>>Just finished my prototype of inter-guest virtio, using >>> networking as an example. Each guest mmaps the other's address >>> space and uses a FIFO for notifications. >>> >>> >> >> Isn't that a security hole (hole? chasm)? If the two guests can >> access each other's memory, they might as well be just one guest, and >> communicate internally. >> > > Each guest's host userspace mmaps the other guest's address space. > The userspace then does a copy on both the tx and rx paths. > Well, that's better security-wise (I'd still prefer to avoid it, so we can run each guest under a separate uid), but then we lose performance wise. > Conceivably, this could be done as a read-only mapping so that each > guest userspace copies only the rx packets. That's about as secure as > you're going to get with this approach I think. > Maybe we can terminate the virtio queue in the host kernel as a pipe, and splice pipes together. That gives us guest-guest and guest-process communications, and if you use aio the kernel can use a dma engine for the copy. -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Rusty Russell wrote: > Hi all, > >Just finished my prototype of inter-guest virtio, using networking as an > example. Each guest mmaps the other's address space and uses a FIFO for > notifications. > >There are two issues with this approach. The first is that neither guest > can change its mappings. See patch 1. Avi mentioned that with MMU notifiers, it may be possible to introduce a new kernel mechanism whereas you could map an arbitrary region of one process's memory into another process. This would address this problem quite nicely. > The second is that our feature > configuration is "host presents, guest chooses" which breaks down when we > don't know the capabilities of each guest. In particular, TSO capability for > networking. >There are three possible solutions: > 1) Just offer the lowest common denominator to both sides (ie. no features). >This is what I do with lguest in these patches. > 2) Offer something and handle the case where one Guest accepts and another >doesn't by emulating it. ie. de-TSO the packets manually. > 3) "Hot unplug" the device from the guest which asks for the greater features, >then re-add it offering less features. Requires hotplug in the guest OS. > 4) Add a feature negotiation feature. The feature that gets set is the "feature negotiate" feature. If a guest doesn't support feature negotiation, you end up with the least-common denominator (no features). If both guests support feature negotiation, you can then add something new to determine the true common subset. > I haven't tuned or even benchmarked these patches, but it pings! > Very nice! It's particularly cool that it was possible entirely in userspace. Regards, Anthony Liguori > Rusty. > > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ > ___ > kvm-devel mailing list > kvm-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/kvm-devel > - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Avi Kivity wrote: > Rusty Russell wrote: > >> Hi all, >> >>Just finished my prototype of inter-guest virtio, using networking as an >> example. Each guest mmaps the other's address space and uses a FIFO for >> notifications. >> >> >> > > Isn't that a security hole (hole? chasm)? If the two guests can access > each other's memory, they might as well be just one guest, and > communicate internally. > Each guest's host userspace mmaps the other guest's address space. The userspace then does a copy on both the tx and rx paths. Conceivably, this could be done as a read-only mapping so that each guest userspace copies only the rx packets. That's about as secure as you're going to get with this approach I think. Regards, Anthony Liguori > My feeling is that the host needs to copy the data, using dma if > available. Another option is to have one guest map the other's memory > for read and write, while the other guest is unprivileged. This allows > one privileged guest to provide services for other, unprivileged guests, > like domain 0 or driver domains in Xen. > > - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Rusty Russell wrote: > Hi all, > >Just finished my prototype of inter-guest virtio, using networking as an > example. Each guest mmaps the other's address space and uses a FIFO for > notifications. > > Isn't that a security hole (hole? chasm)? If the two guests can access each other's memory, they might as well be just one guest, and communicate internally. My feeling is that the host needs to copy the data, using dma if available. Another option is to have one guest map the other's memory for read and write, while the other guest is unprivileged. This allows one privileged guest to provide services for other, unprivileged guests, like domain 0 or driver domains in Xen. -- Any sufficiently difficult bug is indistinguishable from a feature. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] [RFC PATCH 0/4] Inter-guest virtio I/O example with lguest
Hi all, Just finished my prototype of inter-guest virtio, using networking as an example. Each guest mmaps the other's address space and uses a FIFO for notifications. There are two issues with this approach. The first is that neither guest can change its mappings. See patch 1. The second is that our feature configuration is "host presents, guest chooses" which breaks down when we don't know the capabilities of each guest. In particular, TSO capability for networking. There are three possible solutions: 1) Just offer the lowest common denominator to both sides (ie. no features). This is what I do with lguest in these patches. 2) Offer something and handle the case where one Guest accepts and another doesn't by emulating it. ie. de-TSO the packets manually. 3) "Hot unplug" the device from the guest which asks for the greater features, then re-add it offering less features. Requires hotplug in the guest OS. I haven't tuned or even benchmarked these patches, but it pings! Rusty. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
