Re: custom virt-io support (in user-mode-linux)
On 22/05/2019 14:02, Johannes Berg wrote: Hi, While my main interest is mostly in UML right now [1] I've CC'ed the qemu and virtualization lists because something similar might actually apply to other types of virtualization. I'm thinking about adding virt-io support to UML, but the tricky part is that while I want to use the virt-io basics (because it's a nice interface from the 'inside'), I don't actually want the stock drivers that are part of the kernel now (like virtio-net etc.) but rather something that integrates with wifi (probably building on hwsim). The 'inside' interfaces aren't really a problem - just have a specific device ID for this, and then write a normal virtio kernel driver for it. The 'outside' interfaces are where my thinking breaks down right now. Looking at lkl, the outside is just all implemented in lkl as code that gets linked to the library, so in UML terms it'd just be extra 'outside' code like the timer handling or other netdev stuff we have today. Looking at qemu, it's of course also implemented there, and then interfaces with the real network, console abstraction, etc. However, like I said above, I really need something very custom and not likely to make it upstream to any project (because what point is that if you cannot connect to the rest of the environment I'm building), so I'm thinking that perhaps it should be possible to write an abstract 'outside' that lets you interact with it really from out-of-process? Perhaps through some kind of shared memory segment? I think that gets tricky with virt-io doing DMA (I think it does?) though, so that part would have to be implemented directly and not out-of-process? But really that's why I'm asking - is there a better way than to just link the device-side virt-io code into the same binary (be it lkl lib, uml binary, qemu binary)? Thanks, johannes [1] Actually, I've considered using qemu, but it doesn't have virtualized time and doesn't seem to support TSC virtualization. I guess I could remove TSC from the guest CPU and add a virtualized HPET, but I've yet to convince myself this works - on UML I made virtual time as a prototype already: https://patchwork.ozlabs.org/patch/1095814/ (though my real goal isn't to just skip time forward when the host goes idle, it's to sync with other simulated components) ___ linux-um mailing list linux...@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-um I have looked at using virtio semantics in UML in the past around the point when I wanted to make the recvmmsg/sendmmsg vector drivers common in UML and QEMU. It is certainly possible, I went for the native approach at the end though. -- Anton R. Ivanov https://www.kot-begemot.co.uk/ ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: custom virt-io support (in user-mode-linux)
On 22/05/2019 14:46, Johannes Berg wrote:
Hi Anton,
I'm thinking about adding virt-io support to UML, but the tricky part is
that while I want to use the virt-io basics (because it's a nice
interface from the 'inside'), I don't actually want the stock drivers
that are part of the kernel now (like virtio-net etc.) but rather
something that integrates with wifi (probably building on hwsim).
I have looked at using virtio semantics in UML in the past around the
point when I wanted to make the recvmmsg/sendmmsg vector drivers common
in UML and QEMU. It is certainly possible,
I went for the native approach at the end though.
Hmm. I'm not sure what you mean by either :-)
Is there any commonality between the vector drivers?
I was looking purely from a network driver perspective.
I had two options - either do a direct read/write as it does today or
implement the ring/king semantics and read/write from that.
I decided to not bother with the latter and read/write directly from/to
skbs.
I can't see how
that'd work without a bus abstraction (like virtio) in qemu? I mean, the
kernel driver just calls uml_vector_sendmmsg(), which I'd say belongs
more to the 'outside world', but that can't really be done in qemu?
Ok, I guess then I see what you mean by 'native' though.
Similarly, of course, I can implement arbitrary virt-io devices - just
the kernel side doesn't call a function like uml_vector_sendmmsg()
directly, but instead the virt-io model, and the model calls the
function, which essentially is the same just with a (convenient)
abstraction layer.
But this leaves the fundamental fact the model code ("vector_user.c" or
a similar "virtio_user.c") is still part of the build.
I guess what I'm thinking is have something like "virtio_user_rpc.c"
that uses some appropriate RPC to interact with the real model. IOW,
rather than having all the model-specific logic actually be here (like
vector_user.c actually knows how to send network packets over a real
socket fd), try to call out to some RPC that contains the real model.
Now that I thought about it further, I guess my question boils down to
"did anyone ever think about doing RPC for Virt-IO instead of putting
the entire device model into the hypervisor/emulator/...".
Virtio in general no. UML specifically - yes. I have thought of mapping
out all key device calls to RPCs for a few applications. The issue is
that it is fairly difficult to make all of this function cleanly without
blocking in strange places.
You may probably want to look at the UML UBD driver. That is an example
of moving out all processing to an external thread and talking to it via
a request/response API. While it still expects shared memory and needs
access to UML address space the model should be more amenable to
replacing various calls with RPCs as you have now left the rest of the
kernel to run while you are processing the RPC. It also provides you
with RPC completion interrupts, etc as a side effect.
So you basically have UML -> Thread -> RPCs -> Model?
johannes
___
linux-um mailing list
linux...@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: custom virt-io support (in user-mode-linux)
Hi Anton,
> > I'm thinking about adding virt-io support to UML, but the tricky part is
> > that while I want to use the virt-io basics (because it's a nice
> > interface from the 'inside'), I don't actually want the stock drivers
> > that are part of the kernel now (like virtio-net etc.) but rather
> > something that integrates with wifi (probably building on hwsim).
> I have looked at using virtio semantics in UML in the past around the
> point when I wanted to make the recvmmsg/sendmmsg vector drivers common
> in UML and QEMU. It is certainly possible,
>
> I went for the native approach at the end though.
Hmm. I'm not sure what you mean by either :-)
Is there any commonality between the vector drivers? I can't see how
that'd work without a bus abstraction (like virtio) in qemu? I mean, the
kernel driver just calls uml_vector_sendmmsg(), which I'd say belongs
more to the 'outside world', but that can't really be done in qemu?
Ok, I guess then I see what you mean by 'native' though.
Similarly, of course, I can implement arbitrary virt-io devices - just
the kernel side doesn't call a function like uml_vector_sendmmsg()
directly, but instead the virt-io model, and the model calls the
function, which essentially is the same just with a (convenient)
abstraction layer.
But this leaves the fundamental fact the model code ("vector_user.c" or
a similar "virtio_user.c") is still part of the build.
I guess what I'm thinking is have something like "virtio_user_rpc.c"
that uses some appropriate RPC to interact with the real model. IOW,
rather than having all the model-specific logic actually be here (like
vector_user.c actually knows how to send network packets over a real
socket fd), try to call out to some RPC that contains the real model.
Now that I thought about it further, I guess my question boils down to
"did anyone ever think about doing RPC for Virt-IO instead of putting
the entire device model into the hypervisor/emulator/...".
johannes
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
