Re: [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
On Fri, Sep 08, 2023 at 01:03:26PM +0100, Alex Bennée wrote: > > Stefan Hajnoczi writes: > > > On Fri, Sep 01, 2023 at 12:00:18PM +0100, Alex Bennée wrote: > >> Currently QEMU has to know some details about the VirtIO device > >> supported by a vhost-user daemon to be able to setup the guest. This > >> makes it hard for QEMU to add support for additional vhost-user > >> daemons without adding specific stubs for each additional VirtIO > >> device. > >> > >> This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > >> which the back-end can advertise which allows a probe message to be > >> sent to get all the details QEMU needs to know in one message. > >> > >> Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > >> VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > >> daemons which are capable of handling all aspects of the VirtIO > >> transactions with only a generic stub on the QEMU side. These daemons > >> can also be used without QEMU in situations where there isn't a full > >> VMM managing their setup. > >> > >> Signed-off-by: Alex Bennée > > > > I think the mindset for this change should be "vhost-user is becoming a > > VIRTIO Transport". VIRTIO Transports have a reasonably well-defined > > feature set in the VIRTIO specification. The goal should be to cover > > every VIRTIO Transport operation via vhost-user protocol messages so > > that the VIRTIO device model can be fully conveyed over vhost-user. > > Is it though? The transport is a guest visible construct whereas > vhost-user is purely a backend implementation detail that should be > invisible to the guest. No, the transport is not necessarily guest-visible. The vhost-user model is that the front-end emulates a VIRTIO device and some aspects of that device are delegated to the vhost-user back-end. In other words, the vhost-user device is not the same as the VIRTIO device that the guest sees, but it's still important for the vhost-user back-end to be a VIRTIO Transport because that's how we can be sure it supports the VIRTIO device model properly. > > Also the various backends do things a different set of ways. The > differences between MMIO and PCI are mostly around where config space is > and how IRQs are handled. For CCW we do actually have a set of commands > we can look at: > > #define CCW_CMD_SET_VQ 0x13 > #define CCW_CMD_VDEV_RESET 0x33 > #define CCW_CMD_SET_IND 0x43 > #define CCW_CMD_SET_CONF_IND 0x53 > #define CCW_CMD_SET_IND_ADAPTER 0x73 > #define CCW_CMD_READ_FEAT 0x12 > #define CCW_CMD_WRITE_FEAT 0x11 > #define CCW_CMD_READ_CONF 0x22 > #define CCW_CMD_WRITE_CONF 0x21 > #define CCW_CMD_WRITE_STATUS 0x31 > #define CCW_CMD_READ_VQ_CONF 0x32 > #define CCW_CMD_SET_VIRTIO_REV 0x83 > #define CCW_CMD_READ_STATUS 0x72 > > which I think we already have mappings for. Yes, there are differences between the transports. vhost-user uses eventfds (callfd/kickfd) instead of interrupts. > > Anything less is yet another ad-hoc protocol extension that will lead to > > more bugs and hacks when it turns out some VIRTIO devices cannot be > > expressed due to limitations in the protocol. > > I agree we want to do this right. > > > This requires going through the VIRTIO spec to find a correspondence > > between virtio-pci/virtio-mmio/virtio-ccw's interfaces and vhost-user > > protocol messages. In most cases vhost-user already offers messages and > > your patch adds more of what is missing. I think this effort is already > > very close but missing the final check that it really matches the VIRTIO > > spec. > > > > Please do the comparison against the VIRTIO Transports and then adjust > > this patch to make it clear that the back-end is becoming a full-fledged > > VIRTIO Transport: > > - The name of the patch series should reflect that. > > - The vhost-user protocol feature should be named F_TRANSPORT. > > - The messages added in this patch should have a 1:1 correspondence with > > the VIRTIO spec including using the same terminology for consistency. > > > > Sorry for the hassle, but I think this is a really crucial point where > > we have the chance to make vhost-user work smoothly in the future...but > > only if we can faithfully expose VIRTIO Transport semantics. > > I wonder if first be handled by cleaning up the VirtIO spec to make it > clear what capabilities each transport needs to support? It's a fair point that the VIRTIO spec does not provide an interface definition for the VIRTIO Transport or at least a definitive list of requirements. The requirements are implicit (i.e. it is assumed that very transport provides a way to set the virtqueue descriptor table addresses) so it's necessary to review the existing transports to understand their functionality. If you want to create a list of the requirements for a VIRTIO Transport and propose a patch to the VIRTIO spec then that would be great, but I don't think that stops this patch series. It's
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
A QEMU built-in VIRTIO device will also call virtio_add_queue() for the maximum number of virtqueues. I'm not sure what the concern is about adding as few virtqueues as possible? If the front-end's implementation is inefficient, then it should be optimized so that untouched virtqueues don't consume resources. I don't see the need to try to add a special message to vhost-user to try to reduce the number of virtqueues. Stefan On Fri, 8 Sept 2023 at 08:03, Alex Bennée wrote: > > > Stefan Hajnoczi writes: > > > On Fri, 8 Sept 2023 at 02:43, Alex Bennée wrote: > >> > >> > >> Stefan Hajnoczi writes: > >> > >> > On Tue, Sep 05, 2023 at 10:34:11AM +0100, Alex Bennée wrote: > >> >> > >> >> Albert Esteve writes: > >> >> > >> >> > This looks great! Thanks for this proposal. > >> >> > > >> >> > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée > >> >> > wrote: > >> >> > > >> >> > Currently QEMU has to know some details about the VirtIO device > >> >> > supported by a vhost-user daemon to be able to setup the guest. This > >> >> > makes it hard for QEMU to add support for additional vhost-user > >> >> > daemons without adding specific stubs for each additional VirtIO > >> >> > device. > >> >> > > >> >> > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > >> >> > which the back-end can advertise which allows a probe message to be > >> >> > sent to get all the details QEMU needs to know in one message. > >> >> > > >> >> > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > >> >> > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > >> >> > daemons which are capable of handling all aspects of the VirtIO > >> >> > transactions with only a generic stub on the QEMU side. These daemons > >> >> > can also be used without QEMU in situations where there isn't a full > >> >> > VMM managing their setup. > >> >> > > >> >> > Signed-off-by: Alex Bennée > >> >> > > >> >> > --- > >> >> > v2 > >> >> >- dropped F_STANDALONE in favour of F_PROBE > >> >> >- split probe details across several messages > >> >> >- probe messages don't automatically imply a standalone daemon > >> >> >- add wording where probe details interact (F_MQ/F_CONFIG) > >> >> >- define VMM and make clear QEMU is only one of many potential VMMs > >> >> >- reword commit message > >> >> > --- > >> >> > docs/interop/vhost-user.rst | 90 > >> >> > - > >> >> > hw/virtio/vhost-user.c | 8 > >> >> > 2 files changed, 88 insertions(+), 10 deletions(-) > >> >> > > >> >> > diff --git a/docs/interop/vhost-user.rst > >> >> > b/docs/interop/vhost-user.rst > >> >> > index 5a070adbc1..ba3b5e07b7 100644 > >> >> > --- a/docs/interop/vhost-user.rst > >> >> > +++ b/docs/interop/vhost-user.rst > >> >> > @@ -7,6 +7,7 @@ Vhost-user Protocol > >> >> > .. > >> >> > Copyright 2014 Virtual Open Systems Sarl. > >> >> > Copyright 2019 Intel Corporation > >> >> > + Copyright 2023 Linaro Ltd > >> >> > Licence: This work is licensed under the terms of the GNU GPL, > >> >> > version 2 or later. See the COPYING file in the top-level > >> >> > directory. > >> >> > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the > >> >> > communication, *front-end* and > >> >> > *back-end*. The *front-end* is the application that shares its > >> >> > virtqueues, in > >> >> > our case QEMU. The *back-end* is the consumer of the virtqueues. > >> >> > > >> >> > -In the current implementation QEMU is the *front-end*, and the > >> >> > *back-end* > >> >> > -is the external process consuming the virtio queues, for example a > >> >> > -software Ethernet switch running in user space, such as Snabbswitch, > >> >> > -or a block device back-end processing read & write to a virtual > >> >> > -disk. In order to facilitate interoperability between various > >> >> > back-end > >> >> > -implementations, it is recommended to follow the :ref:`Backend > >> >> > program > >> >> > -conventions `. > >> >> > +In the current implementation a Virtual Machine Manager (VMM) such > >> >> > as > >> >> > +QEMU is the *front-end*, and the *back-end* is the external process > >> >> > +consuming the virtio queues, for example a software Ethernet switch > >> >> > +running in user space, such as Snabbswitch, or a block device > >> >> > back-end > >> >> > +processing read & write to a virtual disk. In order to facilitate > >> >> > +interoperability between various back-end implementations, it is > >> >> > +recommended to follow the :ref:`Backend program conventions > >> >> > +`. > >> >> > > >> >> > The *front-end* and *back-end* can be either a client (i.e. > >> >> > connecting) or > >> >> > server (listening) in the socket communication. > >> >> > > >> >> > +Probing device details > >> >> > +-- > >> >> > + > >> >> > +Traditionally the vhost-user daemon *back-end* shares configuration > >> >> >
Re: [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Stefan Hajnoczi writes: > On Fri, Sep 01, 2023 at 12:00:18PM +0100, Alex Bennée wrote: >> Currently QEMU has to know some details about the VirtIO device >> supported by a vhost-user daemon to be able to setup the guest. This >> makes it hard for QEMU to add support for additional vhost-user >> daemons without adding specific stubs for each additional VirtIO >> device. >> >> This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) >> which the back-end can advertise which allows a probe message to be >> sent to get all the details QEMU needs to know in one message. >> >> Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and >> VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user >> daemons which are capable of handling all aspects of the VirtIO >> transactions with only a generic stub on the QEMU side. These daemons >> can also be used without QEMU in situations where there isn't a full >> VMM managing their setup. >> >> Signed-off-by: Alex Bennée > > I think the mindset for this change should be "vhost-user is becoming a > VIRTIO Transport". VIRTIO Transports have a reasonably well-defined > feature set in the VIRTIO specification. The goal should be to cover > every VIRTIO Transport operation via vhost-user protocol messages so > that the VIRTIO device model can be fully conveyed over vhost-user. Is it though? The transport is a guest visible construct whereas vhost-user is purely a backend implementation detail that should be invisible to the guest. Also the various backends do things a different set of ways. The differences between MMIO and PCI are mostly around where config space is and how IRQs are handled. For CCW we do actually have a set of commands we can look at: #define CCW_CMD_SET_VQ 0x13 #define CCW_CMD_VDEV_RESET 0x33 #define CCW_CMD_SET_IND 0x43 #define CCW_CMD_SET_CONF_IND 0x53 #define CCW_CMD_SET_IND_ADAPTER 0x73 #define CCW_CMD_READ_FEAT 0x12 #define CCW_CMD_WRITE_FEAT 0x11 #define CCW_CMD_READ_CONF 0x22 #define CCW_CMD_WRITE_CONF 0x21 #define CCW_CMD_WRITE_STATUS 0x31 #define CCW_CMD_READ_VQ_CONF 0x32 #define CCW_CMD_SET_VIRTIO_REV 0x83 #define CCW_CMD_READ_STATUS 0x72 which I think we already have mappings for. > Anything less is yet another ad-hoc protocol extension that will lead to > more bugs and hacks when it turns out some VIRTIO devices cannot be > expressed due to limitations in the protocol. I agree we want to do this right. > This requires going through the VIRTIO spec to find a correspondence > between virtio-pci/virtio-mmio/virtio-ccw's interfaces and vhost-user > protocol messages. In most cases vhost-user already offers messages and > your patch adds more of what is missing. I think this effort is already > very close but missing the final check that it really matches the VIRTIO > spec. > > Please do the comparison against the VIRTIO Transports and then adjust > this patch to make it clear that the back-end is becoming a full-fledged > VIRTIO Transport: > - The name of the patch series should reflect that. > - The vhost-user protocol feature should be named F_TRANSPORT. > - The messages added in this patch should have a 1:1 correspondence with > the VIRTIO spec including using the same terminology for consistency. > > Sorry for the hassle, but I think this is a really crucial point where > we have the chance to make vhost-user work smoothly in the future...but > only if we can faithfully expose VIRTIO Transport semantics. I wonder if first be handled by cleaning up the VirtIO spec to make it clear what capabilities each transport needs to support? >> --- >> v2 >> - dropped F_STANDALONE in favour of F_PROBE >> - split probe details across several messages >> - probe messages don't automatically imply a standalone daemon >> - add wording where probe details interact (F_MQ/F_CONFIG) >> - define VMM and make clear QEMU is only one of many potential VMMs >> - reword commit message >> --- >> docs/interop/vhost-user.rst | 90 - >> hw/virtio/vhost-user.c | 8 >> 2 files changed, 88 insertions(+), 10 deletions(-) >> >> diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst >> index 5a070adbc1..ba3b5e07b7 100644 >> --- a/docs/interop/vhost-user.rst >> +++ b/docs/interop/vhost-user.rst >> @@ -7,6 +7,7 @@ Vhost-user Protocol >> .. >>Copyright 2014 Virtual Open Systems Sarl. >>Copyright 2019 Intel Corporation >> + Copyright 2023 Linaro Ltd >>Licence: This work is licensed under the terms of the GNU GPL, >> version 2 or later. See the COPYING file in the top-level >> directory. >> @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, >> *front-end* and >> *back-end*. The *front-end* is the application that shares its virtqueues, >> in >> our case QEMU. The *back-end* is the consumer of the virtqueues. >> >> -In the current implementation QEMU is the
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Stefan Hajnoczi writes: > On Fri, 8 Sept 2023 at 02:43, Alex Bennée wrote: >> >> >> Stefan Hajnoczi writes: >> >> > On Tue, Sep 05, 2023 at 10:34:11AM +0100, Alex Bennée wrote: >> >> >> >> Albert Esteve writes: >> >> >> >> > This looks great! Thanks for this proposal. >> >> > >> >> > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée >> >> > wrote: >> >> > >> >> > Currently QEMU has to know some details about the VirtIO device >> >> > supported by a vhost-user daemon to be able to setup the guest. This >> >> > makes it hard for QEMU to add support for additional vhost-user >> >> > daemons without adding specific stubs for each additional VirtIO >> >> > device. >> >> > >> >> > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) >> >> > which the back-end can advertise which allows a probe message to be >> >> > sent to get all the details QEMU needs to know in one message. >> >> > >> >> > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and >> >> > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user >> >> > daemons which are capable of handling all aspects of the VirtIO >> >> > transactions with only a generic stub on the QEMU side. These daemons >> >> > can also be used without QEMU in situations where there isn't a full >> >> > VMM managing their setup. >> >> > >> >> > Signed-off-by: Alex Bennée >> >> > >> >> > --- >> >> > v2 >> >> >- dropped F_STANDALONE in favour of F_PROBE >> >> >- split probe details across several messages >> >> >- probe messages don't automatically imply a standalone daemon >> >> >- add wording where probe details interact (F_MQ/F_CONFIG) >> >> >- define VMM and make clear QEMU is only one of many potential VMMs >> >> >- reword commit message >> >> > --- >> >> > docs/interop/vhost-user.rst | 90 - >> >> > hw/virtio/vhost-user.c | 8 >> >> > 2 files changed, 88 insertions(+), 10 deletions(-) >> >> > >> >> > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst >> >> > index 5a070adbc1..ba3b5e07b7 100644 >> >> > --- a/docs/interop/vhost-user.rst >> >> > +++ b/docs/interop/vhost-user.rst >> >> > @@ -7,6 +7,7 @@ Vhost-user Protocol >> >> > .. >> >> > Copyright 2014 Virtual Open Systems Sarl. >> >> > Copyright 2019 Intel Corporation >> >> > + Copyright 2023 Linaro Ltd >> >> > Licence: This work is licensed under the terms of the GNU GPL, >> >> > version 2 or later. See the COPYING file in the top-level >> >> > directory. >> >> > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, >> >> > *front-end* and >> >> > *back-end*. The *front-end* is the application that shares its >> >> > virtqueues, in >> >> > our case QEMU. The *back-end* is the consumer of the virtqueues. >> >> > >> >> > -In the current implementation QEMU is the *front-end*, and the >> >> > *back-end* >> >> > -is the external process consuming the virtio queues, for example a >> >> > -software Ethernet switch running in user space, such as Snabbswitch, >> >> > -or a block device back-end processing read & write to a virtual >> >> > -disk. In order to facilitate interoperability between various back-end >> >> > -implementations, it is recommended to follow the :ref:`Backend program >> >> > -conventions `. >> >> > +In the current implementation a Virtual Machine Manager (VMM) such as >> >> > +QEMU is the *front-end*, and the *back-end* is the external process >> >> > +consuming the virtio queues, for example a software Ethernet switch >> >> > +running in user space, such as Snabbswitch, or a block device back-end >> >> > +processing read & write to a virtual disk. In order to facilitate >> >> > +interoperability between various back-end implementations, it is >> >> > +recommended to follow the :ref:`Backend program conventions >> >> > +`. >> >> > >> >> > The *front-end* and *back-end* can be either a client (i.e. >> >> > connecting) or >> >> > server (listening) in the socket communication. >> >> > >> >> > +Probing device details >> >> > +-- >> >> > + >> >> > +Traditionally the vhost-user daemon *back-end* shares configuration >> >> > +responsibilities with the VMM *front-end* which needs to know certain >> >> > +key bits of information about the device. This means the VMM needs to >> >> > +define at least a minimal stub for each VirtIO device it wants to >> >> > +support. If the daemon supports the right set of protocol features the >> >> > +VMM can probe the daemon for the information it needs to setup the >> >> > +device. See :ref:`Probing features for standalone daemons >> >> > +` for more details. >> >> > + >> >> > + >> >> > Support for platforms other than Linux >> >> > -- >> >> > >> >> > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: >> >> > * ``VHOST_USER_GET_VRING_BASE`` >> >>
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
On Fri, 8 Sept 2023 at 02:43, Alex Bennée wrote: > > > Stefan Hajnoczi writes: > > > On Tue, Sep 05, 2023 at 10:34:11AM +0100, Alex Bennée wrote: > >> > >> Albert Esteve writes: > >> > >> > This looks great! Thanks for this proposal. > >> > > >> > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée > >> > wrote: > >> > > >> > Currently QEMU has to know some details about the VirtIO device > >> > supported by a vhost-user daemon to be able to setup the guest. This > >> > makes it hard for QEMU to add support for additional vhost-user > >> > daemons without adding specific stubs for each additional VirtIO > >> > device. > >> > > >> > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > >> > which the back-end can advertise which allows a probe message to be > >> > sent to get all the details QEMU needs to know in one message. > >> > > >> > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > >> > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > >> > daemons which are capable of handling all aspects of the VirtIO > >> > transactions with only a generic stub on the QEMU side. These daemons > >> > can also be used without QEMU in situations where there isn't a full > >> > VMM managing their setup. > >> > > >> > Signed-off-by: Alex Bennée > >> > > >> > --- > >> > v2 > >> >- dropped F_STANDALONE in favour of F_PROBE > >> >- split probe details across several messages > >> >- probe messages don't automatically imply a standalone daemon > >> >- add wording where probe details interact (F_MQ/F_CONFIG) > >> >- define VMM and make clear QEMU is only one of many potential VMMs > >> >- reword commit message > >> > --- > >> > docs/interop/vhost-user.rst | 90 - > >> > hw/virtio/vhost-user.c | 8 > >> > 2 files changed, 88 insertions(+), 10 deletions(-) > >> > > >> > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > >> > index 5a070adbc1..ba3b5e07b7 100644 > >> > --- a/docs/interop/vhost-user.rst > >> > +++ b/docs/interop/vhost-user.rst > >> > @@ -7,6 +7,7 @@ Vhost-user Protocol > >> > .. > >> > Copyright 2014 Virtual Open Systems Sarl. > >> > Copyright 2019 Intel Corporation > >> > + Copyright 2023 Linaro Ltd > >> > Licence: This work is licensed under the terms of the GNU GPL, > >> > version 2 or later. See the COPYING file in the top-level > >> > directory. > >> > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > >> > *front-end* and > >> > *back-end*. The *front-end* is the application that shares its > >> > virtqueues, in > >> > our case QEMU. The *back-end* is the consumer of the virtqueues. > >> > > >> > -In the current implementation QEMU is the *front-end*, and the > >> > *back-end* > >> > -is the external process consuming the virtio queues, for example a > >> > -software Ethernet switch running in user space, such as Snabbswitch, > >> > -or a block device back-end processing read & write to a virtual > >> > -disk. In order to facilitate interoperability between various back-end > >> > -implementations, it is recommended to follow the :ref:`Backend program > >> > -conventions `. > >> > +In the current implementation a Virtual Machine Manager (VMM) such as > >> > +QEMU is the *front-end*, and the *back-end* is the external process > >> > +consuming the virtio queues, for example a software Ethernet switch > >> > +running in user space, such as Snabbswitch, or a block device back-end > >> > +processing read & write to a virtual disk. In order to facilitate > >> > +interoperability between various back-end implementations, it is > >> > +recommended to follow the :ref:`Backend program conventions > >> > +`. > >> > > >> > The *front-end* and *back-end* can be either a client (i.e. > >> > connecting) or > >> > server (listening) in the socket communication. > >> > > >> > +Probing device details > >> > +-- > >> > + > >> > +Traditionally the vhost-user daemon *back-end* shares configuration > >> > +responsibilities with the VMM *front-end* which needs to know certain > >> > +key bits of information about the device. This means the VMM needs to > >> > +define at least a minimal stub for each VirtIO device it wants to > >> > +support. If the daemon supports the right set of protocol features the > >> > +VMM can probe the daemon for the information it needs to setup the > >> > +device. See :ref:`Probing features for standalone daemons > >> > +` for more details. > >> > + > >> > + > >> > Support for platforms other than Linux > >> > -- > >> > > >> > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: > >> > * ``VHOST_USER_GET_VRING_BASE`` > >> > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) > >> > * ``VHOST_USER_GET_INFLIGHT_FD`` (if > >> >
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Stefan Hajnoczi writes: > On Tue, Sep 05, 2023 at 10:34:11AM +0100, Alex Bennée wrote: >> >> Albert Esteve writes: >> >> > This looks great! Thanks for this proposal. >> > >> > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée wrote: >> > >> > Currently QEMU has to know some details about the VirtIO device >> > supported by a vhost-user daemon to be able to setup the guest. This >> > makes it hard for QEMU to add support for additional vhost-user >> > daemons without adding specific stubs for each additional VirtIO >> > device. >> > >> > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) >> > which the back-end can advertise which allows a probe message to be >> > sent to get all the details QEMU needs to know in one message. >> > >> > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and >> > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user >> > daemons which are capable of handling all aspects of the VirtIO >> > transactions with only a generic stub on the QEMU side. These daemons >> > can also be used without QEMU in situations where there isn't a full >> > VMM managing their setup. >> > >> > Signed-off-by: Alex Bennée >> > >> > --- >> > v2 >> >- dropped F_STANDALONE in favour of F_PROBE >> >- split probe details across several messages >> >- probe messages don't automatically imply a standalone daemon >> >- add wording where probe details interact (F_MQ/F_CONFIG) >> >- define VMM and make clear QEMU is only one of many potential VMMs >> >- reword commit message >> > --- >> > docs/interop/vhost-user.rst | 90 - >> > hw/virtio/vhost-user.c | 8 >> > 2 files changed, 88 insertions(+), 10 deletions(-) >> > >> > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst >> > index 5a070adbc1..ba3b5e07b7 100644 >> > --- a/docs/interop/vhost-user.rst >> > +++ b/docs/interop/vhost-user.rst >> > @@ -7,6 +7,7 @@ Vhost-user Protocol >> > .. >> > Copyright 2014 Virtual Open Systems Sarl. >> > Copyright 2019 Intel Corporation >> > + Copyright 2023 Linaro Ltd >> > Licence: This work is licensed under the terms of the GNU GPL, >> > version 2 or later. See the COPYING file in the top-level >> > directory. >> > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, >> > *front-end* and >> > *back-end*. The *front-end* is the application that shares its >> > virtqueues, in >> > our case QEMU. The *back-end* is the consumer of the virtqueues. >> > >> > -In the current implementation QEMU is the *front-end*, and the *back-end* >> > -is the external process consuming the virtio queues, for example a >> > -software Ethernet switch running in user space, such as Snabbswitch, >> > -or a block device back-end processing read & write to a virtual >> > -disk. In order to facilitate interoperability between various back-end >> > -implementations, it is recommended to follow the :ref:`Backend program >> > -conventions `. >> > +In the current implementation a Virtual Machine Manager (VMM) such as >> > +QEMU is the *front-end*, and the *back-end* is the external process >> > +consuming the virtio queues, for example a software Ethernet switch >> > +running in user space, such as Snabbswitch, or a block device back-end >> > +processing read & write to a virtual disk. In order to facilitate >> > +interoperability between various back-end implementations, it is >> > +recommended to follow the :ref:`Backend program conventions >> > +`. >> > >> > The *front-end* and *back-end* can be either a client (i.e. connecting) >> > or >> > server (listening) in the socket communication. >> > >> > +Probing device details >> > +-- >> > + >> > +Traditionally the vhost-user daemon *back-end* shares configuration >> > +responsibilities with the VMM *front-end* which needs to know certain >> > +key bits of information about the device. This means the VMM needs to >> > +define at least a minimal stub for each VirtIO device it wants to >> > +support. If the daemon supports the right set of protocol features the >> > +VMM can probe the daemon for the information it needs to setup the >> > +device. See :ref:`Probing features for standalone daemons >> > +` for more details. >> > + >> > + >> > Support for platforms other than Linux >> > -- >> > >> > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: >> > * ``VHOST_USER_GET_VRING_BASE`` >> > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) >> > * ``VHOST_USER_GET_INFLIGHT_FD`` (if >> > ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) >> > +* ``VHOST_USER_GET_BACKEND_SPECS`` (if >> > ``VHOST_USER_PROTOCOL_F_STANDALONE``) >> > >> > .. seealso:: >> > >> > @@ -396,9 +412,10 @@ must support changing some configuration aspects on >> > the fly. >> >
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
On Tue, Sep 05, 2023 at 10:34:11AM +0100, Alex Bennée wrote: > > Albert Esteve writes: > > > This looks great! Thanks for this proposal. > > > > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée wrote: > > > > Currently QEMU has to know some details about the VirtIO device > > supported by a vhost-user daemon to be able to setup the guest. This > > makes it hard for QEMU to add support for additional vhost-user > > daemons without adding specific stubs for each additional VirtIO > > device. > > > > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > > which the back-end can advertise which allows a probe message to be > > sent to get all the details QEMU needs to know in one message. > > > > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > > daemons which are capable of handling all aspects of the VirtIO > > transactions with only a generic stub on the QEMU side. These daemons > > can also be used without QEMU in situations where there isn't a full > > VMM managing their setup. > > > > Signed-off-by: Alex Bennée > > > > --- > > v2 > >- dropped F_STANDALONE in favour of F_PROBE > >- split probe details across several messages > >- probe messages don't automatically imply a standalone daemon > >- add wording where probe details interact (F_MQ/F_CONFIG) > >- define VMM and make clear QEMU is only one of many potential VMMs > >- reword commit message > > --- > > docs/interop/vhost-user.rst | 90 - > > hw/virtio/vhost-user.c | 8 > > 2 files changed, 88 insertions(+), 10 deletions(-) > > > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > > index 5a070adbc1..ba3b5e07b7 100644 > > --- a/docs/interop/vhost-user.rst > > +++ b/docs/interop/vhost-user.rst > > @@ -7,6 +7,7 @@ Vhost-user Protocol > > .. > > Copyright 2014 Virtual Open Systems Sarl. > > Copyright 2019 Intel Corporation > > + Copyright 2023 Linaro Ltd > > Licence: This work is licensed under the terms of the GNU GPL, > > version 2 or later. See the COPYING file in the top-level > > directory. > > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > > *front-end* and > > *back-end*. The *front-end* is the application that shares its > > virtqueues, in > > our case QEMU. The *back-end* is the consumer of the virtqueues. > > > > -In the current implementation QEMU is the *front-end*, and the *back-end* > > -is the external process consuming the virtio queues, for example a > > -software Ethernet switch running in user space, such as Snabbswitch, > > -or a block device back-end processing read & write to a virtual > > -disk. In order to facilitate interoperability between various back-end > > -implementations, it is recommended to follow the :ref:`Backend program > > -conventions `. > > +In the current implementation a Virtual Machine Manager (VMM) such as > > +QEMU is the *front-end*, and the *back-end* is the external process > > +consuming the virtio queues, for example a software Ethernet switch > > +running in user space, such as Snabbswitch, or a block device back-end > > +processing read & write to a virtual disk. In order to facilitate > > +interoperability between various back-end implementations, it is > > +recommended to follow the :ref:`Backend program conventions > > +`. > > > > The *front-end* and *back-end* can be either a client (i.e. connecting) or > > server (listening) in the socket communication. > > > > +Probing device details > > +-- > > + > > +Traditionally the vhost-user daemon *back-end* shares configuration > > +responsibilities with the VMM *front-end* which needs to know certain > > +key bits of information about the device. This means the VMM needs to > > +define at least a minimal stub for each VirtIO device it wants to > > +support. If the daemon supports the right set of protocol features the > > +VMM can probe the daemon for the information it needs to setup the > > +device. See :ref:`Probing features for standalone daemons > > +` for more details. > > + > > + > > Support for platforms other than Linux > > -- > > > > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: > > * ``VHOST_USER_GET_VRING_BASE`` > > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) > > * ``VHOST_USER_GET_INFLIGHT_FD`` (if > > ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) > > +* ``VHOST_USER_GET_BACKEND_SPECS`` (if > > ``VHOST_USER_PROTOCOL_F_STANDALONE``) > > > > .. seealso:: > > > > @@ -396,9 +412,10 @@ must support changing some configuration aspects on > > the fly. > > Multiple queue support > > -- > > > > -Many devices have a fixed number of virtqueues. In this case the > > front-end > >
Re: [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
On Fri, Sep 01, 2023 at 12:00:18PM +0100, Alex Bennée wrote: > Currently QEMU has to know some details about the VirtIO device > supported by a vhost-user daemon to be able to setup the guest. This > makes it hard for QEMU to add support for additional vhost-user > daemons without adding specific stubs for each additional VirtIO > device. > > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > which the back-end can advertise which allows a probe message to be > sent to get all the details QEMU needs to know in one message. > > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > daemons which are capable of handling all aspects of the VirtIO > transactions with only a generic stub on the QEMU side. These daemons > can also be used without QEMU in situations where there isn't a full > VMM managing their setup. > > Signed-off-by: Alex Bennée I think the mindset for this change should be "vhost-user is becoming a VIRTIO Transport". VIRTIO Transports have a reasonably well-defined feature set in the VIRTIO specification. The goal should be to cover every VIRTIO Transport operation via vhost-user protocol messages so that the VIRTIO device model can be fully conveyed over vhost-user. Anything less is yet another ad-hoc protocol extension that will lead to more bugs and hacks when it turns out some VIRTIO devices cannot be expressed due to limitations in the protocol. This requires going through the VIRTIO spec to find a correspondence between virtio-pci/virtio-mmio/virtio-ccw's interfaces and vhost-user protocol messages. In most cases vhost-user already offers messages and your patch adds more of what is missing. I think this effort is already very close but missing the final check that it really matches the VIRTIO spec. Please do the comparison against the VIRTIO Transports and then adjust this patch to make it clear that the back-end is becoming a full-fledged VIRTIO Transport: - The name of the patch series should reflect that. - The vhost-user protocol feature should be named F_TRANSPORT. - The messages added in this patch should have a 1:1 correspondence with the VIRTIO spec including using the same terminology for consistency. Sorry for the hassle, but I think this is a really crucial point where we have the chance to make vhost-user work smoothly in the future...but only if we can faithfully expose VIRTIO Transport semantics. > > --- > v2 > - dropped F_STANDALONE in favour of F_PROBE > - split probe details across several messages > - probe messages don't automatically imply a standalone daemon > - add wording where probe details interact (F_MQ/F_CONFIG) > - define VMM and make clear QEMU is only one of many potential VMMs > - reword commit message > --- > docs/interop/vhost-user.rst | 90 - > hw/virtio/vhost-user.c | 8 > 2 files changed, 88 insertions(+), 10 deletions(-) > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > index 5a070adbc1..ba3b5e07b7 100644 > --- a/docs/interop/vhost-user.rst > +++ b/docs/interop/vhost-user.rst > @@ -7,6 +7,7 @@ Vhost-user Protocol > .. >Copyright 2014 Virtual Open Systems Sarl. >Copyright 2019 Intel Corporation > + Copyright 2023 Linaro Ltd >Licence: This work is licensed under the terms of the GNU GPL, > version 2 or later. See the COPYING file in the top-level > directory. > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > *front-end* and > *back-end*. The *front-end* is the application that shares its virtqueues, in > our case QEMU. The *back-end* is the consumer of the virtqueues. > > -In the current implementation QEMU is the *front-end*, and the *back-end* > -is the external process consuming the virtio queues, for example a > -software Ethernet switch running in user space, such as Snabbswitch, > -or a block device back-end processing read & write to a virtual > -disk. In order to facilitate interoperability between various back-end > -implementations, it is recommended to follow the :ref:`Backend program > -conventions `. > +In the current implementation a Virtual Machine Manager (VMM) such as > +QEMU is the *front-end*, and the *back-end* is the external process > +consuming the virtio queues, for example a software Ethernet switch > +running in user space, such as Snabbswitch, or a block device back-end > +processing read & write to a virtual disk. In order to facilitate > +interoperability between various back-end implementations, it is > +recommended to follow the :ref:`Backend program conventions > +`. > > The *front-end* and *back-end* can be either a client (i.e. connecting) or > server (listening) in the socket communication. > > +Probing device details > +-- > + > +Traditionally the vhost-user daemon *back-end* shares configuration > +responsibilities with
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
On Tue, Sep 5, 2023 at 11:43 AM Alex Bennée wrote: > > Albert Esteve writes: > > > This looks great! Thanks for this proposal. > > > > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée > wrote: > > > > Currently QEMU has to know some details about the VirtIO device > > supported by a vhost-user daemon to be able to setup the guest. This > > makes it hard for QEMU to add support for additional vhost-user > > daemons without adding specific stubs for each additional VirtIO > > device. > > > > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > > which the back-end can advertise which allows a probe message to be > > sent to get all the details QEMU needs to know in one message. > > > > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > > daemons which are capable of handling all aspects of the VirtIO > > transactions with only a generic stub on the QEMU side. These daemons > > can also be used without QEMU in situations where there isn't a full > > VMM managing their setup. > > > > Signed-off-by: Alex Bennée > > > > --- > > v2 > >- dropped F_STANDALONE in favour of F_PROBE > >- split probe details across several messages > >- probe messages don't automatically imply a standalone daemon > >- add wording where probe details interact (F_MQ/F_CONFIG) > >- define VMM and make clear QEMU is only one of many potential VMMs > >- reword commit message > > --- > > docs/interop/vhost-user.rst | 90 - > > hw/virtio/vhost-user.c | 8 > > 2 files changed, 88 insertions(+), 10 deletions(-) > > > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > > index 5a070adbc1..ba3b5e07b7 100644 > > --- a/docs/interop/vhost-user.rst > > +++ b/docs/interop/vhost-user.rst > > @@ -7,6 +7,7 @@ Vhost-user Protocol > > .. > > Copyright 2014 Virtual Open Systems Sarl. > > Copyright 2019 Intel Corporation > > + Copyright 2023 Linaro Ltd > > Licence: This work is licensed under the terms of the GNU GPL, > > version 2 or later. See the COPYING file in the top-level > > directory. > > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > *front-end* and > > *back-end*. The *front-end* is the application that shares its > virtqueues, in > > our case QEMU. The *back-end* is the consumer of the virtqueues. > > > > -In the current implementation QEMU is the *front-end*, and the > *back-end* > > -is the external process consuming the virtio queues, for example a > > -software Ethernet switch running in user space, such as Snabbswitch, > > -or a block device back-end processing read & write to a virtual > > -disk. In order to facilitate interoperability between various back-end > > -implementations, it is recommended to follow the :ref:`Backend program > > -conventions `. > > +In the current implementation a Virtual Machine Manager (VMM) such as > > +QEMU is the *front-end*, and the *back-end* is the external process > > +consuming the virtio queues, for example a software Ethernet switch > > +running in user space, such as Snabbswitch, or a block device back-end > > +processing read & write to a virtual disk. In order to facilitate > > +interoperability between various back-end implementations, it is > > +recommended to follow the :ref:`Backend program conventions > > +`. > > > > The *front-end* and *back-end* can be either a client (i.e. > connecting) or > > server (listening) in the socket communication. > > > > +Probing device details > > +-- > > + > > +Traditionally the vhost-user daemon *back-end* shares configuration > > +responsibilities with the VMM *front-end* which needs to know certain > > +key bits of information about the device. This means the VMM needs to > > +define at least a minimal stub for each VirtIO device it wants to > > +support. If the daemon supports the right set of protocol features the > > +VMM can probe the daemon for the information it needs to setup the > > +device. See :ref:`Probing features for standalone daemons > > +` for more details. > > + > > + > > Support for platforms other than Linux > > -- > > > > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: > > * ``VHOST_USER_GET_VRING_BASE`` > > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) > > * ``VHOST_USER_GET_INFLIGHT_FD`` (if > ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) > > +* ``VHOST_USER_GET_BACKEND_SPECS`` (if > ``VHOST_USER_PROTOCOL_F_STANDALONE``) > > > > .. seealso:: > > > > @@ -396,9 +412,10 @@ must support changing some configuration aspects > on the fly. > > Multiple queue support > > -- > > > > -Many devices have a fixed number of virtqueues. In this case the > front-end > > -already knows the
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Albert Esteve writes: > This looks great! Thanks for this proposal. > > On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée wrote: > > Currently QEMU has to know some details about the VirtIO device > supported by a vhost-user daemon to be able to setup the guest. This > makes it hard for QEMU to add support for additional vhost-user > daemons without adding specific stubs for each additional VirtIO > device. > > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > which the back-end can advertise which allows a probe message to be > sent to get all the details QEMU needs to know in one message. > > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > daemons which are capable of handling all aspects of the VirtIO > transactions with only a generic stub on the QEMU side. These daemons > can also be used without QEMU in situations where there isn't a full > VMM managing their setup. > > Signed-off-by: Alex Bennée > > --- > v2 >- dropped F_STANDALONE in favour of F_PROBE >- split probe details across several messages >- probe messages don't automatically imply a standalone daemon >- add wording where probe details interact (F_MQ/F_CONFIG) >- define VMM and make clear QEMU is only one of many potential VMMs >- reword commit message > --- > docs/interop/vhost-user.rst | 90 - > hw/virtio/vhost-user.c | 8 > 2 files changed, 88 insertions(+), 10 deletions(-) > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > index 5a070adbc1..ba3b5e07b7 100644 > --- a/docs/interop/vhost-user.rst > +++ b/docs/interop/vhost-user.rst > @@ -7,6 +7,7 @@ Vhost-user Protocol > .. > Copyright 2014 Virtual Open Systems Sarl. > Copyright 2019 Intel Corporation > + Copyright 2023 Linaro Ltd > Licence: This work is licensed under the terms of the GNU GPL, > version 2 or later. See the COPYING file in the top-level > directory. > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > *front-end* and > *back-end*. The *front-end* is the application that shares its virtqueues, > in > our case QEMU. The *back-end* is the consumer of the virtqueues. > > -In the current implementation QEMU is the *front-end*, and the *back-end* > -is the external process consuming the virtio queues, for example a > -software Ethernet switch running in user space, such as Snabbswitch, > -or a block device back-end processing read & write to a virtual > -disk. In order to facilitate interoperability between various back-end > -implementations, it is recommended to follow the :ref:`Backend program > -conventions `. > +In the current implementation a Virtual Machine Manager (VMM) such as > +QEMU is the *front-end*, and the *back-end* is the external process > +consuming the virtio queues, for example a software Ethernet switch > +running in user space, such as Snabbswitch, or a block device back-end > +processing read & write to a virtual disk. In order to facilitate > +interoperability between various back-end implementations, it is > +recommended to follow the :ref:`Backend program conventions > +`. > > The *front-end* and *back-end* can be either a client (i.e. connecting) or > server (listening) in the socket communication. > > +Probing device details > +-- > + > +Traditionally the vhost-user daemon *back-end* shares configuration > +responsibilities with the VMM *front-end* which needs to know certain > +key bits of information about the device. This means the VMM needs to > +define at least a minimal stub for each VirtIO device it wants to > +support. If the daemon supports the right set of protocol features the > +VMM can probe the daemon for the information it needs to setup the > +device. See :ref:`Probing features for standalone daemons > +` for more details. > + > + > Support for platforms other than Linux > -- > > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: > * ``VHOST_USER_GET_VRING_BASE`` > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) > * ``VHOST_USER_GET_INFLIGHT_FD`` (if > ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) > +* ``VHOST_USER_GET_BACKEND_SPECS`` (if ``VHOST_USER_PROTOCOL_F_STANDALONE``) > > .. seealso:: > > @@ -396,9 +412,10 @@ must support changing some configuration aspects on the > fly. > Multiple queue support > -- > > -Many devices have a fixed number of virtqueues. In this case the front-end > -already knows the number of available virtqueues without communicating with > the > -back-end. > +Many devices have a fixed number of virtqueues. In this case the > +*front-end* usually already knows the number of available virtqueues > +without communicating with the back-end.
Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
This looks great! Thanks for this proposal. On Fri, Sep 1, 2023 at 1:00 PM Alex Bennée wrote: > Currently QEMU has to know some details about the VirtIO device > supported by a vhost-user daemon to be able to setup the guest. This > makes it hard for QEMU to add support for additional vhost-user > daemons without adding specific stubs for each additional VirtIO > device. > > This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) > which the back-end can advertise which allows a probe message to be > sent to get all the details QEMU needs to know in one message. > > Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and > VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user > daemons which are capable of handling all aspects of the VirtIO > transactions with only a generic stub on the QEMU side. These daemons > can also be used without QEMU in situations where there isn't a full > VMM managing their setup. > > Signed-off-by: Alex Bennée > > --- > v2 > - dropped F_STANDALONE in favour of F_PROBE > - split probe details across several messages > - probe messages don't automatically imply a standalone daemon > - add wording where probe details interact (F_MQ/F_CONFIG) > - define VMM and make clear QEMU is only one of many potential VMMs > - reword commit message > --- > docs/interop/vhost-user.rst | 90 - > hw/virtio/vhost-user.c | 8 > 2 files changed, 88 insertions(+), 10 deletions(-) > > diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst > index 5a070adbc1..ba3b5e07b7 100644 > --- a/docs/interop/vhost-user.rst > +++ b/docs/interop/vhost-user.rst > @@ -7,6 +7,7 @@ Vhost-user Protocol > .. >Copyright 2014 Virtual Open Systems Sarl. >Copyright 2019 Intel Corporation > + Copyright 2023 Linaro Ltd >Licence: This work is licensed under the terms of the GNU GPL, > version 2 or later. See the COPYING file in the top-level > directory. > @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, > *front-end* and > *back-end*. The *front-end* is the application that shares its > virtqueues, in > our case QEMU. The *back-end* is the consumer of the virtqueues. > > -In the current implementation QEMU is the *front-end*, and the *back-end* > -is the external process consuming the virtio queues, for example a > -software Ethernet switch running in user space, such as Snabbswitch, > -or a block device back-end processing read & write to a virtual > -disk. In order to facilitate interoperability between various back-end > -implementations, it is recommended to follow the :ref:`Backend program > -conventions `. > +In the current implementation a Virtual Machine Manager (VMM) such as > +QEMU is the *front-end*, and the *back-end* is the external process > +consuming the virtio queues, for example a software Ethernet switch > +running in user space, such as Snabbswitch, or a block device back-end > +processing read & write to a virtual disk. In order to facilitate > +interoperability between various back-end implementations, it is > +recommended to follow the :ref:`Backend program conventions > +`. > > The *front-end* and *back-end* can be either a client (i.e. connecting) or > server (listening) in the socket communication. > > +Probing device details > +-- > + > +Traditionally the vhost-user daemon *back-end* shares configuration > +responsibilities with the VMM *front-end* which needs to know certain > +key bits of information about the device. This means the VMM needs to > +define at least a minimal stub for each VirtIO device it wants to > +support. If the daemon supports the right set of protocol features the > +VMM can probe the daemon for the information it needs to setup the > +device. See :ref:`Probing features for standalone daemons > +` for more details. > + > + > Support for platforms other than Linux > -- > > @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: > * ``VHOST_USER_GET_VRING_BASE`` > * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) > * ``VHOST_USER_GET_INFLIGHT_FD`` (if > ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) > +* ``VHOST_USER_GET_BACKEND_SPECS`` (if > ``VHOST_USER_PROTOCOL_F_STANDALONE``) > > .. seealso:: > > @@ -396,9 +412,10 @@ must support changing some configuration aspects on > the fly. > Multiple queue support > -- > > -Many devices have a fixed number of virtqueues. In this case the > front-end > -already knows the number of available virtqueues without communicating > with the > -back-end. > +Many devices have a fixed number of virtqueues. In this case the > +*front-end* usually already knows the number of available virtqueues > +without communicating with the back-end. For standalone daemons this > +number can be can be probed with the ``VHOST_USER_GET_MIN_VQ`` message. > > Some devices do
[RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Currently QEMU has to know some details about the VirtIO device supported by a vhost-user daemon to be able to setup the guest. This makes it hard for QEMU to add support for additional vhost-user daemons without adding specific stubs for each additional VirtIO device. This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE) which the back-end can advertise which allows a probe message to be sent to get all the details QEMU needs to know in one message. Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user daemons which are capable of handling all aspects of the VirtIO transactions with only a generic stub on the QEMU side. These daemons can also be used without QEMU in situations where there isn't a full VMM managing their setup. Signed-off-by: Alex Bennée --- v2 - dropped F_STANDALONE in favour of F_PROBE - split probe details across several messages - probe messages don't automatically imply a standalone daemon - add wording where probe details interact (F_MQ/F_CONFIG) - define VMM and make clear QEMU is only one of many potential VMMs - reword commit message --- docs/interop/vhost-user.rst | 90 - hw/virtio/vhost-user.c | 8 2 files changed, 88 insertions(+), 10 deletions(-) diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index 5a070adbc1..ba3b5e07b7 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -7,6 +7,7 @@ Vhost-user Protocol .. Copyright 2014 Virtual Open Systems Sarl. Copyright 2019 Intel Corporation + Copyright 2023 Linaro Ltd Licence: This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory. @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, *front-end* and *back-end*. The *front-end* is the application that shares its virtqueues, in our case QEMU. The *back-end* is the consumer of the virtqueues. -In the current implementation QEMU is the *front-end*, and the *back-end* -is the external process consuming the virtio queues, for example a -software Ethernet switch running in user space, such as Snabbswitch, -or a block device back-end processing read & write to a virtual -disk. In order to facilitate interoperability between various back-end -implementations, it is recommended to follow the :ref:`Backend program -conventions `. +In the current implementation a Virtual Machine Manager (VMM) such as +QEMU is the *front-end*, and the *back-end* is the external process +consuming the virtio queues, for example a software Ethernet switch +running in user space, such as Snabbswitch, or a block device back-end +processing read & write to a virtual disk. In order to facilitate +interoperability between various back-end implementations, it is +recommended to follow the :ref:`Backend program conventions +`. The *front-end* and *back-end* can be either a client (i.e. connecting) or server (listening) in the socket communication. +Probing device details +-- + +Traditionally the vhost-user daemon *back-end* shares configuration +responsibilities with the VMM *front-end* which needs to know certain +key bits of information about the device. This means the VMM needs to +define at least a minimal stub for each VirtIO device it wants to +support. If the daemon supports the right set of protocol features the +VMM can probe the daemon for the information it needs to setup the +device. See :ref:`Probing features for standalone daemons +` for more details. + + Support for platforms other than Linux -- @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do: * ``VHOST_USER_GET_VRING_BASE`` * ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``) * ``VHOST_USER_GET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``) +* ``VHOST_USER_GET_BACKEND_SPECS`` (if ``VHOST_USER_PROTOCOL_F_STANDALONE``) .. seealso:: @@ -396,9 +412,10 @@ must support changing some configuration aspects on the fly. Multiple queue support -- -Many devices have a fixed number of virtqueues. In this case the front-end -already knows the number of available virtqueues without communicating with the -back-end. +Many devices have a fixed number of virtqueues. In this case the +*front-end* usually already knows the number of available virtqueues +without communicating with the back-end. For standalone daemons this +number can be can be probed with the ``VHOST_USER_GET_MIN_VQ`` message. Some devices do not have a fixed number of virtqueues. Instead the maximum number of virtqueues is chosen by the back-end. The number can depend on host @@ -885,6 +902,23 @@ Protocol features #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15 #define VHOST_USER_PROTOCOL_F_STATUS 16
