[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
> On Tue, May 10, 2016 at 09:00:45AM +, Xie, Huawei wrote:
> > On 5/10/2016 4:42 PM, Michael S. Tsirkin wrote:
> > > On Tue, May 10, 2016 at 08:07:00AM +, Xie, Huawei wrote:
> > >> On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote:
> > >>> On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
> > On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> > > On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
> > >> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> > >>> +static void *
> > >>> +vhost_user_client_reconnect(void *arg)
> > >>> +{
> > >>> + struct reconnect_info *reconn = arg;
> > >>> + int ret;
> > >>> +
> > >>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> > >>> + while (1) {
> > >>> + ret = connect(reconn->fd, (struct sockaddr
> *)>un,
> > >>> + sizeof(reconn->un));
> > >>> + if (ret == 0)
> > >>> + break;
> > >>> + sleep(1);
> > >>> + }
> > >>> +
> > >>> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> > >>> + free(reconn);
> > >>> +
> > >>> + return NULL;
> > >>> +}
> > >>> +
> > >> We could create hundreds of vhost-user ports in OVS. Wihout
> connections
> > >> with QEMU established, those ports are just inactive. This works
> fine in
> > >> server mode.
> > >> With client modes, do we need to create hundreds of vhost
> threads? This
> > >> would be too interruptible.
> > >> How about we create only one thread and do the reconnections
> for all the
> > >> unconnected socket?
> > > Yes, good point and good suggestion. Will do it in v2.
> > Hi Michael:
> > This reminds me another irrelevant issue.
> > In OVS, currently for each vhost port, we create an unix domain
> socket,
> > and QEMU vhost proxy connects to this socket, and we use this to
> > identify the connection. This works fine but is our workaround,
> > otherwise we have no way to identify the connection.
> > Do you think if this is an issue?
> > >> Let us say vhost creates one unix domain socket, with path as
> "sockpath",
> > >> and two virtio ports in two VMS both connect to the same socket with
> the
> > >> following command line
> > >> -chardev socket,id=char0,path=sockpath
> > >> How could vhost identify the connection?
> > > getpeername(2)?
> >
> > getpeer name returns host/port? then it isn't useful.
>
> Maybe but I'm still in the dark. Useful for what?
>
> > The typical scenario in my mind is:
> > We create a OVS port with the name "port1", and when we receive an
> > virtio connection with ID "port1", we attach this virtio interface to
> > the OVS port "port1".
>
> If you are going to listen on a socket, you can create ports
> and attach socket fds to it dynamically as long as you get connections.
> What is wrong with that?
Hi Michael,
I haven't reviewed the patchset fully, but to hopefully provide more clarify on
how OVS uses vHost:
OVS with DPDK needs some way to distinguish vHost connections from one another
so it can switch traffic to the correct port depending on how the switch is
programmed.
At the moment this is achieved by:
1. user provides unique port name eg. 'vhost0' (this is normal behaviour in OVS
- checks are put in place to avoid overlapping port names)
2. DPDK vHost lib creates socket called 'vhost0'
3. VM launched with vhost0 socket // -chardev
socket,id=char0,path=/path/to/vhost0
4. OVS receives 'new_device' vhost callback, checks the name of the device
(virtio_dev->ifname == vhost0?), if the name matches the name provided in step
1, OVS stores the virtio_net *dev pointer
5. OVS uses *dev pointer to send and receive traffic to vhost0 via calls to
vhost library functions eg. enqueue(*dev) / dequeue(*dev)
6. Repeat for multiple vhost devices
We need to make sure that there is still some way to distinguish devices from
one another like in step 4. Let me know if you need any further clarification.
Thanks,
Ciara
>
>
> >
> > >
> > >
> > >> Workarounds:
> > >> vhost creates two unix domain sockets, with path as "sockpath1" and
> > >> "sockpath2" respectively,
> > >> and the virtio ports in two VMS respectively connect to "sockpath1" and
> > >> "sockpath2".
> > >>
> > >> If we have some name string from QEMU over vhost, as you
> mentioned, we
> > >> could create only one socket with path "sockpath".
> > >> User ensure that the names are unique, just as how they do with
> multiple
> > >> sockets.
> > >>
> > > Seems rather fragile.
> >
> > >From the scenario above, it is enough. That is also how it works today
> > in DPDK OVS implementation with multiple sockets.
> > Any other idea?
> >
> > >
> > >>> I'm sorry, I have trouble understanding what you wrote above.
> > >>> What is the issue you are trying to work around?
> > >>>
> > Do we have plan to support identification in
> VHOST_USER_MESSAGE? With
> > the
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On Tue, May 10, 2016 at 09:00:45AM +, Xie, Huawei wrote:
> On 5/10/2016 4:42 PM, Michael S. Tsirkin wrote:
> > On Tue, May 10, 2016 at 08:07:00AM +, Xie, Huawei wrote:
> >> On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote:
> >>> On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
> On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> > On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
> >> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> >>> +static void *
> >>> +vhost_user_client_reconnect(void *arg)
> >>> +{
> >>> + struct reconnect_info *reconn = arg;
> >>> + int ret;
> >>> +
> >>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> >>> + while (1) {
> >>> + ret = connect(reconn->fd, (struct sockaddr
> >>> *)>un,
> >>> + sizeof(reconn->un));
> >>> + if (ret == 0)
> >>> + break;
> >>> + sleep(1);
> >>> + }
> >>> +
> >>> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> >>> + free(reconn);
> >>> +
> >>> + return NULL;
> >>> +}
> >>> +
> >> We could create hundreds of vhost-user ports in OVS. Wihout connections
> >> with QEMU established, those ports are just inactive. This works fine
> >> in
> >> server mode.
> >> With client modes, do we need to create hundreds of vhost threads? This
> >> would be too interruptible.
> >> How about we create only one thread and do the reconnections for all
> >> the
> >> unconnected socket?
> > Yes, good point and good suggestion. Will do it in v2.
> Hi Michael:
> This reminds me another irrelevant issue.
> In OVS, currently for each vhost port, we create an unix domain socket,
> and QEMU vhost proxy connects to this socket, and we use this to
> identify the connection. This works fine but is our workaround,
> otherwise we have no way to identify the connection.
> Do you think if this is an issue?
> >> Let us say vhost creates one unix domain socket, with path as "sockpath",
> >> and two virtio ports in two VMS both connect to the same socket with the
> >> following command line
> >> -chardev socket,id=char0,path=sockpath
> >> How could vhost identify the connection?
> > getpeername(2)?
>
> getpeer name returns host/port? then it isn't useful.
Maybe but I'm still in the dark. Useful for what?
> The typical scenario in my mind is:
> We create a OVS port with the name "port1", and when we receive an
> virtio connection with ID "port1", we attach this virtio interface to
> the OVS port "port1".
If you are going to listen on a socket, you can create ports
and attach socket fds to it dynamically as long as you get connections.
What is wrong with that?
>
> >
> >
> >> Workarounds:
> >> vhost creates two unix domain sockets, with path as "sockpath1" and
> >> "sockpath2" respectively,
> >> and the virtio ports in two VMS respectively connect to "sockpath1" and
> >> "sockpath2".
> >>
> >> If we have some name string from QEMU over vhost, as you mentioned, we
> >> could create only one socket with path "sockpath".
> >> User ensure that the names are unique, just as how they do with multiple
> >> sockets.
> >>
> > Seems rather fragile.
>
> >From the scenario above, it is enough. That is also how it works today
> in DPDK OVS implementation with multiple sockets.
> Any other idea?
>
> >
> >>> I'm sorry, I have trouble understanding what you wrote above.
> >>> What is the issue you are trying to work around?
> >>>
> Do we have plan to support identification in VHOST_USER_MESSAGE? With
> the identification, if vhost as server, we only need to create one
> socket which receives multiple connections, and use the ID in the
> message to identify the connection.
>
> /huawei
> >>> Sending e.g. -name string from qemu over vhost might be useful
> >>> for debugging, but I'm not sure it's a good idea to
> >>> rely on it being unique.
> >>>
> > Thanks.
> >
> > --yliu
> >
>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On Tue, May 10, 2016 at 08:07:00AM +, Xie, Huawei wrote:
> On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote:
> > On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
> >> On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> >>> On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> > +static void *
> > +vhost_user_client_reconnect(void *arg)
> > +{
> > + struct reconnect_info *reconn = arg;
> > + int ret;
> > +
> > + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> > + while (1) {
> > + ret = connect(reconn->fd, (struct sockaddr
> > *)>un,
> > + sizeof(reconn->un));
> > + if (ret == 0)
> > + break;
> > + sleep(1);
> > + }
> > +
> > + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> > + free(reconn);
> > +
> > + return NULL;
> > +}
> > +
> We could create hundreds of vhost-user ports in OVS. Wihout connections
> with QEMU established, those ports are just inactive. This works fine in
> server mode.
> With client modes, do we need to create hundreds of vhost threads? This
> would be too interruptible.
> How about we create only one thread and do the reconnections for all the
> unconnected socket?
> >>> Yes, good point and good suggestion. Will do it in v2.
> >> Hi Michael:
> >> This reminds me another irrelevant issue.
> >> In OVS, currently for each vhost port, we create an unix domain socket,
> >> and QEMU vhost proxy connects to this socket, and we use this to
> >> identify the connection. This works fine but is our workaround,
> >> otherwise we have no way to identify the connection.
> >> Do you think if this is an issue?
>
> Let us say vhost creates one unix domain socket, with path as "sockpath",
> and two virtio ports in two VMS both connect to the same socket with the
> following command line
> -chardev socket,id=char0,path=sockpath
> How could vhost identify the connection?
getpeername(2)?
>
> Workarounds:
> vhost creates two unix domain sockets, with path as "sockpath1" and
> "sockpath2" respectively,
> and the virtio ports in two VMS respectively connect to "sockpath1" and
> "sockpath2".
>
> If we have some name string from QEMU over vhost, as you mentioned, we
> could create only one socket with path "sockpath".
> User ensure that the names are unique, just as how they do with multiple
> sockets.
>
Seems rather fragile.
> > I'm sorry, I have trouble understanding what you wrote above.
> > What is the issue you are trying to work around?
> >
> >> Do we have plan to support identification in VHOST_USER_MESSAGE? With
> >> the identification, if vhost as server, we only need to create one
> >> socket which receives multiple connections, and use the ID in the
> >> message to identify the connection.
> >>
> >> /huawei
> > Sending e.g. -name string from qemu over vhost might be useful
> > for debugging, but I'm not sure it's a good idea to
> > rely on it being unique.
> >
> >>> Thanks.
> >>>
> >>> --yliu
> >>>
>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
> On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> > On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
> >> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> >>> +static void *
> >>> +vhost_user_client_reconnect(void *arg)
> >>> +{
> >>> + struct reconnect_info *reconn = arg;
> >>> + int ret;
> >>> +
> >>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> >>> + while (1) {
> >>> + ret = connect(reconn->fd, (struct sockaddr *)>un,
> >>> + sizeof(reconn->un));
> >>> + if (ret == 0)
> >>> + break;
> >>> + sleep(1);
> >>> + }
> >>> +
> >>> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> >>> + free(reconn);
> >>> +
> >>> + return NULL;
> >>> +}
> >>> +
> >> We could create hundreds of vhost-user ports in OVS. Wihout connections
> >> with QEMU established, those ports are just inactive. This works fine in
> >> server mode.
> >> With client modes, do we need to create hundreds of vhost threads? This
> >> would be too interruptible.
> >> How about we create only one thread and do the reconnections for all the
> >> unconnected socket?
> > Yes, good point and good suggestion. Will do it in v2.
>
> Hi Michael:
> This reminds me another irrelevant issue.
> In OVS, currently for each vhost port, we create an unix domain socket,
> and QEMU vhost proxy connects to this socket, and we use this to
> identify the connection. This works fine but is our workaround,
> otherwise we have no way to identify the connection.
> Do you think if this is an issue?
I'm sorry, I have trouble understanding what you wrote above.
What is the issue you are trying to work around?
> Do we have plan to support identification in VHOST_USER_MESSAGE? With
> the identification, if vhost as server, we only need to create one
> socket which receives multiple connections, and use the ID in the
> message to identify the connection.
>
> /huawei
Sending e.g. -name string from qemu over vhost might be useful
for debugging, but I'm not sure it's a good idea to
rely on it being unique.
>
> >
> > Thanks.
> >
> > --yliu
> >
>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On 5/10/2016 4:42 PM, Michael S. Tsirkin wrote:
> On Tue, May 10, 2016 at 08:07:00AM +, Xie, Huawei wrote:
>> On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote:
>>> On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
>> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
>>> +static void *
>>> +vhost_user_client_reconnect(void *arg)
>>> +{
>>> + struct reconnect_info *reconn = arg;
>>> + int ret;
>>> +
>>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
>>> + while (1) {
>>> + ret = connect(reconn->fd, (struct sockaddr
>>> *)>un,
>>> + sizeof(reconn->un));
>>> + if (ret == 0)
>>> + break;
>>> + sleep(1);
>>> + }
>>> +
>>> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
>>> + free(reconn);
>>> +
>>> + return NULL;
>>> +}
>>> +
>> We could create hundreds of vhost-user ports in OVS. Wihout connections
>> with QEMU established, those ports are just inactive. This works fine in
>> server mode.
>> With client modes, do we need to create hundreds of vhost threads? This
>> would be too interruptible.
>> How about we create only one thread and do the reconnections for all the
>> unconnected socket?
> Yes, good point and good suggestion. Will do it in v2.
Hi Michael:
This reminds me another irrelevant issue.
In OVS, currently for each vhost port, we create an unix domain socket,
and QEMU vhost proxy connects to this socket, and we use this to
identify the connection. This works fine but is our workaround,
otherwise we have no way to identify the connection.
Do you think if this is an issue?
>> Let us say vhost creates one unix domain socket, with path as "sockpath",
>> and two virtio ports in two VMS both connect to the same socket with the
>> following command line
>> -chardev socket,id=char0,path=sockpath
>> How could vhost identify the connection?
> getpeername(2)?
getpeer name returns host/port? then it isn't useful.
The typical scenario in my mind is:
We create a OVS port with the name "port1", and when we receive an
virtio connection with ID "port1", we attach this virtio interface to
the OVS port "port1".
>
>
>> Workarounds:
>> vhost creates two unix domain sockets, with path as "sockpath1" and
>> "sockpath2" respectively,
>> and the virtio ports in two VMS respectively connect to "sockpath1" and
>> "sockpath2".
>>
>> If we have some name string from QEMU over vhost, as you mentioned, we
>> could create only one socket with path "sockpath".
>> User ensure that the names are unique, just as how they do with multiple
>> sockets.
>>
> Seems rather fragile.
>From the scenario above, it is enough. That is also how it works today
in DPDK OVS implementation with multiple sockets.
Any other idea?
>
>>> I'm sorry, I have trouble understanding what you wrote above.
>>> What is the issue you are trying to work around?
>>>
Do we have plan to support identification in VHOST_USER_MESSAGE? With
the identification, if vhost as server, we only need to create one
socket which receives multiple connections, and use the ID in the
message to identify the connection.
/huawei
>>> Sending e.g. -name string from qemu over vhost might be useful
>>> for debugging, but I'm not sure it's a good idea to
>>> rely on it being unique.
>>>
> Thanks.
>
> --yliu
>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On 5/10/2016 3:56 PM, Michael S. Tsirkin wrote:
> On Tue, May 10, 2016 at 07:24:10AM +, Xie, Huawei wrote:
>> On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
>>> On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> +static void *
> +vhost_user_client_reconnect(void *arg)
> +{
> + struct reconnect_info *reconn = arg;
> + int ret;
> +
> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> + while (1) {
> + ret = connect(reconn->fd, (struct sockaddr *)>un,
> + sizeof(reconn->un));
> + if (ret == 0)
> + break;
> + sleep(1);
> + }
> +
> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> + free(reconn);
> +
> + return NULL;
> +}
> +
We could create hundreds of vhost-user ports in OVS. Wihout connections
with QEMU established, those ports are just inactive. This works fine in
server mode.
With client modes, do we need to create hundreds of vhost threads? This
would be too interruptible.
How about we create only one thread and do the reconnections for all the
unconnected socket?
>>> Yes, good point and good suggestion. Will do it in v2.
>> Hi Michael:
>> This reminds me another irrelevant issue.
>> In OVS, currently for each vhost port, we create an unix domain socket,
>> and QEMU vhost proxy connects to this socket, and we use this to
>> identify the connection. This works fine but is our workaround,
>> otherwise we have no way to identify the connection.
>> Do you think if this is an issue?
Let us say vhost creates one unix domain socket, with path as "sockpath",
and two virtio ports in two VMS both connect to the same socket with the
following command line
-chardev socket,id=char0,path=sockpath
How could vhost identify the connection?
Workarounds:
vhost creates two unix domain sockets, with path as "sockpath1" and
"sockpath2" respectively,
and the virtio ports in two VMS respectively connect to "sockpath1" and
"sockpath2".
If we have some name string from QEMU over vhost, as you mentioned, we
could create only one socket with path "sockpath".
User ensure that the names are unique, just as how they do with multiple
sockets.
> I'm sorry, I have trouble understanding what you wrote above.
> What is the issue you are trying to work around?
>
>> Do we have plan to support identification in VHOST_USER_MESSAGE? With
>> the identification, if vhost as server, we only need to create one
>> socket which receives multiple connections, and use the ID in the
>> message to identify the connection.
>>
>> /huawei
> Sending e.g. -name string from qemu over vhost might be useful
> for debugging, but I'm not sure it's a good idea to
> rely on it being unique.
>
>>> Thanks.
>>>
>>> --yliu
>>>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On 5/10/2016 2:08 AM, Yuanhan Liu wrote:
> On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
>> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
>>> +static void *
>>> +vhost_user_client_reconnect(void *arg)
>>> +{
>>> + struct reconnect_info *reconn = arg;
>>> + int ret;
>>> +
>>> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
>>> + while (1) {
>>> + ret = connect(reconn->fd, (struct sockaddr *)>un,
>>> + sizeof(reconn->un));
>>> + if (ret == 0)
>>> + break;
>>> + sleep(1);
>>> + }
>>> +
>>> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
>>> + free(reconn);
>>> +
>>> + return NULL;
>>> +}
>>> +
>> We could create hundreds of vhost-user ports in OVS. Wihout connections
>> with QEMU established, those ports are just inactive. This works fine in
>> server mode.
>> With client modes, do we need to create hundreds of vhost threads? This
>> would be too interruptible.
>> How about we create only one thread and do the reconnections for all the
>> unconnected socket?
> Yes, good point and good suggestion. Will do it in v2.
Hi Michael:
This reminds me another irrelevant issue.
In OVS, currently for each vhost port, we create an unix domain socket,
and QEMU vhost proxy connects to this socket, and we use this to
identify the connection. This works fine but is our workaround,
otherwise we have no way to identify the connection.
Do you think if this is an issue?
Do we have plan to support identification in VHOST_USER_MESSAGE? With
the identification, if vhost as server, we only need to create one
socket which receives multiple connections, and use the ID in the
message to identify the connection.
/huawei
>
> Thanks.
>
> --yliu
>
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> +static void *
> +vhost_user_client_reconnect(void *arg)
> +{
> + struct reconnect_info *reconn = arg;
> + int ret;
> +
> + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> + while (1) {
> + ret = connect(reconn->fd, (struct sockaddr *)>un,
> + sizeof(reconn->un));
> + if (ret == 0)
> + break;
> + sleep(1);
> + }
> +
> + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> + free(reconn);
> +
> + return NULL;
> +}
> +
We could create hundreds of vhost-user ports in OVS. Wihout connections
with QEMU established, those ports are just inactive. This works fine in
server mode.
With client modes, do we need to create hundreds of vhost threads? This
would be too interruptible.
How about we create only one thread and do the reconnections for all the
unconnected socket?
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
On Mon, May 09, 2016 at 04:47:02PM +, Xie, Huawei wrote:
> On 5/7/2016 2:36 PM, Yuanhan Liu wrote:
> > +static void *
> > +vhost_user_client_reconnect(void *arg)
> > +{
> > + struct reconnect_info *reconn = arg;
> > + int ret;
> > +
> > + RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
> > + while (1) {
> > + ret = connect(reconn->fd, (struct sockaddr *)>un,
> > + sizeof(reconn->un));
> > + if (ret == 0)
> > + break;
> > + sleep(1);
> > + }
> > +
> > + vhost_user_add_connection(reconn->fd, reconn->vsocket);
> > + free(reconn);
> > +
> > + return NULL;
> > +}
> > +
>
> We could create hundreds of vhost-user ports in OVS. Wihout connections
> with QEMU established, those ports are just inactive. This works fine in
> server mode.
> With client modes, do we need to create hundreds of vhost threads? This
> would be too interruptible.
> How about we create only one thread and do the reconnections for all the
> unconnected socket?
Yes, good point and good suggestion. Will do it in v2.
Thanks.
--yliu
[dpdk-dev] [PATCH 3/6] vhost: add reconnect ability
Allow reconnecting on failure when both RTE_VHOST_USER_RECONNECT and
RTE_VHOST_USER_CLIENT flags are set.
Reconnecting means two things here:
- when DPDK app starts first and QEMU (as the server) is not started,
without reconnecting, DPDK app would simply fail on vhost-user
registration.
- when QEMU reboots, without reconnecting, you can't re-establish the
connection without restarting DPDK app.
This patch make it work well for both above cases. It simply creates
a new thread, and keep trying calling "connect()", until it succeeds.
Signed-off-by: Yuanhan Liu
---
lib/librte_vhost/rte_virtio_net.h| 1 +
lib/librte_vhost/vhost_user/vhost-net-user.c | 63 +---
2 files changed, 59 insertions(+), 5 deletions(-)
diff --git a/lib/librte_vhost/rte_virtio_net.h
b/lib/librte_vhost/rte_virtio_net.h
index c84e7ab..f354d52 100644
--- a/lib/librte_vhost/rte_virtio_net.h
+++ b/lib/librte_vhost/rte_virtio_net.h
@@ -52,6 +52,7 @@
#include
#define RTE_VHOST_USER_CLIENT (1ULL << 0)
+#define RTE_VHOST_USER_RECONNECT (1ULL << 1)
struct rte_mbuf;
diff --git a/lib/librte_vhost/vhost_user/vhost-net-user.c
b/lib/librte_vhost/vhost_user/vhost-net-user.c
index aa98717..07bce6e 100644
--- a/lib/librte_vhost/vhost_user/vhost-net-user.c
+++ b/lib/librte_vhost/vhost_user/vhost-net-user.c
@@ -59,6 +59,7 @@ struct vhost_user_socket {
char *path;
int listenfd;
int is_server;
+ int reconnect;
};
struct vhost_user_connection {
@@ -78,6 +79,7 @@ struct vhost_user {
static void vhost_user_server_new_connection(int fd, void *data, int *remove);
static void vhost_user_msg_handler(int fd, void *dat, int *remove);
+static int vhost_user_create_client(struct vhost_user_socket *vsocket);
static struct vhost_user vhost_user = {
.fdset = {
@@ -304,6 +306,8 @@ vhost_user_msg_handler(int connfd, void *dat, int *remove)
vid = conn->vid;
ret = read_vhost_message(connfd, );
if (ret <= 0 || msg.request >= VHOST_USER_MAX) {
+ struct vhost_user_socket *vsocket = conn->vsocket;
+
if (ret < 0)
RTE_LOG(ERR, VHOST_CONFIG,
"vhost read message failed\n");
@@ -319,6 +323,9 @@ vhost_user_msg_handler(int connfd, void *dat, int *remove)
free(conn);
vhost_destroy_device(vid);
+ if (vsocket->reconnect)
+ vhost_user_create_client(vsocket);
+
return;
}
@@ -470,6 +477,33 @@ err:
return -1;
}
+struct reconnect_info {
+ struct sockaddr_un un;
+ int fd;
+ struct vhost_user_socket *vsocket;
+};
+
+static void *
+vhost_user_client_reconnect(void *arg)
+{
+ struct reconnect_info *reconn = arg;
+ int ret;
+
+ RTE_LOG(ERR, VHOST_CONFIG, "reconnecting...\n");
+ while (1) {
+ ret = connect(reconn->fd, (struct sockaddr *)>un,
+ sizeof(reconn->un));
+ if (ret == 0)
+ break;
+ sleep(1);
+ }
+
+ vhost_user_add_connection(reconn->fd, reconn->vsocket);
+ free(reconn);
+
+ return NULL;
+}
+
static int
vhost_user_create_client(struct vhost_user_socket *vsocket)
{
@@ -477,22 +511,40 @@ vhost_user_create_client(struct vhost_user_socket
*vsocket)
int ret;
struct sockaddr_un un;
const char *path = vsocket->path;
+ struct reconnect_info *reconn;
+ pthread_t tid;
fd = create_unix_socket(path, , vsocket->is_server);
if (fd < 0)
return -1;
ret = connect(fd, (struct sockaddr *), sizeof(un));
- if (ret < 0) {
- RTE_LOG(ERR, VHOST_CONFIG, "failed to connect to %s: %s\n",
- path, strerror(errno));
+ if (ret == 0) {
+ vhost_user_add_connection(fd, vsocket);
+ return 0;
+ }
+
+ RTE_LOG(ERR, VHOST_CONFIG,
+ "failed to connect to %s: %s\n",
+ path, strerror(errno));
+
+ if (!vsocket->reconnect) {
close(fd);
return -1;
}
- vhost_user_add_connection(fd, vsocket);
+ /* Create a thread to try reconnecting, to not block the caller. */
+ reconn = malloc(sizeof(*reconn));
+ reconn->un = un;
+ reconn->fd = fd;
+ reconn->vsocket = vsocket;
+ ret = pthread_create(, NULL, vhost_user_client_reconnect, reconn);
+ if (ret < 0) {
+ close(fd);
+ RTE_LOG(ERR, VHOST_CONFIG, "failed to create reconnect thread");
+ }
- return 0;
+ return ret;
}
/*
@@ -524,6 +576,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
vsocket->path = strdup(path);
if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
+ vsocket->reconnect = !!(flags & RTE_VHOST_USER_RECONNECT);
