Re: [Qemu-devel] [PATCH 6/7] vhost-user: add multiple queue support

2015-09-09 Thread Ouyang, Changchun 


> -Original Message-
> From: Yuanhan Liu [mailto:yuanhan@linux.intel.com]
> Sent: Tuesday, September 8, 2015 3:39 PM
> To: qemu-devel@nongnu.org
> Cc: m...@redhat.com; Ouyang, Changchun; Yuanhan Liu
> Subject: [PATCH 6/7] vhost-user: add multiple queue support
> 
> From: Ouyang Changchun <changchun.ouy...@intel.com>
> 
> This patch is initially based a patch from Nikolay Nikolaev.
> 
> Here is the latest version for adding vhost-user multiple queue support, by
> creating a nc and vhost_net pair for each queue.
> 
> 
>  static int vhost_user_start(VhostUserState *s)  {
>  VhostNetOptions options;
> +VHostNetState *vhost_net;
> +int max_queues;
> +int i = 0;
> 
> -if (vhost_user_running(s)) {
> +if (s->running)
>  return 0;
> -}
> 
>  options.backend_type = VHOST_BACKEND_TYPE_USER;
> -options.net_backend = >nc;
>  options.opaque = s->chr;
> 
> -s->vhost_net = vhost_net_init();
> +options.net_backend = s->peers[i].nc;
> +vhost_net = s->peers[i++].vhost_net = vhost_net_init();
> +
> +max_queues = vhost_net_get_max_queues(vhost_net);
> +if (s->queues >= max_queues) {

use '>' rather than '>=' here?

Re: [Qemu-devel] [PATCH v6 1/2] vhost-user: add multi queue support

2015-09-01 Thread Ouyang, Changchun 


> -Original Message-
> From: Michael S. Tsirkin [mailto:m...@redhat.com]
> Sent: Tuesday, September 1, 2015 10:10 PM
> To: Yuanhan Liu
> Cc: snabb-de...@googlegroups.com; thibaut.col...@6wind.com; qemu-
> de...@nongnu.org; n.nikol...@virtualopensystems.com; l...@snabb.co;
> Long, Thomas; Ouyang, Changchun
> Subject: Re: [Qemu-devel] [PATCH v6 1/2] vhost-user: add multi queue
> support
> 
> On Tue, Sep 01, 2015 at 08:15:15PM +0800, Yuanhan Liu wrote:
> > On Tue, Sep 01, 2015 at 01:07:11PM +0300, Michael S. Tsirkin wrote:
> > > On Tue, Sep 01, 2015 at 05:13:50PM +0800, Yuanhan Liu wrote:
> > > > On Thu, Aug 13, 2015 at 12:18:38PM +0300, Michael S. Tsirkin wrote:
> > > > > On Wed, Aug 12, 2015 at 02:25:41PM +0800, Ouyang Changchun wrote:
> > > > > > Based on patch by Nikolay Nikolaev:
> > > > > > Vhost-user will implement the multi queue support in a similar
> > > > > > way to what vhost already has - a separate thread for each queue.
> > > > > > To enable the multi queue functionality - a new command line
> > > > > > parameter "queues" is introduced for the vhost-user netdev.
> > > > > >
> > > > > > The RESET_OWNER change is based on commit:
> > > > > >294ce717e0f212ed0763307f3eab72b4a1bdf4d0
> > > > > > If it is reverted, the patch need update for it accordingly.
> > > > > >
> > > > > > Signed-off-by: Nikolay Nikolaev
> > > > > > <n.nikol...@virtualopensystems.com>
> > > > > > Signed-off-by: Changchun Ouyang <changchun.ouy...@intel.com>
> > > > [snip...]
> > > > > > @@ -198,7 +203,7 @@ Message types
> > > > > >
> > > > > >Id: 4
> > > > > >Equivalent ioctl: VHOST_RESET_OWNER
> > > > > > -  Master payload: N/A
> > > > > > +  Master payload: vring state description
> > > > > >
> > > > > >Issued when a new connection is about to be closed. The 
> > > > > > Master
> will no
> > > > > >longer own this connection (and will usually close it).
> > > > >
> > > > > This is an interface change, isn't it?
> > > > > We can't make it unconditionally, need to make it dependent on a
> > > > > protocol flag.
> > > >
> > > > Hi Michael,
> > > >
> > > > I'm wondering why we need a payload here, as we don't do that for
> > > > VHOST_SET_OWNER. I mean, stopping one or few queue pairs when a
> > > > connect is about to be close doesn't make sense to me. Instead, we
> > > > should clean up all queue pair when VHOST_RESET_OWNER message is
> > > > received, right?
> > >
> > > We really should rename VHOST_RESET_OWNER to
> VHOST_RESET_DEVICE.
> >
> > Yeah, second that.
> >
> > BTW, can we simply do the name convertion, just changing
> > VHOST_RESET_OWNER to VHOST_RESET_DEVICE(or VHOST_STOP_DEVICE).
> I guess
> > it's doable in theory as far as we don't change the number. I somehow
> > feel it's not a good practice.
> 
> I think just renaming is fine, we are not changing the protocol at all.

Agree, till now, VHOST_RESET_OWNER is not used pretty much, so no backward 
compatibility issue.
Renaming it is enough.

> 
> > Maybe we could make it as a new vhost message, and mark the old one as
> > obsolete? That doesn't sound perfect, either, as it reserves a number
> > for a message we will not use any more.
> >
> > Also, we may rename VHOST_SET_OWNER to VHOST_INIT_DEVICE?
> 
> I think VHOST_SET_OWNER specified who's the master?
> 

I think VHOST_SET_OWNER is also the message for socket, not for each vring.

> > > And I agree, I don't think it needs a payload.
> >
> > Good to know.
> >
> > >
> > >
> > > > >
> > > > >
> > > > > > diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
> > > > > > 1f25cb3..9cd6c05 100644
> > > > > > --- a/hw/net/vhost_net.c
> > > > > > +++ b/hw/net/vhost_net.c
> > > > [snip...]
> > > > > >  static int net_vhost_user_init(NetClientState *peer, const char
> *device,
> > > > > > -   const char *name, CharDriverState 
> > > > > > *chr)
> > > > > > +

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue support

2015-08-31 Thread Ouyang, Changchun 


> -Original Message-
> From: Michael S. Tsirkin [mailto:m...@redhat.com]
> Sent: Sunday, August 30, 2015 2:16 PM
> To: Ouyang, Changchun
> Cc: snabb-de...@googlegroups.com; qemu-devel@nongnu.org;
> thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
> l...@snabb.co; Long, Thomas; Liu, Yuanhan
> Subject: Re: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue
> support
> 
> On Fri, Aug 28, 2015 at 01:53:39AM +, Ouyang, Changchun wrote:
> > Hi Michael,
> >
> > Thanks for your response!
> >
> > > -Original Message-
> > > From: snabb-de...@googlegroups.com [mailto:snabb-
> > > de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
> > > Sent: Thursday, August 27, 2015 9:05 PM
> > > To: Ouyang, Changchun
> > > Cc: snabb-de...@googlegroups.com; qemu-devel@nongnu.org;
> > > thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
> > > l...@snabb.co; Long, Thomas; Liu, Yuanhan
> > > Subject: Re: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi
> > > queue support
> > >
> > > On Tue, Aug 25, 2015 at 03:25:54AM +, Ouyang, Changchun wrote:
> > > > Hi Michael,
> > > >
> > > > > -Original Message-
> > > > > From: snabb-de...@googlegroups.com [mailto:snabb-
> > > > > de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
> > > > > Sent: Thursday, August 13, 2015 5:19 PM
> > > > > To: Ouyang, Changchun
> > > > > Cc: qemu-devel@nongnu.org; snabb-de...@googlegroups.com;
> > > > > thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
> > > > > l...@snabb.co; Long, Thomas
> > > > > Subject: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi
> > > > > queue support
> > > > >
> > > > > On Wed, Aug 12, 2015 at 02:25:41PM +0800, Ouyang Changchun wrote:
> > > > > > Based on patch by Nikolay Nikolaev:
> > > > > > Vhost-user will implement the multi queue support in a similar
> > > > > > way to what vhost already has - a separate thread for each queue.
> > > > > > To enable the multi queue functionality - a new command line
> > > > > > parameter "queues" is introduced for the vhost-user netdev.
> > > > > >
> > > > > > The RESET_OWNER change is based on commit:
> > > > > >294ce717e0f212ed0763307f3eab72b4a1bdf4d0
> > > > > > If it is reverted, the patch need update for it accordingly.
> > > > > >
> > > > > > Signed-off-by: Nikolay Nikolaev
> > > > > > <n.nikol...@virtualopensystems.com>
> > > > > > Signed-off-by: Changchun Ouyang <changchun.ouy...@intel.com>
> > > > > > ---
> > > > > > Changes since v5:
> > > > > >  - fix the message descption for VHOST_RESET_OWNER in
> > > > > > vhost-user txt
> > > > > >
> > > > > > Changes since v4:
> > > > > >  - remove the unnecessary trailing '\n'
> > > > > >
> > > > > > Changes since v3:
> > > > > >  - fix one typo and wrap one long line
> > > > > >
> > > > > > Changes since v2:
> > > > > >  - fix vq index issue for set_vring_call
> > > > > >When it is the case of VHOST_SET_VRING_CALL, The vq_index
> > > > > > is not
> > > > > initialized before it is used,
> > > > > >thus it could be a random value. The random value leads to
> > > > > > crash in vhost
> > > > > after passing down
> > > > > >to vhost, as vhost use this random value to index an array index.
> > > > > >  - fix the typo in the doc and description
> > > > > >  - address vq index for reset_owner
> > > > > >
> > > > > > Changes since v1:
> > > > > >  - use s->nc.info_str when bringing up/down the backend
> > > > > >
> > > > > >  docs/specs/vhost-user.txt |  7 ++-
> > > > > >  hw/net/vhost_net.c|  3 ++-
> > > > > >  hw/virtio/vhost-user.c| 11 ++-
> > > > > >  net/vhost-user.c  | 37 
> -
> > > > > >  qapi-schema.json  |  6 +-
> > > > > >  qemu-options.hx   |  5 ++

Re: [Qemu-devel] [PATCH v5] vhost-user: add multi queue support

2015-08-30 Thread Ouyang, Changchun 

 -Original Message-
 From: Marcel Apfelbaum [mailto:marcel.apfelb...@gmail.com]
 Sent: Sunday, August 30, 2015 11:28 PM
 To: Ouyang, Changchun; qemu-devel@nongnu.org; m...@redhat.com
 Cc: snabb-de...@googlegroups.com; n.nikol...@virtualopensystems.com;
 l...@snabb.co; Long, Thomas
 Subject: Re: [Qemu-devel] [PATCH v5] vhost-user: add multi queue support
 
 On 05/28/2015 04:23 AM, Ouyang Changchun wrote:
  Based on patch by Nikolay Nikolaev:
  Vhost-user will implement the multi queue support in a similar way to
  what vhost already has - a separate thread for each queue.
  To enable the multi queue functionality - a new command line parameter
  queues is introduced for the vhost-user netdev.
 
  Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
  Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
 Hi,
 
 Can you please share how you did you test it?
 Was it the DPDK vhost-user example? (after the MQ patches, of course) I ask
 because I tried to test it with no success so far.
 Any pointers would be appreciated.
 

Yes, I use DPDK vhost-user example to test it.
Hope the following link help you a bit.
http://article.gmane.org/gmane.comp.networking.dpdk.devel/22758/match=vhost+multiple

Qian and Yuanhan, maybe you can have more try and share more information here. 

Thanks
Changchun

 Thanks,
 Marcel
 
 
  ---
  Changes since v4:
- remove the unnecessary trailing '\n'
 
  Changes since v3:
- fix one typo and wrap one long line
 
  Changes since v2:
- fix vq index issue for set_vring_call
  When it is the case of VHOST_SET_VRING_CALL, The vq_index is not
 initialized before it is used,
  thus it could be a random value. The random value leads to crash in 
  vhost
 after passing down
  to vhost, as vhost use this random value to index an array index.
- fix the typo in the doc and description
- address vq index for reset_owner
 
  Changes since v1:
- use s-nc.info_str when bringing up/down the backend
 
docs/specs/vhost-user.txt |  5 +
hw/net/vhost_net.c|  3 ++-
hw/virtio/vhost-user.c| 11 ++-
net/vhost-user.c  | 37 -
qapi-schema.json  |  6 +-
qemu-options.hx   |  5 +++--
6 files changed, 49 insertions(+), 18 deletions(-)
 
  diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
  index 650bb18..2c8e934 100644
  --- a/docs/specs/vhost-user.txt
  +++ b/docs/specs/vhost-user.txt
  @@ -127,6 +127,11 @@ in the ancillary data:
If Master is unable to send the full message or receives a wrong reply it
 will
close the connection. An optional reconnection mechanism can be
 implemented.
 
  +Multi queue support
  +---
  +The protocol supports multiple queues by setting all index fields in
  +the sent messages to a properly calculated value.
  +
Message types
-
 
  diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
  47f8b89..426b23e 100644
  --- a/hw/net/vhost_net.c
  +++ b/hw/net/vhost_net.c
  @@ -157,6 +157,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions
  *options)
 
net-dev.nvqs = 2;
net-dev.vqs = net-vqs;
  +net-dev.vq_index = net-nc-queue_index;
 
r = vhost_dev_init(net-dev, options-opaque,
   options-backend_type, options-force); @@
  -267,7 +268,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
const VhostOps *vhost_ops = net-dev.vhost_ops;
int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
  -  NULL);
  +  file);
assert(r = 0);
}
}
  diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
  e7ab829..d6f2163 100644
  --- a/hw/virtio/vhost-user.c
  +++ b/hw/virtio/vhost-user.c
  @@ -210,7 +210,12 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
break;
 
case VHOST_SET_OWNER:
  +break;
  +
case VHOST_RESET_OWNER:
  +memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
  +msg.size = sizeof(m.state);
break;
 
case VHOST_SET_MEM_TABLE:
  @@ -253,17 +258,20 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
case VHOST_SET_VRING_NUM:
case VHOST_SET_VRING_BASE:
memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
msg.size = sizeof(m.state);
break;
 
case VHOST_GET_VRING_BASE:
memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
msg.size = sizeof(m.state);
need_reply = 1;
break;
 
case

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue support

2015-08-27 Thread Ouyang, Changchun 
Hi Michael,

Thanks for your response!

 -Original Message-
 From: snabb-de...@googlegroups.com [mailto:snabb-
 de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
 Sent: Thursday, August 27, 2015 9:05 PM
 To: Ouyang, Changchun
 Cc: snabb-de...@googlegroups.com; qemu-devel@nongnu.org;
 thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
 l...@snabb.co; Long, Thomas; Liu, Yuanhan
 Subject: Re: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue
 support
 
 On Tue, Aug 25, 2015 at 03:25:54AM +, Ouyang, Changchun wrote:
  Hi Michael,
 
   -Original Message-
   From: snabb-de...@googlegroups.com [mailto:snabb-
   de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
   Sent: Thursday, August 13, 2015 5:19 PM
   To: Ouyang, Changchun
   Cc: qemu-devel@nongnu.org; snabb-de...@googlegroups.com;
   thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
   l...@snabb.co; Long, Thomas
   Subject: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi
   queue support
  
   On Wed, Aug 12, 2015 at 02:25:41PM +0800, Ouyang Changchun wrote:
Based on patch by Nikolay Nikolaev:
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue functionality - a new command line
parameter queues is introduced for the vhost-user netdev.
   
The RESET_OWNER change is based on commit:
   294ce717e0f212ed0763307f3eab72b4a1bdf4d0
If it is reverted, the patch need update for it accordingly.
   
Signed-off-by: Nikolay Nikolaev
n.nikol...@virtualopensystems.com
Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
Changes since v5:
 - fix the message descption for VHOST_RESET_OWNER in vhost-user
txt
   
Changes since v4:
 - remove the unnecessary trailing '\n'
   
Changes since v3:
 - fix one typo and wrap one long line
   
Changes since v2:
 - fix vq index issue for set_vring_call
   When it is the case of VHOST_SET_VRING_CALL, The vq_index is
not
   initialized before it is used,
   thus it could be a random value. The random value leads to
crash in vhost
   after passing down
   to vhost, as vhost use this random value to index an array index.
 - fix the typo in the doc and description
 - address vq index for reset_owner
   
Changes since v1:
 - use s-nc.info_str when bringing up/down the backend
   
 docs/specs/vhost-user.txt |  7 ++-
 hw/net/vhost_net.c|  3 ++-
 hw/virtio/vhost-user.c| 11 ++-
 net/vhost-user.c  | 37 -
 qapi-schema.json  |  6 +-
 qemu-options.hx   |  5 +++--
 6 files changed, 50 insertions(+), 19 deletions(-)
   
diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 70da3b1..9390f89 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -135,6 +135,11 @@ As older slaves don't support negotiating
protocol features,  a feature bit was dedicated for this purpose:
 #define VHOST_USER_F_PROTOCOL_FEATURES 30
   
+Multi queue support
+---
+The protocol supports multiple queues by setting all index fields
+in the sent messages to a properly calculated value.
+
 Message types
 -
   
@@ -198,7 +203,7 @@ Message types
   
   Id: 4
   Equivalent ioctl: VHOST_RESET_OWNER
-  Master payload: N/A
+  Master payload: vring state description
   
   Issued when a new connection is about to be closed. The Master
 will no
   longer own this connection (and will usually close it).
  
   This is an interface change, isn't it?
   We can't make it unconditionally, need to make it dependent on a
   protocol flag.
 
 
 Pls remember to fix this one.
 
  
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
1f25cb3..9cd6c05 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -159,6 +159,7 @@ struct vhost_net
*vhost_net_init(VhostNetOptions
*options)
   
 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;
   
 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force); @@
-269,7 +270,7 @@ static void vhost_net_stop_one(struct vhost_net
 *net,
 for (file.index = 0; file.index  net-dev.nvqs; ++file.index) 
{
 const VhostOps *vhost_ops = net-dev.vhost_ops;
 int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
-  NULL);
+  file);
 assert(r = 0);
 }
 }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
27ba035..fb11d4c 100644
--- a/hw/virtio

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v6 2/2] vhost-user: new protocol feature for multi queue

2015-08-24 Thread Ouyang, Changchun 
Hi Michael,

 -Original Message-
 From: snabb-de...@googlegroups.com [mailto:snabb-
 de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
 Sent: Thursday, August 13, 2015 5:23 PM
 To: Ouyang, Changchun
 Cc: qemu-devel@nongnu.org; snabb-de...@googlegroups.com;
 thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
 l...@snabb.co; Long, Thomas
 Subject: [snabb-devel] Re: [PATCH v6 2/2] vhost-user: new protocol feature
 for multi queue
 
 On Wed, Aug 12, 2015 at 02:25:42PM +0800, Ouyang Changchun wrote:
  This patch is based on top of vhost-user: protocol updates series
  proposed earlier by Michael S. Tsirkin.
 
  Use new message VHOST_USER_SET_VRING_FLAG to enable and disable
 an
  actual virt queue, which is similar to attach/detach queue for tap device.
  virtio driver on guest doesn't have to use max virt queue pair, it
  could enable any number of virt queue ranging from 1 to max virt queue
 pair.
 
  It requires that VHOST_USER_F_PROTOCOL_FEATURES is present.
 
  Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
  ---
  This is added since v5
 
   docs/specs/vhost-user.txt | 17 +
   hw/net/vhost_net.c| 18 ++
   hw/net/virtio-net.c   |  2 ++
   hw/virtio/vhost-user.c| 35
 +--
   include/hw/virtio/vhost-backend.h |  2 ++
   include/net/vhost_net.h   |  1 +
   6 files changed, 73 insertions(+), 2 deletions(-)
 
  diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
  index 9390f89..cca3e5b 100644
  --- a/docs/specs/vhost-user.txt
  +++ b/docs/specs/vhost-user.txt
  @@ -135,6 +135,10 @@ As older slaves don't support negotiating
  protocol features,  a feature bit was dedicated for this purpose:
   #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
  +The Slave uses vring flag to notify the vhost-user whether one virtq
  +is enabled or not. This request doesn't require replies:
  +#define VHOST_USER_PROTOCOL_F_VRING_FLAG 2
  +
   Multi queue support
   ---
   The protocol supports multiple queues by setting all index fields in
  the sent @@ -306,3 +310,16 @@ Message types
 Bits (0-7) of the payload contain the vring index. Bit 8 is the
 invalid FD flag. This flag is set when there is no file descriptor
 in the ancillary data.
  +
  + * VHOST_USER_SET_VRING_FLAG
  +
  +  Id: 18
  +  Equivalent ioctl: N/A
  +  Master payload: vring state description
  +
  +  Set the flag(enable or disable) in the vring, the vhost user backend
  +  enable or disable the vring according to state.num. Olny legal if 
  feature
  +  bit VHOST_USER_F_PROTOCOL_FEATURES is present in
 VHOST_USER_GET_FEATURE
  +  and feature bit VHOST_USER_PROTOCOL_F_VRING_FLAG is present in
  +  VHOST_USER_GET_PROTOCOL_FEATURES. The vring is enabled when
 state.num is
  +  1, otherwise, the vring is disabled.
  diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
  9cd6c05..5fa341c 100644
  --- a/hw/net/vhost_net.c
  +++ b/hw/net/vhost_net.c
  @@ -405,6 +405,19 @@ VHostNetState *get_vhost_net(NetClientState
 *nc)
 
   return vhost_net;
   }
  +
  +int vhost_set_vring_flag(NetClientState *nc, unsigned int enable) {
  +if (nc-info-type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
  +struct vhost_net *net = get_vhost_net(nc);
  +const VhostOps *vhost_ops = net-dev.vhost_ops;
  +if (vhost_ops-vhost_backend_mq_set_vring_flag)
  +return vhost_ops-vhost_backend_mq_set_vring_flag(net-dev,
 enable);
  +}
  +
  +return 0;
  +}
  +
   #else
   struct vhost_net *vhost_net_init(VhostNetOptions *options)  { @@
  -455,4 +468,9 @@ VHostNetState *get_vhost_net(NetClientState *nc)  {
   return 0;
   }
  +
  +int vhost_set_vring_flag(NetClientState *nc, unsigned int enable) {
  +return 0;
  +}
   #endif
  diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index
  3af6faf..272b77d 100644
  --- a/hw/net/virtio-net.c
  +++ b/hw/net/virtio-net.c
  @@ -396,6 +396,7 @@ static int peer_attach(VirtIONet *n, int index)
   }
 
   if (nc-peer-info-type != NET_CLIENT_OPTIONS_KIND_TAP) {
  +vhost_set_vring_flag(nc-peer, 1);
   return 0;
   }
 
  @@ -411,6 +412,7 @@ static int peer_detach(VirtIONet *n, int index)
   }
 
   if (nc-peer-info-type !=  NET_CLIENT_OPTIONS_KIND_TAP) {
  +vhost_set_vring_flag(nc-peer, 0);
   return 0;
   }
 
  diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
  fb11d4c..d806ce2 100644
  --- a/hw/virtio/vhost-user.c
  +++ b/hw/virtio/vhost-user.c
  @@ -25,7 +25,8 @@
 
   #define VHOST_MEMORY_MAX_NREGIONS8
   #define VHOST_USER_F_PROTOCOL_FEATURES 30 -#define
  VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
  +#define VHOST_USER_PROTOCOL_F_VRING_FLAG 2 #define
  +VHOST_USER_PROTOCOL_FEATURE_MASK 0x7ULL
 
   typedef enum VhostUserRequest {
   VHOST_USER_NONE = 0,
  @@ -45,6 +46,7 @@ typedef enum

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue support

2015-08-24 Thread Ouyang, Changchun 
Hi Michael,

 -Original Message-
 From: snabb-de...@googlegroups.com [mailto:snabb-
 de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
 Sent: Thursday, August 13, 2015 5:19 PM
 To: Ouyang, Changchun
 Cc: qemu-devel@nongnu.org; snabb-de...@googlegroups.com;
 thibaut.col...@6wind.com; n.nikol...@virtualopensystems.com;
 l...@snabb.co; Long, Thomas
 Subject: [snabb-devel] Re: [PATCH v6 1/2] vhost-user: add multi queue
 support
 
 On Wed, Aug 12, 2015 at 02:25:41PM +0800, Ouyang Changchun wrote:
  Based on patch by Nikolay Nikolaev:
  Vhost-user will implement the multi queue support in a similar way to
  what vhost already has - a separate thread for each queue.
  To enable the multi queue functionality - a new command line parameter
  queues is introduced for the vhost-user netdev.
 
  The RESET_OWNER change is based on commit:
 294ce717e0f212ed0763307f3eab72b4a1bdf4d0
  If it is reverted, the patch need update for it accordingly.
 
  Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
  Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
  ---
  Changes since v5:
   - fix the message descption for VHOST_RESET_OWNER in vhost-user txt
 
  Changes since v4:
   - remove the unnecessary trailing '\n'
 
  Changes since v3:
   - fix one typo and wrap one long line
 
  Changes since v2:
   - fix vq index issue for set_vring_call
 When it is the case of VHOST_SET_VRING_CALL, The vq_index is not
 initialized before it is used,
 thus it could be a random value. The random value leads to crash in vhost
 after passing down
 to vhost, as vhost use this random value to index an array index.
   - fix the typo in the doc and description
   - address vq index for reset_owner
 
  Changes since v1:
   - use s-nc.info_str when bringing up/down the backend
 
   docs/specs/vhost-user.txt |  7 ++-
   hw/net/vhost_net.c|  3 ++-
   hw/virtio/vhost-user.c| 11 ++-
   net/vhost-user.c  | 37 -
   qapi-schema.json  |  6 +-
   qemu-options.hx   |  5 +++--
   6 files changed, 50 insertions(+), 19 deletions(-)
 
  diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
  index 70da3b1..9390f89 100644
  --- a/docs/specs/vhost-user.txt
  +++ b/docs/specs/vhost-user.txt
  @@ -135,6 +135,11 @@ As older slaves don't support negotiating
  protocol features,  a feature bit was dedicated for this purpose:
   #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
  +Multi queue support
  +---
  +The protocol supports multiple queues by setting all index fields in
  +the sent messages to a properly calculated value.
  +
   Message types
   -
 
  @@ -198,7 +203,7 @@ Message types
 
 Id: 4
 Equivalent ioctl: VHOST_RESET_OWNER
  -  Master payload: N/A
  +  Master payload: vring state description
 
 Issued when a new connection is about to be closed. The Master will 
  no
 longer own this connection (and will usually close it).
 
 This is an interface change, isn't it?
 We can't make it unconditionally, need to make it dependent on a protocol
 flag.
 
 
  diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
  1f25cb3..9cd6c05 100644
  --- a/hw/net/vhost_net.c
  +++ b/hw/net/vhost_net.c
  @@ -159,6 +159,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions
  *options)
 
   net-dev.nvqs = 2;
   net-dev.vqs = net-vqs;
  +net-dev.vq_index = net-nc-queue_index;
 
   r = vhost_dev_init(net-dev, options-opaque,
  options-backend_type, options-force); @@
  -269,7 +270,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
   for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
   const VhostOps *vhost_ops = net-dev.vhost_ops;
   int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
  -  NULL);
  +  file);
   assert(r = 0);
   }
   }
  diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
  27ba035..fb11d4c 100644
  --- a/hw/virtio/vhost-user.c
  +++ b/hw/virtio/vhost-user.c
  @@ -219,7 +219,12 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
   break;
 
   case VHOST_USER_SET_OWNER:
  +break;
  +
   case VHOST_USER_RESET_OWNER:
  +memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
  +msg.size = sizeof(m.state);
   break;
 
   case VHOST_USER_SET_MEM_TABLE:
  @@ -262,17 +267,20 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
   case VHOST_USER_SET_VRING_NUM:
   case VHOST_USER_SET_VRING_BASE:
   memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
   msg.size = sizeof(m.state);
   break;
 
   case

Re: [Qemu-devel] [PATCH v6 0/2] vhost user: Add live migration

2015-08-12 Thread Ouyang, Changchun 


 -Original Message-
 From: Marc-André Lureau [mailto:marcandre.lur...@gmail.com]
 Sent: Wednesday, August 12, 2015 6:07 PM
 To: Michael S. Tsirkin
 Cc: Thibaut Collet; QEMU; stefa...@redhat.com; Jason Wang; Paolo Bonzini;
 Linhaifeng; Ouyang, Changchun
 Subject: Re: [PATCH v6 0/2] vhost user: Add live migration
 
 Hi
 
 On Wed, Aug 12, 2015 at 9:25 AM, Michael S. Tsirkin m...@redhat.com
 wrote:
  I think these patches need to be rebased on top of Marc Andre's ones,
  and use protocol flags to negotiate capabilities.
  Right?
 
 Correct. His patches should be applied before my migration tests, though.
 

Good point, do we have plan how to make these patches applied serially?
M.s.t's protocol feature patch,  marc andre's patch, Thibaut's patch for adding 
live migration, and my patch for vhost-user multi queue. 

 --
 Marc-André Lureau

Re: [Qemu-devel] Qemu-devel Digest, Vol 149, Issue 266

2015-08-12 Thread Ouyang, Changchun 


 -Original Message-
 Date: Wed, 12 Aug 2015 14:15:54 +0300
 From: Michael S. Tsirkin m...@redhat.com
 To: Marcel Apfelbaum mar...@redhat.com
 Cc: qemu-devel@nongnu.org
 Subject: Re: [Qemu-devel] [PATCH] virtio/vhost: drop unnecessary
   VHOST_SET_VRING call
 Message-ID: 20150812141448-mutt-send-email-...@redhat.com
 Content-Type: text/plain; charset=us-ascii
 
 On Wed, Aug 12, 2015 at 02:10:56PM +0300, Marcel Apfelbaum wrote:
  On 08/12/2015 01:34 PM, Michael S. Tsirkin wrote:
  On Wed, Aug 12, 2015 at 01:19:51PM +0300, Marcel Apfelbaum wrote:
  No need to send VHOST_SET_VRING_CALL to backend before the
  negotiation with the guest is finished.
  
  Signed-off-by: Marcel Apfelbaum mar...@redhat.com
  
  Well - we do need to set it to the masked notifier initially to avoid
  losing events.  You can't just drop it - need to move this call
  somewhere else.

Agree with m.s.t.
We could not drop it.
Vhost-user multi queue also need this.

  What do we need to set?
  I just dropped the call to VHOST_SET_VRING_CALL.
 
  Thanks,
  Marcel
 
 We use two eventfds: masked and unmasked one.
 We switch dynamically dependent on msi mask value.
 Code assumes we start out masked, so we need to match that.
 
 
  
  ---
hw/virtio/vhost.c | 13 +
1 file changed, 1 insertion(+), 12 deletions(-)
  
  diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index
  2712c6f..b448542 100644
  --- a/hw/virtio/vhost.c
  +++ b/hw/virtio/vhost.c
  @@ -875,24 +875,13 @@ static void
 vhost_eventfd_del(MemoryListener
  *listener,
static int vhost_virtqueue_init(struct vhost_dev *dev,
struct vhost_virtqueue *vq, int n)
{
  -struct vhost_vring_file file = {
  -.index = n,
  -};
int r = event_notifier_init(vq-masked_notifier, 0);
  +
if (r  0) {
return r;
}
  
  -file.fd = event_notifier_get_fd(vq-masked_notifier);
  -r = dev-vhost_ops-vhost_call(dev, VHOST_SET_VRING_CALL, file);
  -if (r) {
  -r = -errno;
  -goto fail_call;
  -}
return 0;
  -fail_call:
  -event_notifier_cleanup(vq-masked_notifier);
  -return r;
}
  
static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
  --
  2.1.0

[Qemu-devel] [PATCH v6 1/2] vhost-user: add multi queue support

2015-08-12 Thread Ouyang Changchun 
Based on patch by Nikolay Nikolaev:
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue functionality - a new command line parameter
queues is introduced for the vhost-user netdev.

The RESET_OWNER change is based on commit:
   294ce717e0f212ed0763307f3eab72b4a1bdf4d0
If it is reverted, the patch need update for it accordingly.

Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
Changes since v5:
 - fix the message descption for VHOST_RESET_OWNER in vhost-user txt

Changes since v4:
 - remove the unnecessary trailing '\n'

Changes since v3:
 - fix one typo and wrap one long line

Changes since v2:
 - fix vq index issue for set_vring_call
   When it is the case of VHOST_SET_VRING_CALL, The vq_index is not initialized 
before it is used,
   thus it could be a random value. The random value leads to crash in vhost 
after passing down
   to vhost, as vhost use this random value to index an array index.
 - fix the typo in the doc and description
 - address vq index for reset_owner

Changes since v1:
 - use s-nc.info_str when bringing up/down the backend

 docs/specs/vhost-user.txt |  7 ++-
 hw/net/vhost_net.c|  3 ++-
 hw/virtio/vhost-user.c| 11 ++-
 net/vhost-user.c  | 37 -
 qapi-schema.json  |  6 +-
 qemu-options.hx   |  5 +++--
 6 files changed, 50 insertions(+), 19 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 70da3b1..9390f89 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -135,6 +135,11 @@ As older slaves don't support negotiating protocol 
features,
 a feature bit was dedicated for this purpose:
 #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
+Multi queue support
+---
+The protocol supports multiple queues by setting all index fields in the sent
+messages to a properly calculated value.
+
 Message types
 -
 
@@ -198,7 +203,7 @@ Message types
 
   Id: 4
   Equivalent ioctl: VHOST_RESET_OWNER
-  Master payload: N/A
+  Master payload: vring state description
 
   Issued when a new connection is about to be closed. The Master will no
   longer own this connection (and will usually close it).
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 1f25cb3..9cd6c05 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -159,6 +159,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
 
 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;
 
 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force);
@@ -269,7 +270,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
 for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
 const VhostOps *vhost_ops = net-dev.vhost_ops;
 int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
-  NULL);
+  file);
 assert(r = 0);
 }
 }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 27ba035..fb11d4c 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -219,7 +219,12 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 break;
 
 case VHOST_USER_SET_OWNER:
+break;
+
 case VHOST_USER_RESET_OWNER:
+memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
+msg.size = sizeof(m.state);
 break;
 
 case VHOST_USER_SET_MEM_TABLE:
@@ -262,17 +267,20 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,
 case VHOST_USER_SET_VRING_NUM:
 case VHOST_USER_SET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 break;
 
 case VHOST_USER_GET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 need_reply = 1;
 break;
 
 case VHOST_USER_SET_VRING_ADDR:
 memcpy(msg.addr, arg, sizeof(struct vhost_vring_addr));
+msg.addr.index += dev-vq_index;
 msg.size = sizeof(m.addr);
 break;
 
@@ -280,7 +288,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 case VHOST_USER_SET_VRING_CALL:
 case VHOST_USER_SET_VRING_ERR:
 file = arg;
-msg.u64 = file-index  VHOST_USER_VRING_IDX_MASK;
+msg.u64 = (file-index + dev-vq_index)  VHOST_USER_VRING_IDX_MASK;
 msg.size = sizeof(m.u64);
 if (ioeventfd_enabled()  file-fd  0) {
 fds[fd_num++]

[Qemu-devel] [PATCH v6 0/2] vhost-user multi queue support

2015-08-12 Thread Ouyang Changchun 
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue functionality - a new command line parameter
queues is introduced for the vhost-user netdev.

Use new message VHOST_USER_SET_VRING_FLAG to enable and disable an
actual virt queue, which is similar to attach/detach queue for tap device.
virtio driver on guest doesn't have to use max virt queue pair, it could
enable any number of virt queue ranging from 1 to max virt queue pair.
 
It requires that VHOST_USER_F_PROTOCOL_FEATURES is present.

Any comment is welcome for the patch set.

It has dependency on a few other patch(set):
vhost-user: protocol updates series
proposed earlier by Michael S. Tsirkin

RESET_OWNER change is based on commit:
294ce717e0f212ed0763307f3eab72b4a1bdf4d0  

Changchun Ouyang (2):
  vhost-user: add multi queue support
  vhost-user: new protocol feature for multi queue

 docs/specs/vhost-user.txt | 24 +++-
 hw/net/vhost_net.c| 21 +-
 hw/net/virtio-net.c   |  2 ++
 hw/virtio/vhost-user.c| 46 ---
 include/hw/virtio/vhost-backend.h |  2 ++
 include/net/vhost_net.h   |  1 +
 net/vhost-user.c  | 37 ---
 qapi-schema.json  |  6 -
 qemu-options.hx   |  5 +++--
 9 files changed, 123 insertions(+), 21 deletions(-)

-- 
1.8.4.2

[Qemu-devel] [PATCH v6 2/2] vhost-user: new protocol feature for multi queue

2015-08-12 Thread Ouyang Changchun 
This patch is based on top of vhost-user: protocol updates series
proposed earlier by Michael S. Tsirkin.

Use new message VHOST_USER_SET_VRING_FLAG to enable and disable an
actual virt queue, which is similar to attach/detach queue for tap device.
virtio driver on guest doesn't have to use max virt queue pair, it could
enable any number of virt queue ranging from 1 to max virt queue pair.

It requires that VHOST_USER_F_PROTOCOL_FEATURES is present.

Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
This is added since v5

 docs/specs/vhost-user.txt | 17 +
 hw/net/vhost_net.c| 18 ++
 hw/net/virtio-net.c   |  2 ++
 hw/virtio/vhost-user.c| 35 +--
 include/hw/virtio/vhost-backend.h |  2 ++
 include/net/vhost_net.h   |  1 +
 6 files changed, 73 insertions(+), 2 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 9390f89..cca3e5b 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -135,6 +135,10 @@ As older slaves don't support negotiating protocol 
features,
 a feature bit was dedicated for this purpose:
 #define VHOST_USER_F_PROTOCOL_FEATURES 30
 
+The Slave uses vring flag to notify the vhost-user whether one virtq is enabled
+or not. This request doesn't require replies:
+#define VHOST_USER_PROTOCOL_F_VRING_FLAG 2
+
 Multi queue support
 ---
 The protocol supports multiple queues by setting all index fields in the sent
@@ -306,3 +310,16 @@ Message types
   Bits (0-7) of the payload contain the vring index. Bit 8 is the
   invalid FD flag. This flag is set when there is no file descriptor
   in the ancillary data.
+
+ * VHOST_USER_SET_VRING_FLAG
+
+  Id: 18
+  Equivalent ioctl: N/A
+  Master payload: vring state description
+
+  Set the flag(enable or disable) in the vring, the vhost user backend
+  enable or disable the vring according to state.num. Olny legal if feature
+  bit VHOST_USER_F_PROTOCOL_FEATURES is present in VHOST_USER_GET_FEATURE
+  and feature bit VHOST_USER_PROTOCOL_F_VRING_FLAG is present in
+  VHOST_USER_GET_PROTOCOL_FEATURES. The vring is enabled when state.num is
+  1, otherwise, the vring is disabled.
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 9cd6c05..5fa341c 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -405,6 +405,19 @@ VHostNetState *get_vhost_net(NetClientState *nc)
 
 return vhost_net;
 }
+
+int vhost_set_vring_flag(NetClientState *nc, unsigned int enable)
+{
+if (nc-info-type == NET_CLIENT_OPTIONS_KIND_VHOST_USER) {
+struct vhost_net *net = get_vhost_net(nc);
+const VhostOps *vhost_ops = net-dev.vhost_ops;
+if (vhost_ops-vhost_backend_mq_set_vring_flag)
+return vhost_ops-vhost_backend_mq_set_vring_flag(net-dev, 
enable);
+}
+
+return 0;
+}
+
 #else
 struct vhost_net *vhost_net_init(VhostNetOptions *options)
 {
@@ -455,4 +468,9 @@ VHostNetState *get_vhost_net(NetClientState *nc)
 {
 return 0;
 }
+
+int vhost_set_vring_flag(NetClientState *nc, unsigned int enable)
+{
+return 0;
+}
 #endif
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 3af6faf..272b77d 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -396,6 +396,7 @@ static int peer_attach(VirtIONet *n, int index)
 }
 
 if (nc-peer-info-type != NET_CLIENT_OPTIONS_KIND_TAP) {
+vhost_set_vring_flag(nc-peer, 1);
 return 0;
 }
 
@@ -411,6 +412,7 @@ static int peer_detach(VirtIONet *n, int index)
 }
 
 if (nc-peer-info-type !=  NET_CLIENT_OPTIONS_KIND_TAP) {
+vhost_set_vring_flag(nc-peer, 0);
 return 0;
 }
 
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index fb11d4c..d806ce2 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -25,7 +25,8 @@
 
 #define VHOST_MEMORY_MAX_NREGIONS8
 #define VHOST_USER_F_PROTOCOL_FEATURES 30
-#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x0ULL
+#define VHOST_USER_PROTOCOL_F_VRING_FLAG 2
+#define VHOST_USER_PROTOCOL_FEATURE_MASK 0x7ULL
 
 typedef enum VhostUserRequest {
 VHOST_USER_NONE = 0,
@@ -45,6 +46,7 @@ typedef enum VhostUserRequest {
 VHOST_USER_SET_VRING_ERR = 14,
 VHOST_USER_GET_PROTOCOL_FEATURES = 15,
 VHOST_USER_SET_PROTOCOL_FEATURES = 16,
+VHOST_USER_SET_VRING_FLAG = 18,
 VHOST_USER_MAX
 } VhostUserRequest;
 
@@ -399,6 +401,34 @@ static int vhost_user_init(struct vhost_dev *dev, void 
*opaque)
 return 0;
 }
 
+static int vhost_user_set_vring_flag(struct vhost_dev *dev, unsigned int 
enable)
+{
+VhostUserMsg msg = { 0 };
+int err;
+
+assert(dev-vhost_ops-backend_type == VHOST_BACKEND_TYPE_USER);
+
+if ((dev-backend_features  ( 1ULL  VHOST_USER_F_PROTOCOL_FEATURES)) == 
0)
+return -1;
+
+if ((dev-protocol_features  (1ULL  VHOST_USER_PROTOCOL_F_VRING_FLAG)) 
== 0)
+return -1;
+
+

Re: [Qemu-devel] [PATCH 4/4] vhost-user: unit test for new messages

2015-07-23 Thread Ouyang, Changchun 


 -Original Message-
 From: Michael S. Tsirkin [mailto:m...@redhat.com]
 Sent: Friday, July 17, 2015 10:10 PM
 To: qemu-devel@nongnu.org
 Cc: marcandre.lur...@gmail.com; haifeng@huawei.com;
 thibaut.col...@6wind.com; Ouyang, Changchun; f...@redhat.com; Peter
 Maydell; Nikolay Nikolaev; Gonglei; Stefan Hajnoczi
 Subject: [PATCH 4/4] vhost-user: unit test for new messages
 
 Data is empty for now, but do make sure master sets the new feature bit flag.
 
 Signed-off-by: Michael S. Tsirkin m...@redhat.com
 ---
  tests/vhost-user-test.c | 19 +++
  1 file changed, 19 insertions(+)
 
 diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c index
 75fedf0..228acb6 100644
 --- a/tests/vhost-user-test.c
 +++ b/tests/vhost-user-test.c
 @@ -53,6 +53,8 @@
 
  #define VHOST_MEMORY_MAX_NREGIONS8
 
 +#define VHOST_USER_F_PROTOCOL_FEATURES 30
 +
  typedef enum VhostUserRequest {
  VHOST_USER_NONE = 0,
  VHOST_USER_GET_FEATURES = 1,
 @@ -69,6 +71,8 @@ typedef enum VhostUserRequest {
  VHOST_USER_SET_VRING_KICK = 12,
  VHOST_USER_SET_VRING_CALL = 13,
  VHOST_USER_SET_VRING_ERR = 14,
 +VHOST_USER_GET_PROTOCOL_FEATURES = 15,
 +VHOST_USER_SET_PROTOCOL_FEATURES = 16,
  VHOST_USER_MAX
  } VhostUserRequest;
 
 @@ -293,11 +297,26 @@ static void chr_read(void *opaque, const uint8_t
 *buf, int size)
  /* send back features to qemu */
  msg.flags |= VHOST_USER_REPLY_MASK;
  msg.size = sizeof(m.u64);
 +msg.u64 = 0x1ULL  VHOST_USER_F_PROTOCOL_FEATURES;
 +p = (uint8_t *) msg;
 +qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size);
 +break;
 +
 +case VHOST_USER_SET_FEATURES:
 + g_assert_cmpint(msg.u64  (0x1ULL 
 VHOST_USER_F_PROTOCOL_FEATURES),
 + !=, 0ULL);
 +break;
 +
 +case VHOST_USER_GET_PROTOCOL_FEATURES:

Do we also need add test codes for the case: VHOST_USER_SET_PROTOCOL_FEATURES?

 +/* send back features to qemu */
 +msg.flags |= VHOST_USER_REPLY_MASK;
 +msg.size = sizeof(m.u64);
  msg.u64 = 0;
  p = (uint8_t *) msg;
  qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size);
  break;
 
 +
  case VHOST_USER_GET_VRING_BASE:
  /* send back vring base to qemu */
  msg.flags |= VHOST_USER_REPLY_MASK;
 --
 MST

Re: [Qemu-devel] Qemu-devel Digest, Vol 148, Issue 574

2015-07-14 Thread Ouyang, Changchun 


 -Original Message-
 From: qemu-devel-bounces+changchun.ouyang=intel@nongnu.org
 [mailto:qemu-devel-bounces+changchun.ouyang=intel@nongnu.org]
 On Behalf Of qemu-devel-requ...@nongnu.org
 Sent: Monday, July 13, 2015 11:28 PM
 To: qemu-devel@nongnu.org
 Subject: Qemu-devel Digest, Vol 148, Issue 574
 
 Send Qemu-devel mailing list submissions to
   qemu-devel@nongnu.org
 
 To subscribe or unsubscribe via the World Wide Web, visit
   https://lists.nongnu.org/mailman/listinfo/qemu-devel
 or, via email, send a message with subject or body 'help' to
   qemu-devel-requ...@nongnu.org
 
 You can reach the person managing the list at
   qemu-devel-ow...@nongnu.org
 
 When replying, please edit your Subject line so it is more specific than Re:
 Contents of Qemu-devel digest...
 
 
 Today's Topics:
 
1. Re: [PATCH v4 1/5] cpu: Provide vcpu throttling interface
   (Paolo Bonzini)
2. Re: [PATCH] net: vhost-user reconnect (Michael S. Tsirkin)
3. [RFC PATCH qemu v2 0/5] vfio: SPAPR IOMMU v2 (memory
   preregistration support) (Alexey Kardashevskiy)
 
 
 --
 
 Message: 1
 Date: Mon, 13 Jul 2015 17:14:32 +0200
 From: Paolo Bonzini pbonz...@redhat.com
 To: jjhe...@linux.vnet.ibm.com, afaer...@suse.de,
   amit.s...@redhat.com,   dgilb...@redhat.com,
 borntrae...@de.ibm.com,
   quint...@redhat.com,qemu-devel@nongnu.org
 Subject: Re: [Qemu-devel] [PATCH v4 1/5] cpu: Provide vcpu throttling
   interface
 Message-ID: 55a3d5d8.7070...@redhat.com
 Content-Type: text/plain; charset=windows-1252
 
 
 
 On 13/07/2015 16:43, Jason J. Herne wrote:
 
  +CPU_FOREACH(cpu) {
  +async_run_on_cpu(cpu, cpu_throttle_thread, NULL);
  +}
  +
  +timer_mod(throttle_timer,
  qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL_RT) +
  +   CPU_THROTTLE_TIMESLICE); }
 
  This could cause callbacks to pile up I think.  David, do you have
  any idea how to fix it?
 
  I'm not sure how callbacks can pile up here. If the vcpus are running
  then their thread's will execute the callbacks. If they are not
  running then the use of QEMU_CLOCK_VIRTUAL_RT will prevent the
  callbacks from stacking because the timer is not running, right?
 
 Couldn't the iothread starve the VCPUs?  They need to take the iothread lock
 in order to process the callbacks.
 
 Paolo
 
 
 
 --
 
 Message: 2
 Date: Mon, 13 Jul 2015 18:18:51 +0300
 From: Michael S. Tsirkin m...@redhat.com
 To: Matus Fabian -X (matfabia - Pantheon Technologies SRO at Cisco)
   matfa...@cisco.com
 Cc: pbonz...@redhat.com pbonz...@redhat.com,
   jasow...@redhat.com
   jasow...@redhat.com,  qemu-devel@nongnu.org
   qemu-devel@nongnu.org,stefa...@redhat.com
 stefa...@redhat.com,
   Damjan Marion \(damarion\) damar...@cisco.com
 Subject: Re: [Qemu-devel] [PATCH] net: vhost-user reconnect
 Message-ID: 20150713181721-mutt-send-email-...@redhat.com
 Content-Type: text/plain; charset=us-ascii
 
 On Mon, Jul 13, 2015 at 02:44:37PM +, Matus Fabian -X (matfabia -
 Pantheon Technologies SRO at Cisco) wrote:
  If userspace switch restarts it will reconnect to unix socket but QEMU
  will not send any vhost-user information and that basically means that
  userspace switch restart requires restart of VM.
  Fix detects that userspace switch is disconnected and notify VM that
  link status is down. After user space switch is reconnected QEMU send
  vhost-user information to the userspace switch and notify VM that link is
 up.
 
  Signed-off-by: Matus Fabian matfa...@cisco.com
 
 This only works most of the time.  Reconnect can't be implemented without
 guest changes.
 See http://mid.gmane.org/1434945048-27958-1-git-send-email-
 muk...@igel.co.jp
 

What's the status for these 2 patch sets?
Will they be applied into 2.4? or 2.5?

Just need choose one from both to apply? or need apply both to provide a good 
enough functionality of reconnection?

Thanks
Changchun


 
  ---
   hw/net/vhost_net.c | 126
 +
   hw/net/virtio-net.c|  62 
   hw/virtio/vhost.c  | 137
 +
   hw/virtio/virtio.c |  24 
   include/hw/virtio/vhost.h  |   8 +++
   include/hw/virtio/virtio.h |   1 +
   include/net/vhost_net.h|   7 +++
   include/sysemu/char.h  |   3 +
   net/vhost-user.c   |  24 
   qemu-char.c| 101 +
   10 files changed, 470 insertions(+), 23 deletions(-)
 
  diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index
  9bd360b..16a31d7 100644
  --- a/hw/net/vhost_net.c
  +++ b/hw/net/vhost_net.c
  @@ -167,6 +167,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions
 *options)
   if (r  0) {
   goto fail;
   }
  +net-dev.features |= 1  VIRTIO_NET_F_STATUS;
   if

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v5] vhost-user: add multi queue support

2015-07-08 Thread Ouyang, Changchun 


 -Original Message-
 From: snabb-de...@googlegroups.com [mailto:snabb-
 de...@googlegroups.com] On Behalf Of Maxime Leroy
 Sent: Thursday, July 9, 2015 6:01 AM
 To: Michael S. Tsirkin
 Cc: Ouyang, Changchun; snabb-de...@googlegroups.com; Marcel
 Apfelbaum; qemu-devel@nongnu.org; Nikolay Nikolaev; Luke Gorrie; Long,
 Thomas; rk...@redhat.com
 Subject: [snabb-devel] Re: [Qemu-devel] [PATCH v5] vhost-user: add multi
 queue support
 
 Hi Michael,
 
 On Wed, Jul 8, 2015 at 4:29 PM, Michael S. Tsirkin m...@redhat.com wrote:
  On Thu, May 28, 2015 at 09:23:06AM +0800, Ouyang Changchun wrote:
  Based on patch by Nikolay Nikolaev:
  Vhost-user will implement the multi queue support in a similar way to
  what vhost already has - a separate thread for each queue.
  To enable the multi queue functionality - a new command line
  parameter queues is introduced for the vhost-user netdev.
 
  Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
  Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
 
  So testing turned up a significant issue with the protocol extension
  in this one.  Specifically, remote has no idea how many queues guest
  actually wants to use (it's dynamic, guest changes this at any time).
  We need support for enabling and disabling queues dynamically.

Do you mean we need control queue to negotiate the actual queue number between
Guest and host? Or something like that

 
  Given we are past hard freeze, and given no one uses this yet (dpdk
  upstream did not merge supporting protocol), I think the best thing to
  do is to disable this functionality for 2.4.
  I will send a patch to do this shortly.
 
 You are making a wrong statement, we already use multiqueue for vhost-
 user and we expected to have this support officially integrated in qemu 2.4.
 
 Libvirt 1.2.17 has been released with multiqueue support for vhost-user.
 (http://libvirt.org/git/?p=libvirt.git;a=commit;h=366c22f2bcf1ddb8253c123f93
 fd18d1ba9eacd6)
 It checks against the version of qemu (i.e. 2.4)  to know if multiqueue is
 supported or not by qemu.
 (http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=7971723b985b9adc27122a
 3503e7ab38ced2b57f;hp=e7f5510ef2d28ca0ae0ed5751b1fd3018130d6c1)
 
 Dynamically enabling/disabling queue between host/guest is a nice feature
 to have.
 But it's not mandatory. You can still configure manually guest and host to use
 the same number of queues.

Same number of queues on host and guest can work normally, I have validated it 
with dpdk.

 

Maybe we could consider still having this in 2.4,
And have an enhancement patch set to implement dynamically enabling/disabling 
in 2.5 or 2.4.x
After extending the vhost-user spec. 

Thanks
Changchun

[Qemu-devel] [PATCH v5] vhost-user: add multi queue support

2015-05-27 Thread Ouyang Changchun 
Based on patch by Nikolay Nikolaev:
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue functionality - a new command line parameter
queues is introduced for the vhost-user netdev.

Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
Changes since v4:
 - remove the unnecessary trailing '\n'

Changes since v3:
 - fix one typo and wrap one long line

Changes since v2:
 - fix vq index issue for set_vring_call
   When it is the case of VHOST_SET_VRING_CALL, The vq_index is not initialized 
before it is used,
   thus it could be a random value. The random value leads to crash in vhost 
after passing down
   to vhost, as vhost use this random value to index an array index.
 - fix the typo in the doc and description
 - address vq index for reset_owner

Changes since v1:
 - use s-nc.info_str when bringing up/down the backend

 docs/specs/vhost-user.txt |  5 +
 hw/net/vhost_net.c|  3 ++-
 hw/virtio/vhost-user.c| 11 ++-
 net/vhost-user.c  | 37 -
 qapi-schema.json  |  6 +-
 qemu-options.hx   |  5 +++--
 6 files changed, 49 insertions(+), 18 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 650bb18..2c8e934 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -127,6 +127,11 @@ in the ancillary data:
 If Master is unable to send the full message or receives a wrong reply it will
 close the connection. An optional reconnection mechanism can be implemented.
 
+Multi queue support
+---
+The protocol supports multiple queues by setting all index fields in the sent
+messages to a properly calculated value.
+
 Message types
 -
 
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 47f8b89..426b23e 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -157,6 +157,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
 
 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;
 
 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force);
@@ -267,7 +268,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
 for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
 const VhostOps *vhost_ops = net-dev.vhost_ops;
 int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
-  NULL);
+  file);
 assert(r = 0);
 }
 }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e7ab829..d6f2163 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -210,7 +210,12 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 break;
 
 case VHOST_SET_OWNER:
+break;
+
 case VHOST_RESET_OWNER:
+memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
+msg.size = sizeof(m.state);
 break;
 
 case VHOST_SET_MEM_TABLE:
@@ -253,17 +258,20 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,
 case VHOST_SET_VRING_NUM:
 case VHOST_SET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 break;
 
 case VHOST_GET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 need_reply = 1;
 break;
 
 case VHOST_SET_VRING_ADDR:
 memcpy(msg.addr, arg, sizeof(struct vhost_vring_addr));
+msg.addr.index += dev-vq_index;
 msg.size = sizeof(m.addr);
 break;
 
@@ -271,7 +279,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 case VHOST_SET_VRING_CALL:
 case VHOST_SET_VRING_ERR:
 file = arg;
-msg.u64 = file-index  VHOST_USER_VRING_IDX_MASK;
+msg.u64 = (file-index + dev-vq_index)  VHOST_USER_VRING_IDX_MASK;
 msg.size = sizeof(m.u64);
 if (ioeventfd_enabled()  file-fd  0) {
 fds[fd_num++] = file-fd;
@@ -313,6 +321,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 error_report(Received bad msg size.);
 return -1;
 }
+msg.state.index -= dev-vq_index;
 memcpy(arg, msg.state, sizeof(struct vhost_vring_state));
 break;
 default:
diff --git a/net/vhost-user.c b/net/vhost-user.c
index 1d86a2b..904d8af 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -121,35 +121,39 @@ static void net_vhost_user_event(void *opaque, int

[Qemu-devel] [PATCH v4] vhost-user: add multi queue support

2015-05-22 Thread Ouyang Changchun 
Based on patch by Nikolay Nikolaev:
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue functionality - a new command line parameter
queues is introduced for the vhost-user netdev.

Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
Changes since v3:
 - fix one typo and wrap one long line

Changes since v2:
 - fix vq index issue for set_vring_call
   When it is the case of VHOST_SET_VRING_CALL, The vq_index is not initialized 
before it is used,
   thus it could be a random value. The random value leads to crash in vhost 
after passing down
   to vhost, as vhost use this random value to index an array index.
 - fix the typo in the doc and description
 - address vq index for reset_owner

Changes since v1:
 - use s-nc.info_str when bringing up/down the backend

 docs/specs/vhost-user.txt |  5 +
 hw/net/vhost_net.c|  3 ++-
 hw/virtio/vhost-user.c| 11 ++-
 net/vhost-user.c  | 37 -
 qapi-schema.json  |  6 +-
 qemu-options.hx   |  5 +++--
 6 files changed, 49 insertions(+), 18 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 650bb18..2c8e934 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -127,6 +127,11 @@ in the ancillary data:
 If Master is unable to send the full message or receives a wrong reply it will
 close the connection. An optional reconnection mechanism can be implemented.
 
+Multi queue support
+---
+The protocol supports multiple queues by setting all index fields in the sent
+messages to a properly calculated value.
+
 Message types
 -
 
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 47f8b89..426b23e 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -157,6 +157,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
 
 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;
 
 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force);
@@ -267,7 +268,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
 for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
 const VhostOps *vhost_ops = net-dev.vhost_ops;
 int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
-  NULL);
+  file);
 assert(r = 0);
 }
 }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e7ab829..d6f2163 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -210,7 +210,12 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 break;
 
 case VHOST_SET_OWNER:
+break;
+
 case VHOST_RESET_OWNER:
+memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
+msg.size = sizeof(m.state);
 break;
 
 case VHOST_SET_MEM_TABLE:
@@ -253,17 +258,20 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,
 case VHOST_SET_VRING_NUM:
 case VHOST_SET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 break;
 
 case VHOST_GET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 need_reply = 1;
 break;
 
 case VHOST_SET_VRING_ADDR:
 memcpy(msg.addr, arg, sizeof(struct vhost_vring_addr));
+msg.addr.index += dev-vq_index;
 msg.size = sizeof(m.addr);
 break;
 
@@ -271,7 +279,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 case VHOST_SET_VRING_CALL:
 case VHOST_SET_VRING_ERR:
 file = arg;
-msg.u64 = file-index  VHOST_USER_VRING_IDX_MASK;
+msg.u64 = (file-index + dev-vq_index)  VHOST_USER_VRING_IDX_MASK;
 msg.size = sizeof(m.u64);
 if (ioeventfd_enabled()  file-fd  0) {
 fds[fd_num++] = file-fd;
@@ -313,6 +321,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 error_report(Received bad msg size.);
 return -1;
 }
+msg.state.index -= dev-vq_index;
 memcpy(arg, msg.state, sizeof(struct vhost_vring_state));
 break;
 default:
diff --git a/net/vhost-user.c b/net/vhost-user.c
index 1d86a2b..41c8a27 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -121,35 +121,39 @@ static void net_vhost_user_event(void *opaque, int event)
 case CHR_EVENT_OPENED:

[Qemu-devel] [PATCH v3] vhost-user: add multi queue support

2015-05-21 Thread Ouyang Changchun 
From: Changchun Ouyang changchun.ouy...@intel.com

Based on patch by Nikolay Nikolaev:
Vhost-user will implement the multi queue support in a similar way
to what vhost already has - a separate thread for each queue.
To enable the multi queue funcionality - a new command line parameter
queues is introduced for the vhost-user netdev.

Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
Signed-off-by: Changchun Ouyang changchun.ouy...@intel.com
---
Changes since v2:
 - fix vq index issue for set_vring_call
   When it is the case of VHOST_SET_VRING_CALL, The vq_index is not initialized 
before it is used,
   thus it could be a random value. The random value lead to crash in vhost 
after passing down
   to vhost, as vhost use this random value to index an array index.
 - fix the typo in the doc and description
 - address vq index for reset_owner

Changes since v1:
 - use s-nc.info_str when bringing up/down the backend

 docs/specs/vhost-user.txt |  5 +
 hw/net/vhost_net.c|  3 ++-
 hw/virtio/vhost-user.c| 11 ++-
 net/vhost-user.c  | 37 -
 qapi-schema.json  |  6 +-
 qemu-options.hx   |  5 +++--
 6 files changed, 49 insertions(+), 18 deletions(-)

diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
index 650bb18..2c8e934 100644
--- a/docs/specs/vhost-user.txt
+++ b/docs/specs/vhost-user.txt
@@ -127,6 +127,11 @@ in the ancillary data:
 If Master is unable to send the full message or receives a wrong reply it will
 close the connection. An optional reconnection mechanism can be implemented.
 
+Multi queue support
+---
+The protocol supports multiple queues by setting all index fields in the sent
+messages to a properly calculated value.
+
 Message types
 -
 
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 47f8b89..426b23e 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -157,6 +157,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
 
 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;
 
 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force);
@@ -267,7 +268,7 @@ static void vhost_net_stop_one(struct vhost_net *net,
 for (file.index = 0; file.index  net-dev.nvqs; ++file.index) {
 const VhostOps *vhost_ops = net-dev.vhost_ops;
 int r = vhost_ops-vhost_call(net-dev, VHOST_RESET_OWNER,
-  NULL);
+  file);
 assert(r = 0);
 }
 }
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e7ab829..d6f2163 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -210,7 +210,12 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 break;
 
 case VHOST_SET_OWNER:
+break;
+
 case VHOST_RESET_OWNER:
+memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
+msg.size = sizeof(m.state);
 break;
 
 case VHOST_SET_MEM_TABLE:
@@ -253,17 +258,20 @@ static int vhost_user_call(struct vhost_dev *dev, 
unsigned long int request,
 case VHOST_SET_VRING_NUM:
 case VHOST_SET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 break;
 
 case VHOST_GET_VRING_BASE:
 memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
+msg.state.index += dev-vq_index;
 msg.size = sizeof(m.state);
 need_reply = 1;
 break;
 
 case VHOST_SET_VRING_ADDR:
 memcpy(msg.addr, arg, sizeof(struct vhost_vring_addr));
+msg.addr.index += dev-vq_index;
 msg.size = sizeof(m.addr);
 break;
 
@@ -271,7 +279,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 case VHOST_SET_VRING_CALL:
 case VHOST_SET_VRING_ERR:
 file = arg;
-msg.u64 = file-index  VHOST_USER_VRING_IDX_MASK;
+msg.u64 = (file-index + dev-vq_index)  VHOST_USER_VRING_IDX_MASK;
 msg.size = sizeof(m.u64);
 if (ioeventfd_enabled()  file-fd  0) {
 fds[fd_num++] = file-fd;
@@ -313,6 +321,7 @@ static int vhost_user_call(struct vhost_dev *dev, unsigned 
long int request,
 error_report(Received bad msg size.);
 return -1;
 }
+msg.state.index -= dev-vq_index;
 memcpy(arg, msg.state, sizeof(struct vhost_vring_state));
 break;
 default:
diff --git a/net/vhost-user.c b/net/vhost-user.c
index 1d86a2b..41c8a27 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -121,35 +121,39 @@ static void net_vhost_user_event(void *opaque, int event)
 case CHR_EVENT_OPENED:
 vhost_user_start(s);

Re: [Qemu-devel] [snabb-devel] Re: [PATCH v2] vhost-user: add multi queue support

2015-04-08 Thread Ouyang, Changchun 
Hi guys,

 -Original Message-
 From: snabb-de...@googlegroups.com [mailto:snabb-
 de...@googlegroups.com] On Behalf Of Michael S. Tsirkin
 Sent: Monday, April 6, 2015 11:07 PM
 To: Nikolay Nikolaev
 Cc: Long, Thomas; snabb-de...@googlegroups.com; ebl...@redhat.com;
 qemu-devel@nongnu.org; t...@virtualopensystems.com
 Subject: [snabb-devel] Re: [PATCH v2] vhost-user: add multi queue support
 
 On Sat, Jan 24, 2015 at 02:22:29PM +0200, Nikolay Nikolaev wrote:
  Vhost-user will implement the multiqueueu support in a similar way to
  what
 
 multiqueue
 
  vhost already has - a separate thread for each queue.
 
  To enable the multiqueue funcionality - a new command line parameter
  queues is introduced for the vhost-user netdev.
 
  Changes since v1:
   - use s-nc.info_str when bringing up/down the backend
 
  Signed-off-by: Nikolay Nikolaev n.nikol...@virtualopensystems.com
  ---
   docs/specs/vhost-user.txt |5 +
   hw/virtio/vhost-user.c|6 +-
   net/vhost-user.c  |   39 +--
   qapi-schema.json  |6 +-
   qemu-options.hx   |5 +++--
   5 files changed, 43 insertions(+), 18 deletions(-)
 
  diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
  index 650bb18..d7b208c 100644
  --- a/docs/specs/vhost-user.txt
  +++ b/docs/specs/vhost-user.txt
 
 I've been thinking that the protocol might be a useful addition to the virtio
 spec. For this, as a minimum you would have to submit this document as a
 comment to virtio TC with a proposal to include it in the virtio spec.
 See
 https://www.oasis-
 open.org/committees/comments/index.php?wg_abbrev=virtio
 
 Can you do this?
 
 We can take it from there, though I would encourage your company to join
 as a contributor.
 
 
  @@ -127,6 +127,11 @@ in the ancillary data:
   If Master is unable to send the full message or receives a wrong
  reply it will  close the connection. An optional reconnection mechanism can
 be implemented.
 
  +Multi queue suport
  +-
  +The protocol supports multiple queues by setting all index fields in
  +the sent messages to a properly calculated value.
  +
 
 Something that's not clear from this document is what happens with control
 VQ.
 Can you clarify please?
 
 
   Message types
   -
 
  diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
  aefe0bb..83ebcaa 100644
  --- a/hw/virtio/vhost-user.c
  +++ b/hw/virtio/vhost-user.c
  @@ -253,17 +253,20 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
   case VHOST_SET_VRING_NUM:
   case VHOST_SET_VRING_BASE:
   memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
   msg.size = sizeof(m.state);
   break;
 
   case VHOST_GET_VRING_BASE:
   memcpy(msg.state, arg, sizeof(struct vhost_vring_state));
  +msg.state.index += dev-vq_index;
   msg.size = sizeof(m.state);
   need_reply = 1;
   break;
 
   case VHOST_SET_VRING_ADDR:
   memcpy(msg.addr, arg, sizeof(struct vhost_vring_addr));
  +msg.addr.index += dev-vq_index;
   msg.size = sizeof(m.addr);
   break;
 
  @@ -271,7 +274,7 @@ static int vhost_user_call(struct vhost_dev *dev,
 unsigned long int request,
   case VHOST_SET_VRING_CALL:
   case VHOST_SET_VRING_ERR:
   file = arg;
  -msg.u64 = file-index  VHOST_USER_VRING_IDX_MASK;
  +msg.u64 = (file-index + dev-vq_index) 
  + VHOST_USER_VRING_IDX_MASK;

I identify one vq_index issue here when it is the case of VHOST_SET_VRING_CALL,
The vq_index is not initialized before it is used here, so it could be a random 
value.
It leads to error in vhost, when this random value is passed to vhost and vhost 
use this random value to set the vring call.
 
I have a quick fix for this, code changes as the following:
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 4e3a061..2fbdb93 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -157,6 +157,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)

 net-dev.nvqs = 2;
 net-dev.vqs = net-vqs;
+net-dev.vq_index = net-nc-queue_index;

 r = vhost_dev_init(net-dev, options-opaque,
options-backend_type, options-force);
diff --git a/net/vhost-user.c b/net/vhost-user.c
index a0b4af2..b27190f 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -152,6 +152,7 @@ static int net_vhost_user_init(NetClientState *peer, const 
char *device,
 s-nc.receive_disabled = 1;
 s-chr = chr;
 s-vhostforce = vhostforce;
+s-nc.queue_index = i;

 qemu_chr_add_handlers(s-chr, NULL, NULL, net_vhost_user_event, s);
 }

Would you guys have a look at this issue?
And I also met other issues when I try to run with 2 virtio ports, each one has 
2 queues
But not root cause yet, need further investigate,
So I think it

[Qemu-devel] a question for control queue

2015-01-09 Thread Ouyang, Changchun 
Hi all,
I have a question about the control queue in qemu,
When the qemu have configured the control queue, and guest also negotiated the 
control queue successfully with qemu,
Will the qemu will let vhost know guest try to use control queue to send some 
commands?

Or could the vhost also setup  the control queue to communicate directly with 
control queue on guest?
How to do that?
Hope anyone could shed some lights on  this question.
Thanks in advance!

Thanks
Changchun