subject:"\[ovs\-dev\] \[PATCH v1 branch\-2.9\] netdev\-dpdk\: Reintroduce shared mempools."

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

2018-02-13 Thread Jan Scheurich

Thanks, Ian!
This will give us time to come up with a proper solution for 2.10. Let's work 
on that now.
/Jan

> -Original Message-
> From: ovs-dev-boun...@openvswitch.org 
> [mailto:ovs-dev-boun...@openvswitch.org] On Behalf Of Stokes, Ian
> Sent: Tuesday, 13 February, 2018 16:52
> To: Stokes, Ian <ian.sto...@intel.com>; Kevin Traynor <ktray...@redhat.com>; 
> d...@openvswitch.org
> Cc: Ilya Maximets <i.maxim...@samsung.com>; Antonio Fischetti 
> <antonio.fische...@gmail.com>
> Subject: Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared 
> mempools.
> 
> > >
> > > On 02/13/2018 10:59 AM, Ian Stokes wrote:
> > > > This commit manually reverts the current per port mempool model to
> > > > the previous shared mempool model for DPDK ports.
> > > >
> > > > OVS previously used a shared mempool model for ports with the same
> > > > MTU configuration. This was replaced by a per port mempool model to
> > > > address issues flagged by users such as:
> > > >
> > > > https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/04
> > > > 25
> > > > 60.html
> > > >
> > > > However the per port model has a number of issues including:
> > > >
> > > > 1. Requires an increase in memory resource requirements to support
> > > > the same number of ports as the shared port model.
> > > > 2. Incorrect algorithm for mbuf provisioning for each mempool.
> > > >
> > > > These are considered blocking factors for current deployments of OVS
> > > > when upgrading to OVS 2.9 as a  user may have to redimension memory
> > > > for the same deployment configuration. This may not be possible for
> > > users.
> > > >
> > > > For clarity, the commits whose changes are removed include the
> > > > following:
> > > >
> > > > netdev-dpdk: Create separate memory pool for each port: d555d9b
> > > > netdev-dpdk: fix management of pre-existing mempools: b6b26021d Fix
> > > > mempool names to reflect socket id: f06546a
> > > > netdev-dpdk: skip init for existing mempools: 837c176
> > > > netdev-dpdk: manage failure in mempool name creation: 65056fd
> > > > netdev-dpdk: Reword mp_size as n_mbufs: ad9b5b9
> > > > netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free: a08a115
> > > > netdev-dpdk: Fix mp_name leak on snprintf failure: ec6edc8
> > > > netdev-dpdk: Fix dpdk_mp leak in case of EEXIST: 173ef76
> > > > netdev-dpdk: Factor out struct dpdk_mp: 24e78f9
> > > > netdev-dpdk: Remove unused MAX_NB_MBUF: bc57ed9
> > > > netdev-dpdk: Fix mempool creation with large MTU: af5b0da
> > > >
> > > > Due to the number of commits and period of time they were introduced
> > > > over, a simple revert was not possible. All code from the commits
> > > > above is removed and the shared mempool code reintroduced as it was
> > > > before its replacement.
> > > >
> > > > Code introduced by commit
> > > >
> > > > netdev-dpdk: Add debug appctl to get mempool information: be48173
> > > >
> > > > has been modified to work with the shared mempool model.
> > > >
> > >
> > > There is a couple of small changes for coding stds to this version
> > > from the rfc, but one is not correct (see below). Apart from that...
> >
> > Thanks for flagging this.
> 
> I've pushed this to dpdk_merge_2_9 and its part of the branch 2.9 pull 
> request.
> 
> https://mail.openvswitch.org/pipermail/ovs-dev/2018-February/344413.html
> 
> Note this revert only applies to branch 2.9, I have not applied it to master.
> 
> Thanks
> Ian
> 
> >
> > >
> > > Acked-by: Kevin Traynor <ktray...@redhat.com>
> > > Tested-by: Kevin Traynor <ktray...@redhat.com>
> > >
> >
> > ...
> >
> > > >  ovs_mutex_lock(_mp_mutex);
> > > > -VLOG_DBG("Releasing \"%s\" mempool", mp->name);
> > > > -rte_mempool_free(mp);
> > > > +ovs_assert(dmp->refcount);
> > > > +
> > > > +if (! --dmp->refcount) {
> > >
> > > The space should not be added between ! and --
> > >
> > > http://docs.openvswitch.org/en/latest/internals/contributing/coding-
> > > style/#expressions
> >
> > Good catch, I can fix this before committing if there are no other
&g

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

2018-02-13 Thread Stokes, Ian

> >
> > On 02/13/2018 10:59 AM, Ian Stokes wrote:
> > > This commit manually reverts the current per port mempool model to
> > > the previous shared mempool model for DPDK ports.
> > >
> > > OVS previously used a shared mempool model for ports with the same
> > > MTU configuration. This was replaced by a per port mempool model to
> > > address issues flagged by users such as:
> > >
> > > https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/04
> > > 25
> > > 60.html
> > >
> > > However the per port model has a number of issues including:
> > >
> > > 1. Requires an increase in memory resource requirements to support
> > > the same number of ports as the shared port model.
> > > 2. Incorrect algorithm for mbuf provisioning for each mempool.
> > >
> > > These are considered blocking factors for current deployments of OVS
> > > when upgrading to OVS 2.9 as a  user may have to redimension memory
> > > for the same deployment configuration. This may not be possible for
> > users.
> > >
> > > For clarity, the commits whose changes are removed include the
> > > following:
> > >
> > > netdev-dpdk: Create separate memory pool for each port: d555d9b
> > > netdev-dpdk: fix management of pre-existing mempools: b6b26021d Fix
> > > mempool names to reflect socket id: f06546a
> > > netdev-dpdk: skip init for existing mempools: 837c176
> > > netdev-dpdk: manage failure in mempool name creation: 65056fd
> > > netdev-dpdk: Reword mp_size as n_mbufs: ad9b5b9
> > > netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free: a08a115
> > > netdev-dpdk: Fix mp_name leak on snprintf failure: ec6edc8
> > > netdev-dpdk: Fix dpdk_mp leak in case of EEXIST: 173ef76
> > > netdev-dpdk: Factor out struct dpdk_mp: 24e78f9
> > > netdev-dpdk: Remove unused MAX_NB_MBUF: bc57ed9
> > > netdev-dpdk: Fix mempool creation with large MTU: af5b0da
> > >
> > > Due to the number of commits and period of time they were introduced
> > > over, a simple revert was not possible. All code from the commits
> > > above is removed and the shared mempool code reintroduced as it was
> > > before its replacement.
> > >
> > > Code introduced by commit
> > >
> > > netdev-dpdk: Add debug appctl to get mempool information: be48173
> > >
> > > has been modified to work with the shared mempool model.
> > >
> >
> > There is a couple of small changes for coding stds to this version
> > from the rfc, but one is not correct (see below). Apart from that...
> 
> Thanks for flagging this.

I've pushed this to dpdk_merge_2_9 and its part of the branch 2.9 pull request.

https://mail.openvswitch.org/pipermail/ovs-dev/2018-February/344413.html

Note this revert only applies to branch 2.9, I have not applied it to master.

Thanks
Ian

> 
> >
> > Acked-by: Kevin Traynor 
> > Tested-by: Kevin Traynor 
> >
> 
> ...
> 
> > >  ovs_mutex_lock(_mp_mutex);
> > > -VLOG_DBG("Releasing \"%s\" mempool", mp->name);
> > > -rte_mempool_free(mp);
> > > +ovs_assert(dmp->refcount);
> > > +
> > > +if (! --dmp->refcount) {
> >
> > The space should not be added between ! and --
> >
> > http://docs.openvswitch.org/en/latest/internals/contributing/coding-
> > style/#expressions
> 
> Good catch, I can fix this before committing if there are no other
> comments.
> 
> Thanks
> Ian
> >
> > > +ovs_list_remove(>list_node);
> > > +rte_mempool_free(dmp->mp);
> > > +rte_free(dmp);
> > > + }
> > >  ovs_mutex_unlock(_mp_mutex);  }
> > >
> > > -/* Tries to allocate a new mempool - or re-use an existing one
> > > where
> > > - * appropriate - on requested_socket_id with a size determined by
> > > - * requested_mtu and requested Rx/Tx queues.
> > > - * On success - or when re-using an existing mempool - the new
> > > configuration
> > > - * will be applied.
> > > +/* Tries to allocate new mempool on requested_socket_id with
> > > + * mbuf size corresponding to requested_mtu.
> > > + * On success new configuration will be applied.
> > >   * On error, device will be left unchanged. */  static int
> > > netdev_dpdk_mempool_configure(struct netdev_dpdk *dev)
> > >  OVS_REQUIRES(dev->mutex)
> > >  {
> > >  uint32_t buf_size = dpdk_buf_size(dev->requested_mtu);
> > > -struct rte_mempool *mp;
> > > -int ret = 0;
> > > +struct dpdk_mp *mp;
> > >
> > > -mp = dpdk_mp_create(dev, FRAME_LEN_TO_MTU(buf_size));
> > > +mp = dpdk_mp_get(dev->requested_socket_id,
> > > + FRAME_LEN_TO_MTU(buf_size));
> > >  if (!mp) {
> > >  VLOG_ERR("Failed to create memory pool for netdev "
> > >   "%s, with MTU %d on socket %d: %s\n",
> > >   dev->up.name, dev->requested_mtu, dev-
> > >requested_socket_id,
> > > - rte_strerror(rte_errno));
> > > -ret = rte_errno;
> > > +rte_strerror(rte_errno));
> > > +return rte_errno;
> > >  } else {
> > > -/* If a new MTU was requested and its rounded value equals
> the
> > one
> > > -

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

2018-02-13 Thread Stokes, Ian

> -Original Message-
> From: Kevin Traynor [mailto:ktray...@redhat.com]
> Sent: Tuesday, February 13, 2018 11:36 AM
> To: Stokes, Ian ; d...@openvswitch.org
> Cc: Antonio Fischetti ; Ilya Maximets
> ; Jan Scheurich 
> Subject: Re: [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared
> mempools.
> 
> On 02/13/2018 10:59 AM, Ian Stokes wrote:
> > This commit manually reverts the current per port mempool model to the
> > previous shared mempool model for DPDK ports.
> >
> > OVS previously used a shared mempool model for ports with the same MTU
> > configuration. This was replaced by a per port mempool model to
> > address issues flagged by users such as:
> >
> > https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/0425
> > 60.html
> >
> > However the per port model has a number of issues including:
> >
> > 1. Requires an increase in memory resource requirements to support the
> > same number of ports as the shared port model.
> > 2. Incorrect algorithm for mbuf provisioning for each mempool.
> >
> > These are considered blocking factors for current deployments of OVS
> > when upgrading to OVS 2.9 as a  user may have to redimension memory
> > for the same deployment configuration. This may not be possible for
> users.
> >
> > For clarity, the commits whose changes are removed include the
> > following:
> >
> > netdev-dpdk: Create separate memory pool for each port: d555d9b
> > netdev-dpdk: fix management of pre-existing mempools: b6b26021d Fix
> > mempool names to reflect socket id: f06546a
> > netdev-dpdk: skip init for existing mempools: 837c176
> > netdev-dpdk: manage failure in mempool name creation: 65056fd
> > netdev-dpdk: Reword mp_size as n_mbufs: ad9b5b9
> > netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free: a08a115
> > netdev-dpdk: Fix mp_name leak on snprintf failure: ec6edc8
> > netdev-dpdk: Fix dpdk_mp leak in case of EEXIST: 173ef76
> > netdev-dpdk: Factor out struct dpdk_mp: 24e78f9
> > netdev-dpdk: Remove unused MAX_NB_MBUF: bc57ed9
> > netdev-dpdk: Fix mempool creation with large MTU: af5b0da
> >
> > Due to the number of commits and period of time they were introduced
> > over, a simple revert was not possible. All code from the commits
> > above is removed and the shared mempool code reintroduced as it was
> > before its replacement.
> >
> > Code introduced by commit
> >
> > netdev-dpdk: Add debug appctl to get mempool information: be48173
> >
> > has been modified to work with the shared mempool model.
> >
> 
> There is a couple of small changes for coding stds to this version from
> the rfc, but one is not correct (see below). Apart from that...

Thanks for flagging this.

> 
> Acked-by: Kevin Traynor 
> Tested-by: Kevin Traynor 
> 

...

> >  ovs_mutex_lock(_mp_mutex);
> > -VLOG_DBG("Releasing \"%s\" mempool", mp->name);
> > -rte_mempool_free(mp);
> > +ovs_assert(dmp->refcount);
> > +
> > +if (! --dmp->refcount) {
> 
> The space should not be added between ! and --
> 
> http://docs.openvswitch.org/en/latest/internals/contributing/coding-
> style/#expressions

Good catch, I can fix this before committing if there are no other comments.

Thanks
Ian
> 
> > +ovs_list_remove(>list_node);
> > +rte_mempool_free(dmp->mp);
> > +rte_free(dmp);
> > + }
> >  ovs_mutex_unlock(_mp_mutex);  }
> >
> > -/* Tries to allocate a new mempool - or re-use an existing one where
> > - * appropriate - on requested_socket_id with a size determined by
> > - * requested_mtu and requested Rx/Tx queues.
> > - * On success - or when re-using an existing mempool - the new
> > configuration
> > - * will be applied.
> > +/* Tries to allocate new mempool on requested_socket_id with
> > + * mbuf size corresponding to requested_mtu.
> > + * On success new configuration will be applied.
> >   * On error, device will be left unchanged. */  static int
> > netdev_dpdk_mempool_configure(struct netdev_dpdk *dev)
> >  OVS_REQUIRES(dev->mutex)
> >  {
> >  uint32_t buf_size = dpdk_buf_size(dev->requested_mtu);
> > -struct rte_mempool *mp;
> > -int ret = 0;
> > +struct dpdk_mp *mp;
> >
> > -mp = dpdk_mp_create(dev, FRAME_LEN_TO_MTU(buf_size));
> > +mp = dpdk_mp_get(dev->requested_socket_id,
> > + FRAME_LEN_TO_MTU(buf_size));
> >  if (!mp) {
> >  VLOG_ERR("Failed to create memory pool for netdev "
> >   "%s, with MTU %d on socket %d: %s\n",
> >   dev->up.name, dev->requested_mtu, dev-
> >requested_socket_id,
> > - rte_strerror(rte_errno));
> > -ret = rte_errno;
> > +rte_strerror(rte_errno));
> > +return rte_errno;
> >  } else {
> > -/* If a new MTU was requested and its rounded value equals the
> one
> > - * that is currently used, then the existing mempool is
> returned. */
> > -if

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

2018-02-13 Thread Kevin Traynor

On 02/13/2018 10:59 AM, Ian Stokes wrote:
> This commit manually reverts the current per port mempool model to the
> previous shared mempool model for DPDK ports.
> 
> OVS previously used a shared mempool model for ports with the same MTU
> configuration. This was replaced by a per port mempool model to address
> issues flagged by users such as:
> 
> https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/042560.html
> 
> However the per port model has a number of issues including:
> 
> 1. Requires an increase in memory resource requirements to support the same
> number of ports as the shared port model.
> 2. Incorrect algorithm for mbuf provisioning for each mempool.
> 
> These are considered blocking factors for current deployments of OVS when
> upgrading to OVS 2.9 as a  user may have to redimension memory for the same
> deployment configuration. This may not be possible for users.
> 
> For clarity, the commits whose changes are removed include the
> following:
> 
> netdev-dpdk: Create separate memory pool for each port: d555d9b
> netdev-dpdk: fix management of pre-existing mempools: b6b26021d
> Fix mempool names to reflect socket id: f06546a
> netdev-dpdk: skip init for existing mempools: 837c176
> netdev-dpdk: manage failure in mempool name creation: 65056fd
> netdev-dpdk: Reword mp_size as n_mbufs: ad9b5b9
> netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free: a08a115
> netdev-dpdk: Fix mp_name leak on snprintf failure: ec6edc8
> netdev-dpdk: Fix dpdk_mp leak in case of EEXIST: 173ef76
> netdev-dpdk: Factor out struct dpdk_mp: 24e78f9
> netdev-dpdk: Remove unused MAX_NB_MBUF: bc57ed9
> netdev-dpdk: Fix mempool creation with large MTU: af5b0da
> 
> Due to the number of commits and period of time they were introduced
> over, a simple revert was not possible. All code from the commits above
> is removed and the shared mempool code reintroduced as it was before its
> replacement.
> 
> Code introduced by commit
> 
> netdev-dpdk: Add debug appctl to get mempool information: be48173
> 
> has been modified to work with the shared mempool model.
> 

There is a couple of small changes for coding stds to this version from
the rfc, but one is not correct (see below). Apart from that...

Acked-by: Kevin Traynor 
Tested-by: Kevin Traynor 

> Cc: Antonio Fischetti 
> Cc: Ilya Maximets 
> Cc: Kevin Traynor 
> Cc: Jan Scheurich 
> Signed-off-by: Ian Stokes 
> ---
>  lib/netdev-dpdk.c | 246 
> ++
>  1 file changed, 138 insertions(+), 108 deletions(-)
> 
> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
> index 94fb163..6f3378b 100644
> --- a/lib/netdev-dpdk.c
> +++ b/lib/netdev-dpdk.c
> @@ -91,13 +91,24 @@ static struct vlog_rate_limit rl = 
> VLOG_RATE_LIMIT_INIT(5, 20);
>  #define NETDEV_DPDK_MBUF_ALIGN  1024
>  #define NETDEV_DPDK_MAX_PKT_LEN 9728
>  
> -/* Min number of packets in the mempool.  OVS tries to allocate a mempool 
> with
> - * roughly estimated number of mbufs: if this fails (because the system 
> doesn't
> - * have enough hugepages) we keep halving the number until the allocation
> - * succeeds or we reach MIN_NB_MBUF */
> +/* Max and min number of packets in the mempool.  OVS tries to allocate a
> + * mempool with MAX_NB_MBUF: if this fails (because the system doesn't have
> + * enough hugepages) we keep halving the number until the allocation succeeds
> + * or we reach MIN_NB_MBUF */
> +
> +#define MAX_NB_MBUF  (4096 * 64)
>  #define MIN_NB_MBUF  (4096 * 4)
>  #define MP_CACHE_SZ  RTE_MEMPOOL_CACHE_MAX_SIZE
>  
> +/* MAX_NB_MBUF can be divided by 2 many times, until MIN_NB_MBUF */
> +BUILD_ASSERT_DECL(MAX_NB_MBUF % ROUND_DOWN_POW2(MAX_NB_MBUF / MIN_NB_MBUF)
> +  == 0);
> +
> +/* The smallest possible NB_MBUF that we're going to try should be a multiple
> + * of MP_CACHE_SZ. This is advised by DPDK documentation. */
> +BUILD_ASSERT_DECL((MAX_NB_MBUF / ROUND_DOWN_POW2(MAX_NB_MBUF / MIN_NB_MBUF))
> +  % MP_CACHE_SZ == 0);
> +
>  /*
>   * DPDK XSTATS Counter names definition
>   */
> @@ -295,6 +306,19 @@ static struct ovs_list dpdk_list 
> OVS_GUARDED_BY(dpdk_mutex)
>  static struct ovs_mutex dpdk_mp_mutex OVS_ACQ_AFTER(dpdk_mutex)
>  = OVS_MUTEX_INITIALIZER;
>  
> +static struct ovs_list dpdk_mp_list OVS_GUARDED_BY(dpdk_mp_mutex)
> += OVS_LIST_INITIALIZER(_mp_list);
> +
> +
> +struct dpdk_mp {
> + struct rte_mempool *mp;
> + int mtu;
> + int socket_id;
> + int refcount;
> + struct ovs_list list_node OVS_GUARDED_BY(dpdk_mp_mutex);
> + };
> +
> +
>  /* There should be one 'struct dpdk_tx_queue' created for
>   * each cpu core. */
>  struct dpdk_tx_queue {
> @@ -371,7 +395,7 @@ struct netdev_dpdk {
>  
>  PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1,
>  struct

[ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

2018-02-13 Thread Ian Stokes

This commit manually reverts the current per port mempool model to the
previous shared mempool model for DPDK ports.

OVS previously used a shared mempool model for ports with the same MTU
configuration. This was replaced by a per port mempool model to address
issues flagged by users such as:

https://mail.openvswitch.org/pipermail/ovs-discuss/2016-September/042560.html

However the per port model has a number of issues including:

1. Requires an increase in memory resource requirements to support the same
number of ports as the shared port model.
2. Incorrect algorithm for mbuf provisioning for each mempool.

These are considered blocking factors for current deployments of OVS when
upgrading to OVS 2.9 as a  user may have to redimension memory for the same
deployment configuration. This may not be possible for users.

For clarity, the commits whose changes are removed include the
following:

netdev-dpdk: Create separate memory pool for each port: d555d9b
netdev-dpdk: fix management of pre-existing mempools: b6b26021d
Fix mempool names to reflect socket id: f06546a
netdev-dpdk: skip init for existing mempools: 837c176
netdev-dpdk: manage failure in mempool name creation: 65056fd
netdev-dpdk: Reword mp_size as n_mbufs: ad9b5b9
netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free: a08a115
netdev-dpdk: Fix mp_name leak on snprintf failure: ec6edc8
netdev-dpdk: Fix dpdk_mp leak in case of EEXIST: 173ef76
netdev-dpdk: Factor out struct dpdk_mp: 24e78f9
netdev-dpdk: Remove unused MAX_NB_MBUF: bc57ed9
netdev-dpdk: Fix mempool creation with large MTU: af5b0da

Due to the number of commits and period of time they were introduced
over, a simple revert was not possible. All code from the commits above
is removed and the shared mempool code reintroduced as it was before its
replacement.

Code introduced by commit

netdev-dpdk: Add debug appctl to get mempool information: be48173

has been modified to work with the shared mempool model.

Cc: Antonio Fischetti 
Cc: Ilya Maximets 
Cc: Kevin Traynor 
Cc: Jan Scheurich 
Signed-off-by: Ian Stokes 
---
 lib/netdev-dpdk.c | 246 ++
 1 file changed, 138 insertions(+), 108 deletions(-)

diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index 94fb163..6f3378b 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -91,13 +91,24 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 
20);
 #define NETDEV_DPDK_MBUF_ALIGN  1024
 #define NETDEV_DPDK_MAX_PKT_LEN 9728
 
-/* Min number of packets in the mempool.  OVS tries to allocate a mempool with
- * roughly estimated number of mbufs: if this fails (because the system doesn't
- * have enough hugepages) we keep halving the number until the allocation
- * succeeds or we reach MIN_NB_MBUF */
+/* Max and min number of packets in the mempool.  OVS tries to allocate a
+ * mempool with MAX_NB_MBUF: if this fails (because the system doesn't have
+ * enough hugepages) we keep halving the number until the allocation succeeds
+ * or we reach MIN_NB_MBUF */
+
+#define MAX_NB_MBUF  (4096 * 64)
 #define MIN_NB_MBUF  (4096 * 4)
 #define MP_CACHE_SZ  RTE_MEMPOOL_CACHE_MAX_SIZE
 
+/* MAX_NB_MBUF can be divided by 2 many times, until MIN_NB_MBUF */
+BUILD_ASSERT_DECL(MAX_NB_MBUF % ROUND_DOWN_POW2(MAX_NB_MBUF / MIN_NB_MBUF)
+  == 0);
+
+/* The smallest possible NB_MBUF that we're going to try should be a multiple
+ * of MP_CACHE_SZ. This is advised by DPDK documentation. */
+BUILD_ASSERT_DECL((MAX_NB_MBUF / ROUND_DOWN_POW2(MAX_NB_MBUF / MIN_NB_MBUF))
+  % MP_CACHE_SZ == 0);
+
 /*
  * DPDK XSTATS Counter names definition
  */
@@ -295,6 +306,19 @@ static struct ovs_list dpdk_list OVS_GUARDED_BY(dpdk_mutex)
 static struct ovs_mutex dpdk_mp_mutex OVS_ACQ_AFTER(dpdk_mutex)
 = OVS_MUTEX_INITIALIZER;
 
+static struct ovs_list dpdk_mp_list OVS_GUARDED_BY(dpdk_mp_mutex)
+= OVS_LIST_INITIALIZER(_mp_list);
+
+
+struct dpdk_mp {
+ struct rte_mempool *mp;
+ int mtu;
+ int socket_id;
+ int refcount;
+ struct ovs_list list_node OVS_GUARDED_BY(dpdk_mp_mutex);
+ };
+
+
 /* There should be one 'struct dpdk_tx_queue' created for
  * each cpu core. */
 struct dpdk_tx_queue {
@@ -371,7 +395,7 @@ struct netdev_dpdk {
 
 PADDED_MEMBERS_CACHELINE_MARKER(CACHE_LINE_SIZE, cacheline1,
 struct ovs_mutex mutex OVS_ACQ_AFTER(dpdk_mutex);
-struct rte_mempool *mp;
+struct dpdk_mp *dpdk_mp;
 
 /* virtio identifier for vhost devices */
 ovsrcu_index vid;
@@ -510,133 +534,132 @@ ovs_rte_pktmbuf_init(struct rte_mempool *mp OVS_UNUSED,
 dp_packet_init_dpdk((struct dp_packet *) pkt, pkt->buf_len);
 }
 
-/* Returns a valid pointer when either of the following is true:
- *  - a new mempool was just created;
- *  - a matching mempool already exists. */
-static struct

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

Re: [ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

[ovs-dev] [PATCH v1 branch-2.9] netdev-dpdk: Reintroduce shared mempools.

5 matches

Site Navigation

Mail list logo

Footer information