Re: [ovs-dev] [PATCH v5 1/2] dpif-netdev: Add SMC cache after EMC cache
> Acked-by: Billy O'Mahony
>
Thanks to all for the work on this.
I've applied this to dpdk_merge branch, it will be part of this week's pull
request.
I rolled patch 2 of the series into the same commit as I don't think it makes
sense to have a broken unit test for the 1st commit.
Thanks
Ian
> > -Original Message-
> > From: Wang, Yipeng1
> > Sent: Tuesday, July 10, 2018 11:14 AM
> > To: d...@openvswitch.org; jan.scheur...@ericsson.com; O Mahony, Billy
> >
> > Cc: Wang, Yipeng1 ; Stokes, Ian
> > ; b...@ovn.org
> > Subject: [PATCH v5 1/2] dpif-netdev: Add SMC cache after EMC cache
> >
> > This patch adds a signature match cache (SMC) after exact match cache
> (EMC).
> > The difference between SMC and EMC is SMC only stores a signature of a
> > flow thus it is much more memory efficient. With same memory space,
> > EMC can store 8k flows while SMC can store 1M flows. It is generally
> > beneficial to turn on SMC but turn off EMC when traffic flow count is
> much larger than EMC size.
> >
> > SMC cache will map a signature to an dp_netdev_flow index in
> > flow_table. Thus, we add two new APIs in cmap for lookup key by index
> and lookup index by key.
> >
> > For now, SMC is an experimental feature that it is turned off by
> > default. One can turn it on using ovsdb options.
> >
> > Signed-off-by: Yipeng Wang
> > Co-authored-by: Jan Scheurich
> > Signed-off-by: Jan Scheurich
> > ---
> > Documentation/topics/dpdk/bridge.rst | 15 ++
> > NEWS | 2 +
> > lib/cmap.c | 74
> > lib/cmap.h | 11 ++
> > lib/dpif-netdev-perf.h | 1 +
> > lib/dpif-netdev.c| 329
> +++
> > tests/pmd.at | 1 +
> > vswitchd/vswitch.xml | 13 ++
> > 8 files changed, 409 insertions(+), 37 deletions(-)
> >
> > diff --git a/Documentation/topics/dpdk/bridge.rst
> > b/Documentation/topics/dpdk/bridge.rst
> > index 63f8a62..df74c02 100644
> > --- a/Documentation/topics/dpdk/bridge.rst
> > +++ b/Documentation/topics/dpdk/bridge.rst
> > @@ -102,3 +102,18 @@ For certain traffic profiles with many parallel
> > flows, it's recommended to set ``N`` to '0' to achieve higher
> forwarding performance.
> >
> > For more information on the EMC refer to :doc:`/intro/install/dpdk` .
> > +
> > +
> > +SMC cache (experimental)
> > +-
> > +
> > +SMC cache or signature match cache is a new cache level after EMC
> cache.
> > +The difference between SMC and EMC is SMC only stores a signature of
> > +a flow thus it is much more memory efficient. With same memory space,
> > +EMC can store 8k flows while SMC can store 1M flows. When traffic
> > +flow count is much larger than EMC size, it is generally beneficial
> > +to turn off EMC and turn on SMC. It is currently turned off by
> > +default and an
> > experimental feature.
> > +
> > +To turn on SMC::
> > +
> > +$ ovs-vsctl --no-wait set Open_vSwitch .
> > + other_config:smc-enable=true
> > diff --git a/NEWS b/NEWS
> > index 92e9b92..f30a1e0 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -44,6 +44,8 @@ Post-v2.9.0
> > ovs-appctl dpif-netdev/pmd-perf-show
> > * Supervision of PMD performance metrics and logging of suspicious
> > iterations
> > + * Add signature match cache (SMC) as experimental feature. When
> > + turned
> > on,
> > + it improves throughput when traffic has many more flows than EMC
> size.
> > - ERSPAN:
> > * Implemented ERSPAN protocol (draft-foschiano-erspan-00.txt) for
> > both kernel datapath and userspace datapath.
> > diff --git a/lib/cmap.c b/lib/cmap.c
> > index 07719a8..cb9cd32 100644
> > --- a/lib/cmap.c
> > +++ b/lib/cmap.c
> > @@ -373,6 +373,80 @@ cmap_find(const struct cmap *cmap, uint32_t hash)
> > hash);
> > }
> >
> > +/* Find a node by the index of the entry of cmap. Index N means the
> > +N/CMAP_K
> > + * bucket and N%CMAP_K entry in that bucket.
> > + * Notice that it is not protected by the optimistic lock
> > +(versioning) because
> > + * it does not compare the hashes. Currently it is only used by the
> > +datapath
> > + * SMC cache.
> > + *
> > + * Return node for the entry of index or NULL if the index beyond
> > +boundary */ const struct cmap_node * cmap_find_by_index(const struct
> > +cmap *cmap, uint32_t index) {
> > +const struct cmap_impl *impl = cmap_get_impl(cmap);
> > +
> > +uint32_t b = index / CMAP_K;
> > +uint32_t e = index % CMAP_K;
> > +
> > +if (b > impl->mask) {
> > +return NULL;
> > +}
> > +
> > +const struct cmap_bucket *bucket = >buckets[b];
> > +
> > +return cmap_node_next(>nodes[e]);
> > +}
> > +
> > +/* Find the index of certain hash value. Currently only used by the
> > +datapath
> > + * SMC cache.
> > + *
> > + * Return the index of the entry if found, or UINT32_MAX if not found.
> > +The
> > + *
Re: [ovs-dev] [PATCH v5 1/2] dpif-netdev: Add SMC cache after EMC cache
Acked-by: Billy O'Mahony
> -Original Message-
> From: Wang, Yipeng1
> Sent: Tuesday, July 10, 2018 11:14 AM
> To: d...@openvswitch.org; jan.scheur...@ericsson.com; O Mahony, Billy
>
> Cc: Wang, Yipeng1 ; Stokes, Ian
> ; b...@ovn.org
> Subject: [PATCH v5 1/2] dpif-netdev: Add SMC cache after EMC cache
>
> This patch adds a signature match cache (SMC) after exact match cache (EMC).
> The difference between SMC and EMC is SMC only stores a signature of a flow
> thus it is much more memory efficient. With same memory space, EMC can
> store 8k flows while SMC can store 1M flows. It is generally beneficial to
> turn on
> SMC but turn off EMC when traffic flow count is much larger than EMC size.
>
> SMC cache will map a signature to an dp_netdev_flow index in flow_table. Thus,
> we add two new APIs in cmap for lookup key by index and lookup index by key.
>
> For now, SMC is an experimental feature that it is turned off by default. One
> can
> turn it on using ovsdb options.
>
> Signed-off-by: Yipeng Wang
> Co-authored-by: Jan Scheurich
> Signed-off-by: Jan Scheurich
> ---
> Documentation/topics/dpdk/bridge.rst | 15 ++
> NEWS | 2 +
> lib/cmap.c | 74
> lib/cmap.h | 11 ++
> lib/dpif-netdev-perf.h | 1 +
> lib/dpif-netdev.c| 329
> +++
> tests/pmd.at | 1 +
> vswitchd/vswitch.xml | 13 ++
> 8 files changed, 409 insertions(+), 37 deletions(-)
>
> diff --git a/Documentation/topics/dpdk/bridge.rst
> b/Documentation/topics/dpdk/bridge.rst
> index 63f8a62..df74c02 100644
> --- a/Documentation/topics/dpdk/bridge.rst
> +++ b/Documentation/topics/dpdk/bridge.rst
> @@ -102,3 +102,18 @@ For certain traffic profiles with many parallel flows,
> it's
> recommended to set ``N`` to '0' to achieve higher forwarding performance.
>
> For more information on the EMC refer to :doc:`/intro/install/dpdk` .
> +
> +
> +SMC cache (experimental)
> +-
> +
> +SMC cache or signature match cache is a new cache level after EMC cache.
> +The difference between SMC and EMC is SMC only stores a signature of a
> +flow thus it is much more memory efficient. With same memory space, EMC
> +can store 8k flows while SMC can store 1M flows. When traffic flow
> +count is much larger than EMC size, it is generally beneficial to turn
> +off EMC and turn on SMC. It is currently turned off by default and an
> experimental feature.
> +
> +To turn on SMC::
> +
> +$ ovs-vsctl --no-wait set Open_vSwitch .
> + other_config:smc-enable=true
> diff --git a/NEWS b/NEWS
> index 92e9b92..f30a1e0 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -44,6 +44,8 @@ Post-v2.9.0
> ovs-appctl dpif-netdev/pmd-perf-show
> * Supervision of PMD performance metrics and logging of suspicious
> iterations
> + * Add signature match cache (SMC) as experimental feature. When turned
> on,
> + it improves throughput when traffic has many more flows than EMC size.
> - ERSPAN:
> * Implemented ERSPAN protocol (draft-foschiano-erspan-00.txt) for
> both kernel datapath and userspace datapath.
> diff --git a/lib/cmap.c b/lib/cmap.c
> index 07719a8..cb9cd32 100644
> --- a/lib/cmap.c
> +++ b/lib/cmap.c
> @@ -373,6 +373,80 @@ cmap_find(const struct cmap *cmap, uint32_t hash)
> hash);
> }
>
> +/* Find a node by the index of the entry of cmap. Index N means the
> +N/CMAP_K
> + * bucket and N%CMAP_K entry in that bucket.
> + * Notice that it is not protected by the optimistic lock (versioning)
> +because
> + * it does not compare the hashes. Currently it is only used by the
> +datapath
> + * SMC cache.
> + *
> + * Return node for the entry of index or NULL if the index beyond
> +boundary */ const struct cmap_node * cmap_find_by_index(const struct
> +cmap *cmap, uint32_t index) {
> +const struct cmap_impl *impl = cmap_get_impl(cmap);
> +
> +uint32_t b = index / CMAP_K;
> +uint32_t e = index % CMAP_K;
> +
> +if (b > impl->mask) {
> +return NULL;
> +}
> +
> +const struct cmap_bucket *bucket = >buckets[b];
> +
> +return cmap_node_next(>nodes[e]);
> +}
> +
> +/* Find the index of certain hash value. Currently only used by the
> +datapath
> + * SMC cache.
> + *
> + * Return the index of the entry if found, or UINT32_MAX if not found.
> +The
> + * function assumes entry index cannot be larger than UINT32_MAX. */
> +uint32_t cmap_find_index(const struct cmap *cmap, uint32_t hash) {
> +const struct cmap_impl *impl = cmap_get_impl(cmap);
> +uint32_t h1 = rehash(impl, hash);
> +uint32_t h2 = other_hash(h1);
> +
> +uint32_t b_index1 = h1 & impl->mask;
> +uint32_t b_index2 = h2 & impl->mask;
> +
> +uint32_t c1, c2;
> +uint32_t index = UINT32_MAX;
> +
> +const struct cmap_bucket *b1 = >buckets[b_index1];
> +const struct
[ovs-dev] [PATCH v5 1/2] dpif-netdev: Add SMC cache after EMC cache
This patch adds a signature match cache (SMC) after exact match
cache (EMC). The difference between SMC and EMC is SMC only stores
a signature of a flow thus it is much more memory efficient. With
same memory space, EMC can store 8k flows while SMC can store 1M
flows. It is generally beneficial to turn on SMC but turn off EMC
when traffic flow count is much larger than EMC size.
SMC cache will map a signature to an dp_netdev_flow index in
flow_table. Thus, we add two new APIs in cmap for lookup key by
index and lookup index by key.
For now, SMC is an experimental feature that it is turned off by
default. One can turn it on using ovsdb options.
Signed-off-by: Yipeng Wang
Co-authored-by: Jan Scheurich
Signed-off-by: Jan Scheurich
---
Documentation/topics/dpdk/bridge.rst | 15 ++
NEWS | 2 +
lib/cmap.c | 74
lib/cmap.h | 11 ++
lib/dpif-netdev-perf.h | 1 +
lib/dpif-netdev.c| 329 +++
tests/pmd.at | 1 +
vswitchd/vswitch.xml | 13 ++
8 files changed, 409 insertions(+), 37 deletions(-)
diff --git a/Documentation/topics/dpdk/bridge.rst
b/Documentation/topics/dpdk/bridge.rst
index 63f8a62..df74c02 100644
--- a/Documentation/topics/dpdk/bridge.rst
+++ b/Documentation/topics/dpdk/bridge.rst
@@ -102,3 +102,18 @@ For certain traffic profiles with many parallel flows,
it's recommended to set
``N`` to '0' to achieve higher forwarding performance.
For more information on the EMC refer to :doc:`/intro/install/dpdk` .
+
+
+SMC cache (experimental)
+-
+
+SMC cache or signature match cache is a new cache level after EMC cache.
+The difference between SMC and EMC is SMC only stores a signature of a flow
+thus it is much more memory efficient. With same memory space, EMC can store 8k
+flows while SMC can store 1M flows. When traffic flow count is much larger than
+EMC size, it is generally beneficial to turn off EMC and turn on SMC. It is
+currently turned off by default and an experimental feature.
+
+To turn on SMC::
+
+$ ovs-vsctl --no-wait set Open_vSwitch . other_config:smc-enable=true
diff --git a/NEWS b/NEWS
index 92e9b92..f30a1e0 100644
--- a/NEWS
+++ b/NEWS
@@ -44,6 +44,8 @@ Post-v2.9.0
ovs-appctl dpif-netdev/pmd-perf-show
* Supervision of PMD performance metrics and logging of suspicious
iterations
+ * Add signature match cache (SMC) as experimental feature. When turned on,
+ it improves throughput when traffic has many more flows than EMC size.
- ERSPAN:
* Implemented ERSPAN protocol (draft-foschiano-erspan-00.txt) for
both kernel datapath and userspace datapath.
diff --git a/lib/cmap.c b/lib/cmap.c
index 07719a8..cb9cd32 100644
--- a/lib/cmap.c
+++ b/lib/cmap.c
@@ -373,6 +373,80 @@ cmap_find(const struct cmap *cmap, uint32_t hash)
hash);
}
+/* Find a node by the index of the entry of cmap. Index N means the N/CMAP_K
+ * bucket and N%CMAP_K entry in that bucket.
+ * Notice that it is not protected by the optimistic lock (versioning) because
+ * it does not compare the hashes. Currently it is only used by the datapath
+ * SMC cache.
+ *
+ * Return node for the entry of index or NULL if the index beyond boundary */
+const struct cmap_node *
+cmap_find_by_index(const struct cmap *cmap, uint32_t index)
+{
+const struct cmap_impl *impl = cmap_get_impl(cmap);
+
+uint32_t b = index / CMAP_K;
+uint32_t e = index % CMAP_K;
+
+if (b > impl->mask) {
+return NULL;
+}
+
+const struct cmap_bucket *bucket = >buckets[b];
+
+return cmap_node_next(>nodes[e]);
+}
+
+/* Find the index of certain hash value. Currently only used by the datapath
+ * SMC cache.
+ *
+ * Return the index of the entry if found, or UINT32_MAX if not found. The
+ * function assumes entry index cannot be larger than UINT32_MAX. */
+uint32_t
+cmap_find_index(const struct cmap *cmap, uint32_t hash)
+{
+const struct cmap_impl *impl = cmap_get_impl(cmap);
+uint32_t h1 = rehash(impl, hash);
+uint32_t h2 = other_hash(h1);
+
+uint32_t b_index1 = h1 & impl->mask;
+uint32_t b_index2 = h2 & impl->mask;
+
+uint32_t c1, c2;
+uint32_t index = UINT32_MAX;
+
+const struct cmap_bucket *b1 = >buckets[b_index1];
+const struct cmap_bucket *b2 = >buckets[b_index2];
+
+do {
+do {
+c1 = read_even_counter(b1);
+for (int i = 0; i < CMAP_K; i++) {
+if (b1->hashes[i] == hash) {
+index = b_index1 * CMAP_K + i;
+ }
+}
+} while (OVS_UNLIKELY(counter_changed(b1, c1)));
+if (index != UINT32_MAX) {
+break;
+}
+do {
+c2 = read_even_counter(b2);
+for (int i = 0; i < CMAP_K; i++) {
+if (b2->hashes[i] == hash) {
+
