Re: [ovs-dev] [PATCH v7 0/9] DPIF + MFEX Inner AVX512
> -Original Message- > From: Eelco Chaudron > Sent: Monday 21 November 2022 14:34 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; kumar.am...@intel.com > Subject: Re: [ovs-dev] [PATCH v7 0/9] DPIF + MFEX Inner AVX512 > > On 12 Oct 2022, at 13:55, Cian Ferriter wrote: > > > This Series of Patchsets introduce the Optimizations for supporting > > tunneled packets in DPIF and MFEX. Along with the optimization various > > refactoring of scalar path is done to be used accross without > > duplication. > > > > Over the Tests we have observed a gain of approximate 20~25% gain in > > performance over the scalar path. > > I'm sorry, it took a while, but see my comments in the individual patches. > These are all based on reading patches. I've not been able to do any actual > testing as I'm waiting to get my hands on an AVX machine. > > Cheers, > > Eelco Hi Eelco, Thanks for your review of this patch set. Most comments are minor, however the more flexible/capable CLI, with more complex configurations of dpif-implementation functions is a larger rework. Given the scope we won't have time to rework for OVS 3.1. For the other patch sets (NVGRE and IPv6, smaller in scope/size) comments/reworks will be done for OVS 3.1. Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 0/9] DPIF + MFEX Inner AVX512
Hi Eelco, Thanks for reaching out, here's our priority list: 1. Actions IPv6. Currently at v3. http://patchwork.ozlabs.org/project/openvswitch/patch/20220926132946.1767182-1-emma.f...@intel.com/ 2. DPIF Recirc AVX512. Currently at v7. http://patchwork.ozlabs.org/project/openvswitch/list/?series=322458 3. NVGRE optimizations. Currently at v2 (I mistakenly omitted the v2 tag when sending the last version, oops). http://patchwork.ozlabs.org/project/openvswitch/list/?series=318621 Thanks, Cian > -Original Message- > From: Eelco Chaudron > Sent: Wednesday 12 October 2022 14:30 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; Amber, Kumar ; Finn, Emma > ; > Stokes, Ian ; Van Haaren, Harry > > Subject: Re: [ovs-dev] [PATCH v7 0/9] DPIF + MFEX Inner AVX512 > > Hi Cian, et al. > > I’m still planning to review all the AVX512-related patches that intel people > have out, but I’m a bit > short on time. > > My next step is to review this set, or are there any other sets that you want > to be reviewed first? > Even better would be a priority list that I can work down. > > Cheers, > > Eelco > > > On 12 Oct 2022, at 13:55, Cian Ferriter wrote: > > > This Series of Patchsets introduce the Optimizations for supporting > > tunneled packets in DPIF and MFEX. Along with the optimization various > > refactoring of scalar path is done to be used accross without > > duplication. > > > > Over the Tests we have observed a gain of approximate 20~25% gain in > > performance over the scalar path. > > > > --- > > v7: > > - Reword bridge.rst documentation to better explain the use of the > > -recirc option and provide examples. This is all based on feedback > > from Sunil. > > > > v6: > > - Fix minor comments. > > - Reworked magic block array and build the offsets from header sizes. > > > > v5: > > - Added comments to decribe method for handling MFEX inner. > > - Fixed garbage passing of incorrect tunnel values. > > > > Kumar Amber (9): > > dpif-netdev: Refactor per thread recirc data allocation. > > dpif-netdev: Refactor hash function to own header. > > dpif-netdev-avx512: Refactor avx512 dpif and create new APIs. > > dpif-netdev-avx512: Add inner packet handling to dpif. > > dpif-netdev: Add function pointer for dpif re-circulate. > > dpif-mfex: Modify set/get MFEX commands to include inner. > > dpif-mfex: Change MFEX fn pointer prototype to include md_is_valid. > > mfex-study: Modify study func to select outer and inner MFEX funcs. > > mfex-avx512: Add support for tunnel packets in avx512 MFEX. > > > > Documentation/topics/dpdk/bridge.rst | 32 +++- > > lib/dpif-netdev-avx512.c | 72 ++--- > > lib/dpif-netdev-extract-avx512.c | 213 +-- > > lib/dpif-netdev-extract-study.c | 133 +++-- > > lib/dpif-netdev-private-dpcls.h | 23 +++ > > lib/dpif-netdev-private-dpif.c | 81 +++--- > > lib/dpif-netdev-private-dpif.h | 37 - > > lib/dpif-netdev-private-extract.c| 33 - > > lib/dpif-netdev-private-extract.h| 19 ++- > > lib/dpif-netdev-private-thread.h | 6 + > > lib/dpif-netdev.c| 67 - > > 11 files changed, 534 insertions(+), 182 deletions(-) > > > > -- > > 2.25.1 > > > > ___ > > dev mailing list > > d...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 0/9] DPIF + MFEX Inner AVX512
> -Original Message- > From: Amber, Kumar > Sent: Thursday 6 October 2022 11:24 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v6 0/9] DPIF + MFEX Inner AVX512 > > This Series of Patchsets introduce the Optimizations for > supporting tunneled packets in DPIF and MFEX. Along with > the optimization various refactoring of scalar > path is done to be used accross without duplication. > > Over the Tests we have observed a gain of approximate 20~25% > gain in performance over the scalar path. > I have sent a v7 of this patchset, so this version is now outdated. Just letting folks know here since I can't mark the v6 as Superseded in Patchwork. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 6/9] dpif-mfex: Modify set/get MFEX commands to include inner.
Hi Sunil, I'll respin the patch set based on these documentation comments with your Ack. Thanks, Cian > -Original Message- > From: dev On Behalf Of Pai G, Sunil > Sent: Tuesday 11 October 2022 11:52 > To: Amber, Kumar ; ovs-dev@openvswitch.org > Cc: f...@sysclose.org; Amber, Kumar ; > i.maxim...@ovn.org > Subject: Re: [ovs-dev] [PATCH v6 6/9] dpif-mfex: Modify set/get MFEX commands > to include inner. > > Hi Amber, > > One comment below , rest looks good to me. > > > -Original Message- > > From: dev On Behalf Of Kumar Amber > > Sent: Thursday, October 6, 2022 3:54 PM > > To: ovs-dev@openvswitch.org > > Cc: i.maxim...@ovn.org; f...@sysclose.org; Amber, Kumar > > > > Subject: [ovs-dev] [PATCH v6 6/9] dpif-mfex: Modify set/get MFEX commands > > to include inner. > > > > The set command is modified to allow the user to select different > > implementations for processing inner packets. > > Also, the get command is modified to indicate both inner and outer MFEX > > implementation in use. > > > > $ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 1024 -recirc > > > > Signed-off-by: Kumar Amber > > Signed-off-by: Cian Ferriter > > Co-authored-by: Cian Ferriter > > --- > > Documentation/topics/dpdk/bridge.rst | 24 ++-- > > lib/dpif-netdev-private-extract.c| 23 ++- > > lib/dpif-netdev-private-extract.h| 6 +- > > lib/dpif-netdev-private-thread.h | 3 +++ > > lib/dpif-netdev.c| 23 +++ > > 5 files changed, 67 insertions(+), 12 deletions(-) > > > > diff --git a/Documentation/topics/dpdk/bridge.rst > > b/Documentation/topics/dpdk/bridge.rst > > index 354f1ced1..dbebea624 100644 > > --- a/Documentation/topics/dpdk/bridge.rst > > +++ b/Documentation/topics/dpdk/bridge.rst > > @@ -293,13 +293,21 @@ command also shows whether the CPU supports each > > implementation:: > > An implementation can be selected manually by the following command:: > > > > $ ovs-appctl dpif-netdev/miniflow-parser-set [-pmd core_id] name \ > > - [study_cnt] > > + [study_cnt] [-recirc] > > > > -The above command has two optional parameters: ``study_cnt`` and > > ``core_id``. > > -The ``core_id`` sets a particular packet parsing function to a specific - > > PMD thread on the core. The third parameter ``study_cnt``, which is > > specific -to ``study`` and ignored by other implementations, means how > > many packets -are needed to choose the best implementation. > > +The above command has three optional parameters: ``study_cnt``, > > +``core_id`` and ``-recirc``. The ``core_id`` sets a particular packet > > +parsing function to a specific PMD thread on the core. The third > > +parameter ``study_cnt``, which is specific to ``study`` and ignored by > > +other implementations, means how many packets are needed to choose the > > +best implementation. > > > > The fourth parameter ``-recirc`` indicates to MFEX > > +to use optimized MFEX inner for processing tunneled inner packets. The > > +optional ``-recirc`` parameter gives flexibility to set different > > +optimized MFEX function on inner and outer, when set to study.t > > I think we can improve this documentation a bit, how about the following: > The optional ``-recirc`` parameter allows OVS to set the optimized MFEX > function for inner packet > processing. > > For example: > - ``name`` : sets the outer implementation to ``name``, inner > defaults to scalar. > - ``name`` + ``recirc`` : sets both inner and outer implementations to > ``name``. > - ``study`` + ``recirc`` : sets inner as well as outer implementations > separately based on the traffic > pattern. > > Makes sense, this explains it better. I'll make the change and build the docs to see how it looks, then respin a new version with your Ack. > > + > > +$ ovs-appctl dpif-netdev/miniflow-parser-set study -recirc > > > > > > Also user can select the ``study`` implementation which studies the > > traffic for a specific number of packets by applying all available > > implementations of @@ -322,6 +330,10 @@ following command:: > > > > $ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 scalar > > > > +``study`` can be selected with packet count and explicit PMD selection > > +along with the ``recirc`` by following command:: > > + > > +$ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 study 1024 > > + -recirc > > > > Other than that, the changes look OK to me, > > Acked-by: Sunil Pai G > > Thanks and regards > Sunil > > > ___ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 9/9] mfex-avx512: Add support for tunnel packets in avx512 MFEX.
> -Original Message- > From: Amber, Kumar > Sent: Thursday 6 October 2022 11:24 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v6 9/9] mfex-avx512: Add support for tunnel packets in avx512 > MFEX. > > This patch adds the necessary support to avx512 mfex to > support handling of tunnel packet type. > > Signed-off-by: Kumar Amber > > --- > v6: > - Fix minor comments from Cian. All the minor comments have been fixed. > - Deduce magic bits through protocol sizes. I like the approach you are taking for the block offsets or protocol sizes in this version a lot more. I think it's clearer how the different offsets are built up from the individual header sizes. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 8/9] mfex-study: Modify study func to select outer and inner MFEX funcs.
> -Original Message- > From: Amber, Kumar > Sent: Thursday 6 October 2022 11:24 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v6 8/9] mfex-study: Modify study func to select outer and > inner MFEX funcs. > > The MFEX study function is split into outer and inner to allow > for independent selection and studying of packets in outer and inner > flows to different ISA optimized miniflow extract implementations. > > Signed-off-by: Kumar Amber > > --- > v6: > - Fix minor comments from Cian. > --- Minor comments have been addressed, thanks for that Amber. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 7/9] dpif-mfex: Change MFEX fn pointer prototype to include md_is_valid.
> -Original Message- > From: Amber, Kumar > Sent: Thursday 6 October 2022 11:24 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v6 7/9] dpif-mfex: Change MFEX fn pointer prototype to > include md_is_valid. > > The md_is_valid parameter is passed from DPIF to MFEX to allow MFEX > functions to detect the tunneling and decide the processing of Inner > packets in static predictable branches. > > Signed-off-by: Kumar Amber > --- > lib/dpif-netdev-avx512.c | 3 ++- > lib/dpif-netdev-extract-avx512.c | 9 + > lib/dpif-netdev-extract-study.c | 6 -- > lib/dpif-netdev-private-extract.c | 6 -- > lib/dpif-netdev-private-extract.h | 13 - > 5 files changed, 23 insertions(+), 14 deletions(-) > Since we are no longer using the "flow_tnl_dst_is_set()" function, I'm happy to ack this change. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5 9/9] mfex-avx512: Add support for tunnel packets in avx512 mfex.
Hi Amber,
Thanks for the patches. I've left some comments below inline.
Thanks,
Cian
> -Original Message-
> From: Amber, Kumar
> Sent: Friday 26 August 2022 00:31
> To: ovs-dev@openvswitch.org
> Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; f...@sysclose.org; Van Haaren, Harry
> ; Amber, Kumar
>
> Subject: [PATCH v5 9/9] mfex-avx512: Add support for tunnel packets in avx512
> mfex.
>
> This patch adds the necessary support to avx512 mfex to
> support handling of tunnel packet type.
>
> Signed-off-by: Kumar Amber
>
> ---
> v5:
> - check metadata IP address to find tunneling is valid or not.
> As dummy-pmd often passes garbage data to dpif.
> ---
> ---
> lib/dpif-netdev-avx512.c | 16 +--
> lib/dpif-netdev-extract-avx512.c | 195 --
> lib/dpif-netdev-private-extract.c | 4 +-
> 3 files changed, 170 insertions(+), 45 deletions(-)
>
> diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c
> index 1c3b67b02..d5c61baff 100644
> --- a/lib/dpif-netdev-avx512.c
> +++ b/lib/dpif-netdev-avx512.c
> @@ -185,15 +185,17 @@ dp_netdev_input_avx512__(struct dp_netdev_pmd_thread
> *pmd,
> }
>
> /* Do a batch minfilow extract into keys. */
> - /* Do a batch minfilow extract into keys, but only for outer packets. */
In the earlier DPIF part of this patchset, I guess you add the above comment
line that you are removing here. But when you add it, I don't think it should
be duplicating the line above. Just add the ", but only for outer packets."
part in the earlier patchset and remove it here, rather than adding a whole
line then removing in a later patch.
> uint32_t mf_mask = 0;
> -if (recirc_depth == 0) {
> -miniflow_extract_func mfex_func;
> -atomic_read_relaxed(>miniflow_extract_opt, _func);
> -if (mfex_func) {
> -mf_mask = mfex_func(packets, keys, batch_size, in_port, pmd,
> +miniflow_extract_func mfex_func;
> +atomic_read_relaxed(>miniflow_extract_opt, _func);
> +miniflow_extract_func mfex_inner_func;
> +atomic_read_relaxed(>miniflow_extract_inner_opt, _inner_func);
> +if (md_is_valid && mfex_inner_func) {
> +mf_mask = mfex_inner_func(packets, keys, batch_size, in_port, pmd,
> + md_is_valid);
> +} else if (!md_is_valid && mfex_func) {
> +mf_mask = mfex_func(packets, keys, batch_size, in_port, pmd,
> md_is_valid);
Align the above line with the 'p' in 'packets' from 2 lines above.
> -}
> }
>
> uint32_t iter = lookup_pkts_bitmask;
> diff --git a/lib/dpif-netdev-extract-avx512.c
> b/lib/dpif-netdev-extract-avx512.c
> index 833e9bd31..4c62bd911 100644
> --- a/lib/dpif-netdev-extract-avx512.c
> +++ b/lib/dpif-netdev-extract-avx512.c
> @@ -360,6 +360,53 @@ _mm512_maskz_permutexvar_epi8_selector(__mmask64 k_shuf,
> __m512i v_shuf,
> MF_WORD(ipv6_dst, 2) | MF_BIT(tp_src) |
> MF_BIT(tp_dst))
> #define MF_IPV6_TCP (MF_IPV6_UDP | MF_BIT(tcp_flags) |
> MF_BIT(arp_tha.ea[2]))
>
> +#define MF_TUNNEL MF_WORD(tunnel, offsetof(struct flow_tnl, metadata) /
> 8)
> +
> +#define MF_ETH_TUNNEL (MF_TUNNEL | MF_ETH)
> +#define MF_ETH_VLAN_TUNNEL (MF_TUNNEL | MF_ETH_VLAN)
> +
> +/* Block offsets represents the offsets into the blocks array of miniflow
> + * and are derived experimentally. Scalar miniflow parses the header
> + * in a fixed order and sequentially in a dynamic fashion thus incrementing
> + * pointer and copying data is enough but in AVX512 since the headers are
> + * parsed using pre-defined masks we need these magic offsets to write
> + * some of the data items at the correct loaction in the blocks array
> + * using below magic numbers.
> + */
> +#define BLK_META_DATA_OFFS9
We could use something like the below instead of hardcoding 9, right?
offsetof(struct flow_tnl, metadata) / sizeof(uint64_t)
That is the number of words that the scalar miniflow_extract() passes to
miniflow_push_words() for the tunnel metadata.
> +#define BLK_IPv4_TCP_FLAG 6
> +#define BLK_VLAN_IPv4_TCP_FLAG7
> +#define BLK_VLAN_PCP 4
> +#define BLK_IPv6_HDR_OFFS 8
> +#define BLK_VLAN_IPv6_HDR_OFFS9
> +#define BLK_IPv6_TCP_FLAG 9
> +#define BLK_VLAN_IPv6_TCP_FLAG10
> +#define BLK_L4_UDP_OFFS 9
> +#define BLK_L4_TCP_OFFS 10
> +#define BLK_VLAN_L4_UDP_OFFS 10
> +#define BLK_VLAN_L4_TCP_OFFS 11
I spent some time thinking about these #defines and whether we can generate
them i
Re: [ovs-dev] [PATCH v5 8/9] mfex-study: Modify study func to select outer and inner mfex funcs.
Hi Amber,
My comments on the patch are below inline.
Thanks,
Cian
> -Original Message-
> From: Amber, Kumar
> Sent: Friday 26 August 2022 00:31
> To: ovs-dev@openvswitch.org
> Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; f...@sysclose.org; Van Haaren, Harry
> ; Amber, Kumar
>
> Subject: [PATCH v5 8/9] mfex-study: Modify study func to select outer and
> inner mfex funcs.
>
> The Mfex study function is split into outer and inner to allow
> for independent selection and studying of packets in outer and inner
> flows to different ISA optimized Mfexs.
For the last line how about:
> flows to different ISA optimized miniflow extract implementations.
>
> Signed-off-by: Kumar Amber
> ---
> lib/dpif-netdev-extract-study.c | 126 +---
> 1 file changed, 83 insertions(+), 43 deletions(-)
>
> diff --git a/lib/dpif-netdev-extract-study.c b/lib/dpif-netdev-extract-study.c
> index 71354cc4c..03d97c64e 100644
> --- a/lib/dpif-netdev-extract-study.c
> +++ b/lib/dpif-netdev-extract-study.c
> @@ -30,7 +30,9 @@ static atomic_uint32_t mfex_study_pkts_count =
> MFEX_MAX_PKT_COUNT;
> /* Struct to hold miniflow study stats. */
> struct study_stats {
> uint32_t pkt_count;
> +uint32_t pkt_inner_count;
> uint32_t impl_hitcount[MFEX_IMPL_MAX];
> +uint32_t impl_inner_hitcount[MFEX_IMPL_MAX];
> };
>
> /* Define per thread data to hold the study stats. */
> @@ -67,6 +69,58 @@ mfex_set_study_pkt_cnt(uint32_t pkt_cmp_count, const char
> *name)
> return -EINVAL;
> }
>
> +
Remove the extra newline here.
> +static inline void
> +mfex_reset_stats(uint32_t *impls_hitcount, uint32_t *pkt_cnt) {
> +/* Reset stats so that study function can be called again
> + * for next traffic type and optimal function ptr can be
> + * chosen.
> + */
I prefer having this comment above the function rather than inside since it
describes what the whole function does.
I've reworded the comment to add 'the's and 'an'. How about this:
Reset stats so that the study function can be called again for the next traffic
type and an optimal function pointer can be chosen.
> +memset(impls_hitcount, 0, sizeof(uint32_t) * MFEX_IMPL_MAX);
> +*pkt_cnt = 0;
> +}
> +
> +static inline void
> +mfex_study_select_best_impls(struct dpif_miniflow_extract_impl *mfex_funcs,
> + uint32_t pkt_cnt, uint32_t *impls_arr,
> + atomic_uintptr_t *pmd_func, char *name)
> +{
> +
> +uint32_t best_func_index = MFEX_IMPL_START_IDX;
> +uint32_t max_hits = 0;
> +
> +for (int i = MFEX_IMPL_START_IDX; i < MFEX_IMPL_MAX; i++) {
> +if (impls_arr[i] > max_hits) {
> +max_hits = impls_arr[i];
> +best_func_index = i;
> +}
> +}
These 2 braces are indented incorrectly. It should be like this:
> +}
> +}
> +
> +/* If 50% of the packets hit, enable the function. */
How about this reword:
If at least 50% of the packets hit the implementation, enable that
implementation.
> +if (max_hits >= (mfex_study_pkts_count / 2)) {
> +atomic_store_relaxed(pmd_func,
> +(uintptr_t) mfex_funcs[best_func_index].extract_func);
> +VLOG_INFO("MFEX %s study chose impl %s: (hits %u/%u pkts)",
> + name, mfex_funcs[best_func_index].name, max_hits,
> + pkt_cnt);
The 'pkt_cnt' doesn't have to be wrapped to the next line.
> +} else {
> +/* Set the implementation to null for default miniflow. */
> +atomic_store_relaxed(pmd_func,
> +(uintptr_t) mfex_funcs[MFEX_IMPL_SCALAR].extract_func);
> +VLOG_INFO("Not enough packets matched (%u/%u), disabling"
> + " optimized MFEX.", max_hits, pkt_cnt);
> +}
> +
> +/* In debug mode show stats for all the counters. */
> +if (VLOG_IS_DBG_ENABLED()) {
> +for (int i = MFEX_IMPL_START_IDX; i < MFEX_IMPL_MAX; i++) {
> +VLOG_DBG("MFEX study results for implementation %s:"
> + " (hits %u/%u pkts)", mfex_funcs[i].name,
> + impls_arr[i], pkt_cnt);
> +}
> +}
> +}
> +
> uint32_t
> mfex_study_traffic(struct dp_packet_batch *packets,
> struct netdev_flow_key *keys,
> @@ -76,10 +130,12 @@ mfex_study_traffic(struct dp_packet_batch *packets,
> {
> uint32_t hitmask = 0;
> uint32_t mask = 0;
> +uint32_t study_cnt_pkts;
> struct dp_netdev_pmd_thread *pmd = pmd_handle;
> struct dpif_m
Re: [ovs-dev] [PATCH v5 7/9] dpif-mfex: Change mfex fn pointer prototype to include md_is_valid.
> -Original Message- > From: Amber, Kumar > Sent: Friday 26 August 2022 00:31 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v5 7/9] dpif-mfex: Change mfex fn pointer prototype to > include md_is_valid. > > The md_is_valid parameter is passed from DPIF to MFEX to allow mfex > functions to detect the tunneling and decide the processing of Inner > packets in static predictable branches. > > Signed-off-by: Kumar Amber > --- > lib/dpif-netdev-avx512.c | 3 ++- > lib/dpif-netdev-extract-avx512.c | 9 + > lib/dpif-netdev-extract-study.c | 6 -- > lib/dpif-netdev-private-extract.c | 6 -- > lib/dpif-netdev-private-extract.h | 13 - > 5 files changed, 23 insertions(+), 14 deletions(-) > Hey Amber, I'm OK with the API changes in this patch if we need them, but after looking at the later 2 patches in the series, I can see that you're using another check in the heart of the MFEX AVX512 code (mfex_avx512_process()): /* Dummy pmd dont always pass correct md_is_valid and hence * need to check the tunnel data to ensure correct behaviour. */ bool tunnel = flow_tnl_dst_is_set(>tunnel); If we need to use the flow_tnl_dst_is_set() check, then it makes me think we don't need the md_is_valid bool at all. Can you check if we can get away with just using the flow_tnl_dst_is_set() check instead? If so, we don't need this patch at all as we could use the flow_tnl_dst_is_set() check for both the study and AVX512 MFEX implementations. Hopefully that makes sense. Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 1/2] dpif-netdev/dpcls: Specialize more subtable signatures.
> -Original Message- > From: Ilya Maximets > Sent: Friday 26 August 2022 19:43 > To: Ferriter, Cian ; ovs-dev@openvswitch.org > Cc: i.maxim...@ovn.org > Subject: Re: [ovs-dev] [PATCH 1/2] dpif-netdev/dpcls: Specialize more > subtable signatures. > > On 8/25/22 16:45, Cian Ferriter wrote: > > The subtable signatures being specialized here were found in an NVGRE > > tunnel scenario. > > > > Signed-off-by: Cian Ferriter > > Hi, Cian. > > Not a review, just a comment. > > Please, give this patch a more descriptive subject. There is already > a patch named exactly like this and that confuses scripts for automated > patchwork state updates. Hey Ilya, Apologies, I've updated the patch subject now. Was on PTO after I sent the patch. Thanks for pointing this out. Thanks, Cian > > Best regards, Ilya Maximets. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] acinclude: Improve vpopcntdq build check.
> -Original Message-
> From: Stokes, Ian
> Sent: Thursday 11 August 2022 15:08
> To: Pai G, Sunil ; Ferriter, Cian
> ; ovs-
> d...@openvswitch.org
> Cc: Ilya Maximets
> Subject: RE: [ovs-dev] [PATCH] acinclude: Improve vpopcntdq build check.
>
> > > -Original Message-
> > > From: dev On Behalf Of Cian
> > Ferriter
> > > Sent: Thursday, August 11, 2022 4:07 PM
> > > To: ovs-dev@openvswitch.org
> > > Subject: [ovs-dev] [PATCH] acinclude: Improve vpopcntdq build check.
> > >
> > > Support for vpopcntdq instruction generation by the compiler was
> > already
> > > checked in the OVS_CHECK_AVX512 AC function by checking if the
> > compiler
> > > accepted the -mavx512vpopcntdq option. However, there can be
> > situations
> > > where the compiler supports vpopcntdq generation but the assembler
> > doesn't
> > > support the instruction.
> >
> > Quite unfortunate
> >
> > >
> > > The below OVS_CHECK_AVX512VPOPCNTDQ AC function will check for
> > both
> > > compiler and assembler support for the vpopcntdq instruction.
> > >
> > > Signed-off-by: Cian Ferriter
> > > ---
> > > acinclude.m4 | 2 +-
> > > m4/openvswitch.m4 | 29 +
> > > 2 files changed, 30 insertions(+), 1 deletion(-)
> > >
> >
> > The patch looks good Cian,
>
> So tested on a system that hits this problem and this patch resolves the
> issue.
>
Thanks for testing this Ian. Also, I forgot to add your reported by tag, feel
free to add it since you reported the issue to me.
Reported-by: Ian Stokes
> One thing I noticed is that this issue only becomes apparent on the system
> with the following commit
>
> cb1c64007734 ("acinclude: Add seperate checks for AVX512 ISA.")
>
> So I'm thinking we should have a fixes tag for the patch also? Also it should
> be applied to master and
Yes, agreed, let's add the Fixes tag pointing to the commit you mentioned above.
Fixes: cb1c64007734 ("acinclude: Add seperate checks for AVX512 ISA.")
> 3.0, but should it be backported to 2.17? The issue wasn’t seen on that
> branch but wondering if it
> could still occur there?
I think this patch follows on from other improvements to the OVS AVX512 build
which have been added over the last few releases. Since those other
improvements weren't backported, I don't think it makes sense to backport just
this one.
>
> Regards
> Ian
>
> >
> > Acked-by: Sunil Pai G
I'm hoping the Acked-by, Reported-by and Fixes tags can be added on commit, if
not I'm happy to make another version.
Thanks,
Cian
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2] dpif-netdev: Simplify AVX512 build time checks to enhance readability.
> -Original Message- > From: Pai G, Sunil > Sent: Monday 4 July 2022 13:27 > To: d...@openvswitch.org > Cc: echau...@redhat.com; Finn, Emma ; Van Haaren, Harry > ; Ferriter, Cian > Subject: [PATCH v2] dpif-netdev: Simplify AVX512 build time checks to enhance > readability. > > The preprocessor comparison string to check AVX512 capabilities are > lengthy and effecting user readability. Simpify this by aliasing the checks. > > Suggested-by: Eelco Chaudron > Signed-off-by: Sunil Pai G > > --- > v2: rebase on master, added alias for DPCLS. > Remove the acks since there are changes introduced because of > rebase. > --- V2 changes make sense. There's are extra places where the build check is simplified because of "dpif-netdev: Refactor AVX512 runtime checks" change. It makes sense to add an alias for the DPCLS build time checks because it's used in more than one place now, where that wasn't the case for the v1. I ran the same set of tests that I usually run on these build time checks (build with GCC 4.8, 4.9, 5 and 9) to hit cases where the compiler supports the different subsets of AVX512 ISA. All LGTM. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 2/2] dpif-avx512: Add support for simple match lookup.
> -Original Message-
> From: Pai G, Sunil
> Sent: Thursday 7 July 2022 10:23
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Subject: RE: [ovs-dev] [PATCH 2/2] dpif-avx512: Add support for simple match
> lookup.
>
> Hi Cian,
>
> > +if (!f) {
> > +/* Any miss in Simple Match means an upcall is needed.
> > Fall
> > + * back to the scalar DPIF to do this. */
> > +return -1;
> > +}
> > +
> > +rules[i] = >cr;
> > +n_simple_hit++;
> > +hwol_emc_smc_hitmask |= (1 << i);
>
>
> Nit: this could be improved a bit:
>
> hwol_emc_smc_hitmask |= (UINT32_C(1) << i);
Good spot, I'll fix this in the next version, thanks Sunil.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v11 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
> -Original Message- > From: Amber, Kumar > Sent: Friday 1 July 2022 13:29 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v11 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles > > Add AVX512 Ipv6 optimized profile for vlan/IPv6/UDP and > vlan/IPv6/TCP, IPv6/UDP and IPv6/TCP. > > MFEX autovalidaton test-case already has the IPv6 support for > validating against the scalar mfex. > > Signed-off-by: Kumar Amber > Signed-off-by: Harry van Haaren > Co-authored-by: Harry van Haaren > > --- > v11: > - Rebase on top of David AVX512 ISA runtime check. > v10: > - Rebase on top of Partial Avx512 changes patch. > v9: > - Fix Ubscan memory alinged access. > v8: > - Rename defines for packet offsets. > v7: > - Fix Lenght checks for plen. > v5: > - Add variable length checks for IPv6 and TCP. > v4: > - Rebase to master. > v2: > - Fix CI build error. > - Fix check-patch sign-offs. > --- My ack from the v10 still applies the rebase. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] dpif-netdev: Simplify AVX512 build time checks to enhance readability.
> -Original Message- > From: Pai G, Sunil > Sent: Thursday 23 June 2022 08:01 > To: d...@openvswitch.org > Cc: echau...@redhat.com; Finn, Emma ; Van Haaren, Harry > ; Ferriter, Cian > Subject: [PATCH] dpif-netdev: Simplify AVX512 build time checks to enhance > readability. > > The preprocessor comparison string to check AVX512 capabilities are > lengthy and effecting user readability. Simpify this by aliasing the checks. > > Suggested-by: Eelco Chaudron > Signed-off-by: Sunil Pai G > --- > lib/dpif-netdev-private-dpif.c| 6 -- > lib/dpif-netdev-private-extract.c | 3 +-- > lib/dpif-netdev-private-extract.h | 10 +- > 3 files changed, 10 insertions(+), 9 deletions(-) > I looked to see if we could apply the simplification done below for DPIF and MFEX to the DPCLS code, but I can see that the AVX512 build checks are only performed in one place for the DPCLS, so it doesn't make sense to do it for DPCLS. I guess if there are more places to perform the DPCLS AVX512 build checks in the future, then we can add a similar alias. For now what you've done makes sense. All the changes LGTM. I also tested on the usual GCC 4.8, 4.9, 5 and 9 to test with all the different AVX512 ISA generating capabilities present and not. This all still works as expected. Thanks for making the code better here Sunil. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5 5/5] acinclude: Add seperate checks for AVX512 ISA.
> -Original Message- > From: Ilya Maximets > Sent: Tuesday 31 May 2022 12:00 > To: Ferriter, Cian ; Pai G, Sunil > ; ovs- > d...@openvswitch.org > Cc: i.maxim...@ovn.org > Subject: Re: [ovs-dev] [PATCH v5 5/5] acinclude: Add seperate checks for > AVX512 ISA. > > On 5/18/22 10:51, Ferriter, Cian wrote: > > > > > >> -Original Message- > >> From: Pai G, Sunil > >> Sent: Tuesday 17 May 2022 15:52 > >> To: Ferriter, Cian ; ovs-dev@openvswitch.org > >> Cc: Van Haaren, Harry > >> Subject: RE: [PATCH v5 5/5] acinclude: Add seperate checks for AVX512 ISA. > >> > >>> Checking for each of the required AVX512 ISA separately will allow the > >>> compiler to generate some AVX512 code where there is some support in the > >>> compiler rather than only generating all AVX512 code when all of it is > >>> supported or no AVX512 code at all. > >>> > >>> For example, in GCC 4.9 where there is just support for AVX512F, this > >>> patch will allow building the AVX512 DPIF. > >>> > >>> Another example, in GCC 5 and 6, most AVX512 code can be generated, just > >>> without AVX512VPOPCNTDQ support. > >>> > >>> Signed-off-by: Cian Ferriter > >>> > >>> --- > >>> v5: > >>> * Create a selector function for the permutexvar implementations based > >>> on Sunil's feedback on the v4. This hides the complexity of compile > >>> time and run time selection of permutexvar implementations. > >>> * Add a comment explaining why VPOPCNTDQ_TARGET is defined and used. > >>> > >>> v4: > >>> * Combine the 3 commits which added checks for AVX512 ISA into this > >>> single commit since the first 2 commits were only useful and active > >>> when the 3rd commit was applied. This also takes care of Sunil's > >>> comment about explaining that the first 2 commits are precursors. > >>> * Don't check for AVX512DQ availability in the compiler. This ISA isn't > >>> used in OVS. > >>> * Put all AVX512 ISA checks in the OVS_CHECK_AVX512 macro as per Sunil's > >>> feedback. > >>> * Define a function in acinclude.m4, (OVS_CONDITIONAL_CC_OPTION_DEFINE), > >>> to help with checking for AVX512 ISA support in the compiler. > >>> * Remove the '__AVX512VPOPCNTDQ__' check. Use the HAVE_AVX512* pattern > >>> consistently with all AVX512 ISA checks instead. Fixup the comment > >>> explaining the _mm512_popcnt_epi64_wrapper() function to reflect this. > >> > >> > >> Based on the comments and responses to v4 and changes implemented in v5 , > >> > >> Acked-by: Sunil Pai G > > > > Thanks for the reviews on this and other patches in the series Sunil! > > I didn't test with many different compilers, but the patch set looks > OK to me. Applied. Thanks! > Hey Ilya, Thanks for applying the patch set! Sorry for the delay in responding to the below points. > One small thing I noticed though is that this code is using the > __builtin_constant_p() which requires GCC > 4 and also not available > in MSVC. GCC > 4 is probably not a problem, since older versions > will not support avx512 instructions anyway. Support for MSVC would I sent up a standalone patch to fix this, since it should be easy to review: http://patchwork.ozlabs.org/project/openvswitch/patch/20220621155248.3031687-1-cian.ferri...@intel.com/ > be great to have. Currently assembler checks are not working properly > for MSVC in AppVeyor, that prevents the code from being built, even > though the compiler does support all the options: > > checking binutils avx512 assembler checks passing... Assembler messages: > Fatal error: no compiled in support for x86_64 > no > rm: cannot lstat `build-aux/binutils_avx512_check.o': No such file or > directory > checking whether build-aux/cccl accepts -mavx512f... yes > checking whether build-aux/cccl accepts -mavx512bw... yes > checking whether build-aux/cccl accepts -mavx512vbmi... yes > checking whether build-aux/cccl accepts -mavx512vpopcntdq... yes > > It seems like it tries to run the wrong assembler. It is using > the 'as' provided by the mingw, while it should, probably, run > 'ml' or 'ml64' which is actually the assembler that MSVC will use. > We may, probably, also skip the assembler check for MSVC since > the assembler bug is gcc-specific (?). > I added a fix along the lines of what you described in this patch set: http://patchwork.ozlabs.org/project/openvswitch/list/?series=306022 Basically we only perform the assembler check when we detect GCC as the compiler. I've tested this with the Appveyor Windows CI and I can see that the BINUTILS check passes but I'm seeing other warnings. I've put more details in the cover letter for the RFC patch: http://patchwork.ozlabs.org/project/openvswitch/cover/20220622154943.3110933-1-cian.ferri...@intel.com/ > Best regards, Ilya Maximets. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v10 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
> -Original Message-
> From: Amber, Kumar
> Sent: Tuesday 31 May 2022 15:01
> To: ovs-dev@openvswitch.org
> Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; david.march...@redhat.com; f...@sysclose.org; Van
> Haaren, Harry
> ; Amber, Kumar
> Subject: [PATCH v10 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
>
> Add AVX512 Ipv6 optimized profile for vlan/IPv6/UDP and
> vlan/IPv6/TCP, IPv6/UDP and IPv6/TCP.
>
> MFEX autovalidaton test-case already has the IPv6 support for
> validating against the scalar mfex.
>
> Signed-off-by: Kumar Amber
> Signed-off-by: Harry van Haaren
> Co-authored-by: Harry van Haaren
>
> ---
> v10:
> - Rebase on top of Partial Avx512 changes patch.
Thanks for rebasing on top of my AVX512 build changes and following the same
patterns with your changes here Amber, from a code review it all looks good.
I also tested with GCC 4.8, 4.9, 5 and 9. They are all working as expected with
the respective configure outputs, config.h content and actual AVX512
implementations correctly working.
> v9:
> - Fix Ubscan memory alinged access.
> +
> +/* IPv6 Protocol specific helper functions, for handling L4 UDP/TCP. */
> +static inline void
> +mfex_handle_ipv6_l4(const uint8_t *ports, uint64_t *block)
> +{
> +memcpy(block, ports, sizeof(uint32_t));
> +}
> +
ACK the new memcpy approach to be UB safe, looking at the ASM before and
afterwards, it looks the same to me. Nice
Acked-by: Cian Ferriter
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6] tests/mfex: Improve pcap script for mfex tests.
> -Original Message- > From: Amber, Kumar > Sent: Thursday 26 May 2022 12:55 > To: ovs-dev@openvswitch.org > Cc: Ferriter, Cian ; i.maxim...@ovn.org; > echau...@redhat.com; Stokes, Ian > ; Van Haaren, Harry ; > Amber, Kumar > > Subject: [PATCH v6] tests/mfex: Improve pcap script for mfex tests. > > The mfex pcap generation script is improved for varied length > traffic and also removes the hard coded mfex_pcap and instead uses > the script itself to generate complex traffic patterns for testing. > > Signed-off-by: Kumar Amber > Acked-by: Cian Ferriter > The missing directory error is fixed in this version of the patch, thanks Amber. I think this way of using the directory created for each test (tests/system-dpdk-testsuite.dir/6/) is much better. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v8 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
> -Original Message- > From: Amber, Kumar > Sent: Thursday 26 May 2022 11:09 > To: ovs-dev@openvswitch.org > Cc: echau...@redhat.com; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; david.march...@redhat.com; f...@sysclose.org; Van > Haaren, Harry > ; Amber, Kumar > Subject: [PATCH v8 3/4] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles > > Add AVX512 Ipv6 optimized profile for vlan/IPv6/UDP and > vlan/IPv6/TCP, IPv6/UDP and IPv6/TCP. > > MFEX autovalidaton test-case already has the IPv6 support for > validating against the scalar mfex. > > Signed-off-by: Kumar Amber > Signed-off-by: Harry van Haaren > Co-authored-by: Harry van Haaren > > --- > v8: > - Rename defines for packet offsets. > v7: > - Fix Lenght checks for plen. > v5: > - Add variable length checks for IPv6 and TCP. > v4: > - Rebase to master. > v2: > - Fix CI build error. > - Fix check-patch sign-offs. > --- Hi Amber, Thanks for fixing the length checks, the minimum IPV6 packet test I was running before which was failing, is now passing, the length checks look good to me. I'm seeing ~20% improvement in performance for a simple P2P with IPV6 packets Before: AVX512 DPIF + DPCLS and Scalar MFEX After: AVX512 DPIF + MFEX + DPCLS I'm happy to Ack the patch: Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5] tests/mfex: Improve pcap script for mfex tests.
> >>> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index
> >>> 7d2715c4a..ac83e5a57 100644
> >>> --- a/tests/system-dpdk.at
> >>> +++ b/tests/system-dpdk.at
> >>> @@ -226,17 +226,19 @@ dnl
> >>> --
> >>>
> >>> dnl Add standard DPDK PHY port
> >>> AT_SETUP([OVS-DPDK - MFEX Autovalidator])
> >>> AT_KEYWORDS([dpdk])
> >>> -
> >>> +OVS_DPDK_PRE_CHECK()
> >>> OVS_DPDK_START()
> >>> -
> >>> -dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl
> >>> add-br br0 -- set bridge br0 datapath_type=netdev])
> >>> -AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dpdk
> >>> options:dpdk-
> >> devargs=net_pcap1,rx_pcap=$srcdir/pcap/mfex_test.pcap,inf
> >>> inite_rx=1], [], [stdout], [stderr]) -AT_CHECK([ovs-vsctl show], [],
> >>> [stdout])
> >>> -
> >>> AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d |
> >>> grep "True"], [], [dnl
> >>> ])
> >>>
> >>> +AT_SKIP_IF([! $PYTHON3 -c "import scapy"], [], [])
> >>> +AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir 2000], [], [stdout])
> >>
> >> Have you tested this patch on an MFEX machine on a clean branch? I do not
> >> have an AVX512 machine, but if I move this above the check I get errors due
> >> to missing tests/pcap/ directory.
> >>
> >> --- /dev/null 2022-04-25 09:31:38.866748964 +0200
> >> +++
> >> /home/echaudron/Documents/review/ovs_akumar_mfeximp/ovs_github/te
> >> sts/system-dpdk-testsuite.dir/at-groups/6/stderr 2022-05-25
> >> 11:17:40.116614366 +0200
> >> @@ -0,0 +1,6 @@
> >> +Traceback (most recent call last):
> >> + File
> >> "/home/echaudron/Documents/review/ovs_akumar_mfeximp/ovs_github/t
> >> ests/system-dpdk-testsuite.dir/6/../.././mfex_fuzzy.py", line 22, in
> >>
> >> +pktdump = PcapWriter(path, append=False, sync=True)
> >> + File "/usr/lib/python3.10/site-packages/scapy/utils.py", line 1686, in
> >> __init__
> >> +self.f = open(filename, append and "ab" or "wb", bufsz)
> >> +FileNotFoundError: [Errno 2] No such file or directory:
> >> '../.././pcap/fuzzy.pcap'
> >> stdout:
> >>
> >
> > Yes, I have tested on AVX512 machine.
> >
> > If you compare this patch with the reorder you suggested in the first
> > reply, I had to do a bit of
> > Reorder in the first suggestion because I saw the same error:
> >
> > We need to add a port to ovs to run the "get-mfex" command for skip to work
> > thus kept the
> > Add port before the AT_SKIP("mfex-get).
> >
> > But kept the Pcap generation then subsequent netdev bridge addition later
> > after skip.
> > If you move the netdev bridge addition before pcap generation, you will get
> > the file missing error.
>
> This is not the problem, you removed the only test/pcap/… file which will
> result in the pcap directory
> being removed from the GitHub repo, so it no longer exist, hence you get the
> error.
>
> Whoever reviewed your patches should have run into this if he applied the
> patches thought git am.
>
Hi Eelco and Amber,
Oops, I missed this, good catch. While I did apply and test this patch, the key
thing I missed was making sure all files are cleaned up from previous OVS
builds and tests. So I was testing with a still existing test/pcap/ directory.
Apologies for that. I will make sure to test patches from a clean OVS repo
state in the future.
I tested this patch again after a "git clean -fdx" to make clean up build and
test files and can reproduce the error that Eelco sees:
+Traceback (most recent call last):
+ File "../.././mfex_fuzzy.py", line 18, in
+pktdump = PcapWriter(path, append=False, sync=True)
+ File "/usr/local/lib/python3.8/dist-packages/scapy/utils.py", line 1686, in
__init__
+self.f = open(filename, append and "ab" or "wb", bufsz)
+FileNotFoundError: [Errno 2] No such file or directory:
'../.././pcap/fuzzy.pcap'
stdout:
./system-dpdk.at:232: exit code was 1, expected 0
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] classifier: adjust segment boundary to execute prerequisite processing
> -Original Message-
> From: Aaron Conole
> Sent: Thursday 19 May 2022 21:53
> To: Ferriter, Cian
> Cc: d...@openvswitch.org; Flavio Leitner ; Ilya Maximets
> ; Rashid
> Khan
> Subject: Re: [ovs-dev] [PATCH] classifier: adjust segment boundary to execute
> prerequisite processing
>
> "Ferriter, Cian" writes:
>
> > Hi Aaron,
>
> Hi Cian,
>
> >
> >
> >> ---
> >> include/openvswitch/flow.h | 7 +++
> >> tests/classifier.at| 27 +++
> >> 2 files changed, 30 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/include/openvswitch/flow.h b/include/openvswitch/flow.h
> >> index 3054015d93..df10cf579e 100644
> >> --- a/include/openvswitch/flow.h
> >> +++ b/include/openvswitch/flow.h
> >> @@ -141,15 +141,14 @@ struct flow {
> >> uint8_t nw_tos; /* IP ToS (including DSCP and ECN). */
> >> uint8_t nw_ttl; /* IP TTL/Hop Limit. */
> >> uint8_t nw_proto; /* IP protocol or low 8 bits of ARP
> >> opcode. */
> >> +/* L4 (64-bit aligned) */
> >> struct in6_addr nd_target; /* IPv6 neighbor discovery (ND) target. */
> >> struct eth_addr arp_sha;/* ARP/ND source hardware address. */
> >> struct eth_addr arp_tha;/* ARP/ND target hardware address. */
> >> -ovs_be16 tcp_flags; /* TCP flags/ICMPv6 ND options type.
> >> - * With L3 to avoid matching L4. */
> >> +ovs_be16 tcp_flags; /* TCP flags/ICMPv6 ND options type. */
> >> ovs_be16 pad2; /* Pad to 64 bits. */
> >> struct ovs_key_nsh nsh; /* Network Service Header keys */
> >>
> >> -/* L4 (64-bit aligned) */
> >> ovs_be16 tp_src;/* TCP/UDP/SCTP source port/ICMP type. */
> >> ovs_be16 tp_dst;/* TCP/UDP/SCTP destination port/ICMP
> >> code. */
> >> ovs_be16 ct_tp_src; /* CT original tuple source port/ICMP
> >> type. */
> >> @@ -179,7 +178,7 @@ BUILD_ASSERT_DECL(offsetof(struct flow,
> >> igmp_group_ip4) + sizeof(uint32_t)
> >
> > About the following BUILD_ASSRT_DECL which is in include/openvswitch/flow.h
> > directly below struct
> flow.h:
> > /* Remember to update FLOW_WC_SEQ when changing 'struct flow'. */
> > BUILD_ASSERT_DECL(offsetof(struct flow, igmp_group_ip4) + sizeof(uint32_t)
> > == sizeof(struct flow_tnl) + sizeof(struct ovs_key_nsh) +
> > 300
> > && FLOW_WC_SEQ == 42);
> >
> > Should we increment the FLOW_WC_SEQ value? The comment reads:
> > /* This sequence number should be incremented whenever anything involving
> > flows
> > * or the wildcarding of flows changes. This will cause build assertion
> > * failures in places which likely need to be updated. */
> >
> > We haven't changed anything in the order or content of struct flow but we
> > have changed (fixed) how
> flows are wildcarded. It doesn't look like any of the code asserting
> FLOW_WC_SEQ == 42 and relying on
> struct flow order and content needs updating.
> > Just wondering your/others thoughts about whether to increment to 43 or not.
>
> I decided not to increment it because it's really a subtable
> configuration for the classifier rather than a flow structure related
> change. In ofproto/ofpcoto.c we initialize the classifier with the map
> stored in flow.c that holds the various segments. I consider it more of
> a classifier configuration, in that sense. As you note, it doesn't
> modify the layout or size of flow struct. Maybe we can have a more
> precise comment around that area.
>
That makes sense, I agree.
> >> enum {
> >> FLOW_SEGMENT_1_ENDS_AT = offsetof(struct flow, dl_dst),
> >> FLOW_SEGMENT_2_ENDS_AT = offsetof(struct flow, nw_src),
> >> -FLOW_SEGMENT_3_ENDS_AT = offsetof(struct flow, tp_src),
> >> +FLOW_SEGMENT_3_ENDS_AT = offsetof(struct flow, nd_target),
> >> };
> >> BUILD_ASSERT_DECL(FLOW_SEGMENT_1_ENDS_AT % sizeof(uint64_t) == 0);
> >> BUILD_ASSERT_DECL(FLOW_SEGMENT_2_ENDS_AT % sizeof(uint64_t) == 0);
> >
> > IIUC, we are moving the L4 boundary 56B earlier in struct flow. This
> > means L3/L4 segment boundary is still at a U64 boundary. I know we
> > have BUILD_ASSERTs checking this, but I just thought to double check
> > myself.
>
> Yes - it's 448 bits or 7 u64 words. This is a bit lucky for us -
> otherwise we might have had to g
Re: [ovs-dev] [PATCH dpdk-latest] system-dpdk: Update vhost tests to be compatible with DPDK 22.03.
> -Original Message-
> From: Pai G, Sunil
> Sent: Tuesday 17 May 2022 15:11
> To: d...@openvswitch.org
> Cc: Stokes, Ian ; david.march...@redhat.com;
> maxime.coque...@redhat.com;
> Ferriter, Cian
> Subject: [PATCH dpdk-latest] system-dpdk: Update vhost tests to be compatible
> with DPDK 22.03.
>
> The DPDK commit [1] improves the socket layer logs in the vhost library
> to ease log filtering and debugging.
> Update the system-dpdk vhost tests to reflect this change.
>
> [1] c85c35b1d447 ("vhost: improve socket layer logs")
>
> Signed-off-by: Sunil Pai G
I've tested the before and after cases here with latest OVS master and latest
DPDK main and I can verify that:
All 3 vhost-user tests fail before:
3: OVS-DPDK - add vhost-user-client port FAILED (system-dpdk.at:66)
4: OVS-DPDK - ping vhost-user portsFAILED (system-dpdk.at:100)
5: OVS-DPDK - ping vhost-user-client ports FAILED (system-dpdk.at:175)
All 3 pass after:
3: OVS-DPDK - add vhost-user-client port ok
4: OVS-DPDK - ping vhost-user portsok
5: OVS-DPDK - ping vhost-user-client ports ok
The AT changes look good to me, I'm happy to Ack.
Acked-by: Cian Ferriter
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] classifier: adjust segment boundary to execute prerequisite processing
Hi Aaron,
> ---
> include/openvswitch/flow.h | 7 +++
> tests/classifier.at| 27 +++
> 2 files changed, 30 insertions(+), 4 deletions(-)
>
> diff --git a/include/openvswitch/flow.h b/include/openvswitch/flow.h
> index 3054015d93..df10cf579e 100644
> --- a/include/openvswitch/flow.h
> +++ b/include/openvswitch/flow.h
> @@ -141,15 +141,14 @@ struct flow {
> uint8_t nw_tos; /* IP ToS (including DSCP and ECN). */
> uint8_t nw_ttl; /* IP TTL/Hop Limit. */
> uint8_t nw_proto; /* IP protocol or low 8 bits of ARP opcode.
> */
> +/* L4 (64-bit aligned) */
> struct in6_addr nd_target; /* IPv6 neighbor discovery (ND) target. */
> struct eth_addr arp_sha;/* ARP/ND source hardware address. */
> struct eth_addr arp_tha;/* ARP/ND target hardware address. */
> -ovs_be16 tcp_flags; /* TCP flags/ICMPv6 ND options type.
> - * With L3 to avoid matching L4. */
> +ovs_be16 tcp_flags; /* TCP flags/ICMPv6 ND options type. */
> ovs_be16 pad2; /* Pad to 64 bits. */
> struct ovs_key_nsh nsh; /* Network Service Header keys */
>
> -/* L4 (64-bit aligned) */
> ovs_be16 tp_src;/* TCP/UDP/SCTP source port/ICMP type. */
> ovs_be16 tp_dst;/* TCP/UDP/SCTP destination port/ICMP code.
> */
> ovs_be16 ct_tp_src; /* CT original tuple source port/ICMP type.
> */
> @@ -179,7 +178,7 @@ BUILD_ASSERT_DECL(offsetof(struct flow, igmp_group_ip4) +
> sizeof(uint32_t)
About the following BUILD_ASSRT_DECL which is in include/openvswitch/flow.h
directly below struct flow.h:
/* Remember to update FLOW_WC_SEQ when changing 'struct flow'. */
BUILD_ASSERT_DECL(offsetof(struct flow, igmp_group_ip4) + sizeof(uint32_t)
== sizeof(struct flow_tnl) + sizeof(struct ovs_key_nsh) + 300
&& FLOW_WC_SEQ == 42);
Should we increment the FLOW_WC_SEQ value? The comment reads:
/* This sequence number should be incremented whenever anything involving flows
* or the wildcarding of flows changes. This will cause build assertion
* failures in places which likely need to be updated. */
We haven't changed anything in the order or content of struct flow but we have
changed (fixed) how flows are wildcarded. It doesn't look like any of the code
asserting FLOW_WC_SEQ == 42 and relying on struct flow order and content needs
updating.
Just wondering your/others thoughts about whether to increment to 43 or not.
> enum {
> FLOW_SEGMENT_1_ENDS_AT = offsetof(struct flow, dl_dst),
> FLOW_SEGMENT_2_ENDS_AT = offsetof(struct flow, nw_src),
> -FLOW_SEGMENT_3_ENDS_AT = offsetof(struct flow, tp_src),
> +FLOW_SEGMENT_3_ENDS_AT = offsetof(struct flow, nd_target),
> };
> BUILD_ASSERT_DECL(FLOW_SEGMENT_1_ENDS_AT % sizeof(uint64_t) == 0);
> BUILD_ASSERT_DECL(FLOW_SEGMENT_2_ENDS_AT % sizeof(uint64_t) == 0);
IIUC, we are moving the L4 boundary 56B earlier in struct flow. This means
L3/L4 segment boundary is still at a U64 boundary. I know we have BUILD_ASSERTs
checking this, but I just thought to double check myself.
> diff --git a/tests/classifier.at b/tests/classifier.at
> index cdcd72c156..a0a8fe0359 100644
> --- a/tests/classifier.at
> +++ b/tests/classifier.at
> @@ -129,6 +129,33 @@ Datapath actions: 3
> OVS_VSWITCHD_STOP(["/'prefixes' with incompatible field: ipv6_label/d"])
> AT_CLEANUP
>
> +AT_SETUP([flow classifier - ipv6 ND dependancy])
> +OVS_VSWITCHD_START
> +add_of_ports br0 1 2
> +AT_DATA([flows.txt], [dnl
> + table=0,priority=100,ipv6,ipv6_src=1000::/10 actions=resubmit(,1)
> + table=0,priority=0 actions=NORMAL
> + table=1,priority=110,ipv6,ipv6_dst=1000::3 actions=resubmit(,2)
> + table=1,priority=100,ipv6,ipv6_dst=1000::4 actions=resubmit(,2)
> + table=1,priority=0 actions=NORMAL
> +
> table=2,priority=120,icmp6,nw_ttl=255,icmp_type=135,icmp_code=0,nd_target=1000::1
> actions=NORMAL
> + table=2,priority=100,tcp actions=NORMAL
> + table=2,priority=100,icmp6 actions=NORMAL
> + table=2,priority=0 actions=NORMAL
> +])
> +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> +
> +# test ICMPv6 echo request (which should have no nd_target field)
> +AT_CHECK([ovs-appctl ofproto/trace br0
> "in_port=1,eth_src=f6:d2:b0:19:5e:7b,eth_dst=d2:49:19:91:78:fe,dl_type=0x86dd,ipv6_src=1000::3,ipv6_ds
> t=1000::4,nw_proto=58,icmpv6_type=128,icmpv6_code=0"], [0], [stdout])
> +AT_CHECK([tail -2 stdout], [0],
> + [Megaflow:
> recirc_id=0,eth,icmp6,in_port=1,dl_src=f6:d2:b0:19:5e:7b,dl_dst=d2:49:19:91:78:fe,ipv6_src=1000::/10,i
> pv6_dst=1000::4,nw_ttl=0,nw_frag=no
> +Datapath actions: 100,2
> +])
> +
> +AT_CLEANUP
> +
> +
> +
Nit: The other tests in the file have one line after the "AT_CLEANUP", maybe
this should be the same instead of 3 lines?
> AT_BANNER([conjunctive match])
>
> AT_SETUP([single conjunctive match])
I've applied the patch to the latest master, run the unit
Re: [ovs-dev] [PATCH] dpif-netdev: Only hash port number when necessary.
> -Original Message- > From: Ilya Maximets > Sent: Wednesday 18 May 2022 11:35 > To: Mike Pattrick ; Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > Subject: Re: [ovs-dev] [PATCH] dpif-netdev: Only hash port number when > necessary. > > On 5/10/22 17:03, Mike Pattrick wrote: > > On Fri, Apr 29, 2022 at 11:44 AM Cian Ferriter > > wrote: > >> > >> The hash of the port number is only needed when a DPCLS needs to be > >> created. Move the hash calculation inside the if to accomplish this. > >> > >> Signed-off-by: Cian Ferriter > >> --- > > > > Good catch! > > > > Acked-by: Mike Pattrick > > I'd guess that compiler will produce identical code in most cases, > but the change makes the code a bit cleaner. > > Applied. Thanks! > > Best regards, Ilya Maximets. Thanks for the review Mike and for applying Ilya. Agreed, the compiler should clean this up for us. I was thinking of the change as a code readability improvement rather than an optimization. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v5 5/5] acinclude: Add seperate checks for AVX512 ISA.
> -Original Message- > From: Pai G, Sunil > Sent: Tuesday 17 May 2022 15:52 > To: Ferriter, Cian ; ovs-dev@openvswitch.org > Cc: Van Haaren, Harry > Subject: RE: [PATCH v5 5/5] acinclude: Add seperate checks for AVX512 ISA. > > > Checking for each of the required AVX512 ISA separately will allow the > > compiler to generate some AVX512 code where there is some support in the > > compiler rather than only generating all AVX512 code when all of it is > > supported or no AVX512 code at all. > > > > For example, in GCC 4.9 where there is just support for AVX512F, this > > patch will allow building the AVX512 DPIF. > > > > Another example, in GCC 5 and 6, most AVX512 code can be generated, just > > without AVX512VPOPCNTDQ support. > > > > Signed-off-by: Cian Ferriter > > > > --- > > v5: > > * Create a selector function for the permutexvar implementations based > > on Sunil's feedback on the v4. This hides the complexity of compile > > time and run time selection of permutexvar implementations. > > * Add a comment explaining why VPOPCNTDQ_TARGET is defined and used. > > > > v4: > > * Combine the 3 commits which added checks for AVX512 ISA into this > > single commit since the first 2 commits were only useful and active > > when the 3rd commit was applied. This also takes care of Sunil's > > comment about explaining that the first 2 commits are precursors. > > * Don't check for AVX512DQ availability in the compiler. This ISA isn't > > used in OVS. > > * Put all AVX512 ISA checks in the OVS_CHECK_AVX512 macro as per Sunil's > > feedback. > > * Define a function in acinclude.m4, (OVS_CONDITIONAL_CC_OPTION_DEFINE), > > to help with checking for AVX512 ISA support in the compiler. > > * Remove the '__AVX512VPOPCNTDQ__' check. Use the HAVE_AVX512* pattern > > consistently with all AVX512 ISA checks instead. Fixup the comment > > explaining the _mm512_popcnt_epi64_wrapper() function to reflect this. > > > Based on the comments and responses to v4 and changes implemented in v5 , > > Acked-by: Sunil Pai G Thanks for the reviews on this and other patches in the series Sunil! ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v4 5/5] acinclude: Add seperate checks for AVX512 ISA.
> -Original Message-
> From: Pai G, Sunil
> Sent: Thursday 5 May 2022 11:26
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Cc: Van Haaren, Harry
> Subject: RE: [PATCH v4 5/5] acinclude: Add seperate checks for AVX512 ISA.
>
> Hi Cian,
>
> Few comments inline.
>
>
>
Hi Sunil,
Thanks for the review. Responses inline.
> > diff --git a/acinclude.m4 b/acinclude.m4 index 61e88105f..7b2889a40 100644
> > --- a/acinclude.m4
> > +++ b/acinclude.m4
> > @@ -73,16 +73,13 @@ AC_DEFUN([OVS_CHECK_DPIF_AVX512_DEFAULT], [
> >
> > dnl OVS_CHECK_AVX512
> > dnl
> > -dnl Checks if compiler and binutils supports AVX512.
> > +dnl Checks if compiler and binutils supports various AVX512 ISA.
> > AC_DEFUN([OVS_CHECK_AVX512], [
> >OVS_CHECK_BINUTILS_AVX512
> > - OVS_CHECK_CC_OPTION(
> > -[-mavx512f -mavx512vpopcntdq], [ovs_have_cc_mavx512f=yes],
> > [ovs_have_cc_mavx512f=no])
> > - AM_CONDITIONAL([HAVE_AVX512F], [test $ovs_have_cc_mavx512f = yes])
> > - if test "$ovs_have_cc_mavx512f" = yes; then
> > -AC_DEFINE([HAVE_AVX512F], [1],
> > - [Define to 1 if compiler supports AVX512.])
> > - fi
> > + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f], [HAVE_AVX512F])
> > + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512bw], [HAVE_AVX512BW])
> > + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vbmi], [HAVE_AVX512VBMI])
> > + OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vpopcntdq],
> > + [HAVE_AVX512VPOPCNTDQ])
> > ])
> >
> > dnl OVS_ENABLE_WERROR
> > @@ -1360,6 +1357,19 @@ AC_DEFUN([OVS_CONDITIONAL_CC_OPTION],
> > AM_CONDITIONAL([$2], [test $ovs_have_cc_option = yes])]) dnl
> > --
> >
> > +dnl OVS_CONDITIONAL_CC_OPTION_DEFINE([OPTION], [CONDITIONAL]) dnl Check
> > +whether the given C compiler OPTION is accepted.
> > +dnl If so, enable the given Automake CONDITIONAL and define it.
> > +dnl Example: OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f],
> > +[HAVE_AVX512F]) AC_DEFUN([OVS_CONDITIONAL_CC_OPTION_DEFINE],
> > + [OVS_CHECK_CC_OPTION(
> > +[$1], [ovs_have_cc_option=yes], [ovs_have_cc_option=no])
> > + AM_CONDITIONAL([$2], [test $ovs_have_cc_option = yes])
> > + if test "$ovs_have_cc_option" = yes; then
> > + AC_DEFINE([$2], [1],
> > + [Define to 1 if compiler supports the '$1' option.])
> > + fi])
> > +
>
> Thanks for creating a common version to check the CPUID flags.
>
>
>
> > # Build core vswitch libraries as before lib_libopenvswitch_la_SOURCES =
> > \ diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-
> > avx512.c
> > index 2815bf480..7011eb00f 100644
> > --- a/lib/dpif-netdev-extract-avx512.c
> > +++ b/lib/dpif-netdev-extract-avx512.c
>
>
> > @@ -110,7 +110,9 @@ _mm512_maskz_permutex2var_epi8_skx(__mmask64 k_mask,
>
>
> Since we now include this file only when AVX512BW is available, I guess we
> can remove the
> __attribute__((target("avx512bw"))) for _mm512_maskz_permutex2var_epi8_skx ?
>
>
Yes, good catch, we can and should remove this attribute target. This attribute
target was unnecessary even before the changes in this patch. The -mavx512bw
CFLAG is set for the entire AVX512 OVS library before and after the patch set.
I'll remove this attribute target in the "dpif-netdev-extract: Remove
unnecessary compiler targets." patch (3/5). I'll keep your Ack on patch 3/5
since you suggested this change.
> >
> > /* Wrapper function required to enable ISA. */ static inline __m512i
> > +#if HAVE_AVX512VBMI
> > __attribute__((__target__("avx512vbmi")))
> > +#endif
> > _mm512_maskz_permutexvar_epi8_wrap(__mmask64 kmask, __m512i idx, __m512i
> > a) {
> > return _mm512_maskz_permutexvar_epi8(kmask, idx, a); @@ -481,7 +483,7
> > @@ mfex_avx512_process(struct dp_packet_batch *packets,
> > odp_port_t in_port,
> > void *pmd_handle OVS_UNUSED,
> > const enum MFEX_PROFILES profile_id,
> > -const uint32_t use_vbmi)
> > +const uint32_t use_vbmi OVS_UNUSED)
> > {
> > uint32_t hitmask = 0;
> > struct dp_packet *packet;
> > @@ -544,7 +546,11 @@ mfex_avx512_process(struct dp_packet_batch *packets,
> > */
> > __m512i v512_zeros = _mm512_setzero_si512();
> > __m512i v_blk0;
> > +#if HAVE_AVX512VBMI
> > if (__builtin_constant_p(use_vbmi) && u
Re: [ovs-dev] [PATCH] Documentation: Clarify QEMU version requirement.
> -Original Message- > From: Ilya Maximets > Sent: Wednesday 4 May 2022 22:31 > To: Ferriter, Cian ; ovs-dev@openvswitch.org > Cc: i.maxim...@ovn.org > Subject: Re: [ovs-dev] [PATCH] Documentation: Clarify QEMU version > requirement. > > On 4/26/22 18:25, Cian Ferriter wrote: > > The QEMU version requirement of >= 2.7 is for vhost-user-client ports > > specifically. > > > > Signed-off-by: Cian Ferriter > > --- > > Documentation/topics/dpdk/vhost-user.rst | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > Applied. Thanks! > > Best regards, Ilya Maximets. Thanks Ilya! ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2 3/4] Miniflow_extract: Refactor miniflow_extract into api.
> -Original Message-
> From: Amber, Kumar
> Sent: Sunday 27 March 2022 08:09
> To: ovs-dev@openvswitch.org
> Cc: Stokes, Ian ; echau...@redhat.com;
> ktray...@redhat.com; i.maxim...@ovn.org;
> Ferriter, Cian ; f...@sysclose.org; Van Haaren, Harry
> ; Amber, Kumar
> Subject: [PATCH v2 3/4] Miniflow_extract: Refactor miniflow_extract into api.
I had a look at previous since the 'miniflow_extract:' commit title beginning
looked strange. Commits which change the actual miniflow_extract() function
code or surrounding code typically have 'flow:' as the beginning of the commit
title. Can we use that?
>
> Miniflow extract used to takes the ABI parameter struct
> miniflow which was removed and added inside
> the struct netdev_flow_key and at many places temperory
> structs were created inside the functions which could be
> cleaned in favour of a uniform API.
>
> Signed-off-by: Kumar Amber
> ---
> lib/dpif-netdev-avx512.c | 2 +-
> lib/dpif-netdev-private-extract.c | 2 +-
> lib/dpif-netdev.c | 2 +-
> lib/flow.c| 28 +++-
> lib/flow.h| 6 +-
> ofproto/ofproto.c | 10 --
> 6 files changed, 31 insertions(+), 19 deletions(-)
>
> diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c
> index b7131ba3f..76eeecc9a 100644
> --- a/lib/dpif-netdev-avx512.c
> +++ b/lib/dpif-netdev-avx512.c
> @@ -211,7 +211,7 @@ dp_netdev_input_outer_avx512(struct dp_netdev_pmd_thread
> *pmd,
>
> if (!mfex_hit) {
> /* Do a scalar miniflow extract into keys. */
> -miniflow_extract(packet, >mf);
> +miniflow_extract(packet, key);
> }
>
> /* Cache TCP and byte values for all packets. */
> diff --git a/lib/dpif-netdev-private-extract.c
> b/lib/dpif-netdev-private-extract.c
> index 4b2f12015..8538d069f 100644
> --- a/lib/dpif-netdev-private-extract.c
> +++ b/lib/dpif-netdev-private-extract.c
> @@ -251,7 +251,7 @@ dpif_miniflow_extract_autovalidator(struct
> dp_packet_batch *packets,
> /* Run scalar miniflow_extract to get default result. */
> DP_PACKET_BATCH_FOR_EACH (i, packet, packets) {
> pkt_metadata_init(>md, in_port);
> -miniflow_extract(packet, [i].mf);
> +miniflow_extract_(packet, [i]);
>
> /* Store known good metadata to compare with optimized metadata. */
> good_l2_5_ofs[i] = packet->l2_5_ofs;
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 5fc68bdbe..fd3fe510d 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -8144,7 +8144,7 @@ dfc_processing(struct dp_netdev_pmd_thread *pmd,
> }
> }
>
> -miniflow_extract(packet, >mf);
> +miniflow_extract(packet, key);
> key->len = 0; /* Not computed yet. */
> key->hash =
> (md_is_valid == false)
> diff --git a/lib/flow.c b/lib/flow.c
> index dd523c889..127de2d7a 100644
> --- a/lib/flow.c
> +++ b/lib/flow.c
> @@ -35,6 +35,8 @@
> #include "jhash.h"
> #include "openvswitch/match.h"
> #include "dp-packet.h"
> +#include "dpif-netdev-private-thread.h"
Why is this header include added as part of this patch?
> +#include "dpif-netdev-private-dpcls.h"
> #include "openflow/openflow.h"
> #include "packets.h"
> #include "odp-util.h"
> @@ -633,15 +635,18 @@ parse_nsh(const void **datap, size_t *sizep, struct
> ovs_key_nsh *key)
> void
> flow_extract(struct dp_packet *packet, struct flow *flow)
> {
> -struct {
> -struct miniflow mf;
> -uint64_t buf[FLOW_U64S];
> -} m;
This is the value add of this refactor. Not having multiple places in the code
where we build a "netdev_flow_key like" struct is good IMO.
But it comes at the cost of including a extra header in those places. I think
it's worth it.
> +
> +struct netdev_flow_key key;
>
> COVERAGE_INC(flow_extract);
>
> -miniflow_extract(packet, );
> -miniflow_expand(, flow);
> +/* Changing parameter to key will not affect anything as
> + * buff array is still followed by the mf bit map inside
> + * netdev_flow_key, thus there wont be any impact on offset
> + * calculations which were done earlier.
> + */
This seems like a comment to explain why you are making a change. Comments in
the code should explain why things are done a certain way, not why something
was changed from one way to another way. That type of explanation belongs in a
commit message.
> +miniflow_extract(packet, );
> +
Re: [ovs-dev] [PATCH v2 2/4] dpif-netdev: Refactor hashing function.
Is this patch necessary? I don't see the usage of
dpif_netdev_packet_get_rss_hash() change in the patch set. I can also remove
this patch from the series and still build and test the way I did before.
Am I missing something here?
> -Original Message-
> From: Amber, Kumar
> Sent: Sunday 27 March 2022 08:09
> To: ovs-dev@openvswitch.org
> Cc: Stokes, Ian ; echau...@redhat.com;
> ktray...@redhat.com; i.maxim...@ovn.org;
> Ferriter, Cian ; f...@sysclose.org; Van Haaren, Harry
> ; Amber, Kumar
> Subject: [PATCH v2 2/4] dpif-netdev: Refactor hashing function.
>
> The changes moves the get_rss hashing function to its
> own .h files so that it can be used accross ovs which
> was previously only limited to dpif-netdev.c.
>
> Signed-off-by: Kumar Amber
> Signed-off-by: Cian Ferriter
> Co-authored-by: Cian Ferriter
> ---
> lib/dpif-netdev-private-dpcls.h | 23 +++
> lib/dpif-netdev.c | 22 --
> 2 files changed, 23 insertions(+), 22 deletions(-)
>
> diff --git a/lib/dpif-netdev-private-dpcls.h b/lib/dpif-netdev-private-dpcls.h
> index 0d5da73c7..a86ea449b 100644
> --- a/lib/dpif-netdev-private-dpcls.h
> +++ b/lib/dpif-netdev-private-dpcls.h
> @@ -25,6 +25,7 @@
>
> #include "cmap.h"
> #include "openvswitch/thread.h"
> +#include "dpif-netdev-private-dpif.h"
>
> #ifdef __cplusplus
> extern "C" {
> @@ -124,6 +125,28 @@ dpif_netdev_packet_get_rss_hash_orig_pkt(struct
> dp_packet *packet,
> return hash;
> }
>
> +static inline uint32_t
> +dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
> +const struct miniflow *mf)
> +{
> +uint32_t hash;
> +
> +if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
> +hash = dp_packet_get_rss_hash(packet);
> +} else {
> +hash = miniflow_hash_5tuple(mf, 0);
> +dp_packet_set_rss_hash(packet, hash);
> +}
> +
> +/* The RSS hash must account for the recirculation depth to avoid
> + * collisions in the exact match cache */
> +uint32_t recirc_depth = *recirc_depth_get();
> +if (OVS_UNLIKELY(recirc_depth)) {
> +hash = hash_finish(hash, recirc_depth);
> +}
> +return hash;
> +}
> +
> /* Allow other implementations to call dpcls_lookup() for subtable search. */
> bool
> dpcls_lookup(struct dpcls *cls, const struct netdev_flow_key *keys[],
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 5c2123e0c..5fc68bdbe 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -7787,28 +7787,6 @@ dp_netdev_upcall(struct dp_netdev_pmd_thread *pmd,
> struct dp_packet *packet_,
> actions, wc, put_actions, dp->upcall_aux);
> }
>
> -static inline uint32_t
> -dpif_netdev_packet_get_rss_hash(struct dp_packet *packet,
> -const struct miniflow *mf)
> -{
> -uint32_t hash, recirc_depth;
> -
> -if (OVS_LIKELY(dp_packet_rss_valid(packet))) {
> -hash = dp_packet_get_rss_hash(packet);
> -} else {
> -hash = miniflow_hash_5tuple(mf, 0);
> -dp_packet_set_rss_hash(packet, hash);
> -}
> -
> -/* The RSS hash must account for the recirculation depth to avoid
> - * collisions in the exact match cache */
> -recirc_depth = *recirc_depth_get_unsafe();
> -if (OVS_UNLIKELY(recirc_depth)) {
> -hash = hash_finish(hash, recirc_depth);
> -}
> -return hash;
> -}
> -
> struct packet_batch_per_flow {
> unsigned int byte_count;
> uint16_t tcp_flags;
> --
> 2.25.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2 4/4] miniflow_extract: Add autovalidator support to miniflow_extract.
> -Original Message-
> From: Amber, Kumar
> Sent: Sunday 27 March 2022 08:09
> To: ovs-dev@openvswitch.org
> Cc: Stokes, Ian ; echau...@redhat.com;
> ktray...@redhat.com; i.maxim...@ovn.org;
> Ferriter, Cian ; f...@sysclose.org; Van Haaren, Harry
> ; Amber, Kumar
> Subject: [PATCH v2 4/4] miniflow_extract: Add autovalidator support to
> miniflow_extract.
>
> The patch adds the flag based switch between choice of using
> miniflow_extract in normal pipeline or select mfex_autovalidator
> in debug and test builds.
>
> The compile time flag used to select autoval can be done using option:
>
> ./configure CFLAGS="--enable-mfex-default-autovalidator"
>
> Signed-off-by: Kumar Amber
> ---
> lib/dpif-netdev-private-extract.c | 4 ++--
> lib/flow.c| 24
> 2 files changed, 26 insertions(+), 2 deletions(-)
>
> diff --git a/lib/dpif-netdev-private-extract.c
> b/lib/dpif-netdev-private-extract.c
> index 8538d069f..0d7091caa 100644
> --- a/lib/dpif-netdev-private-extract.c
> +++ b/lib/dpif-netdev-private-extract.c
> @@ -124,8 +124,8 @@ dpif_miniflow_extract_init(void)
> /* For the first call, this will be choosen based on the
> * compile time flag.
> */
> -VLOG_INFO("Default MFEX Extract implementation is %s.\n",
> - mfex_impls[mfex_idx].name);
> +VLOG_DBG("Default MFEX Extract implementation is %s.\n",
> + mfex_impls[mfex_idx].name);
> atomic_store_relaxed(mfex_func, (uintptr_t) mfex_impls
> [mfex_idx].extract_func);
> }
> diff --git a/lib/flow.c b/lib/flow.c
> index 127de2d7a..ddec31523 100644
> --- a/lib/flow.c
> +++ b/lib/flow.c
> @@ -37,6 +37,7 @@
> #include "dp-packet.h"
> #include "dpif-netdev-private-thread.h"
> #include "dpif-netdev-private-dpcls.h"
> +#include "dpif-netdev-private-extract.h"
> #include "openflow/openflow.h"
> #include "packets.h"
> #include "odp-util.h"
> @@ -1121,7 +1122,30 @@ miniflow_extract_(struct dp_packet *packet, struct
> netdev_flow_key *key)
> void
> miniflow_extract(struct dp_packet *packet, struct netdev_flow_key *key)
> {
> +#ifdef MFEX_AUTOVALIDATOR_DEFAULT
> +static struct ovsthread_once once_enable = OVSTHREAD_ONCE_INITIALIZER;
> +if (ovsthread_once_start(_enable)) {
> +dpif_miniflow_extract_init();
> +ovsthread_once_done(_enable);
> +}
> +struct dp_packet_batch packets;
> +const struct pkt_metadata *md = >md;
> +dp_packet_batch_init();
> +dp_packet_batch_add(, packet);
> +const uint32_t recirc_depth = *recirc_depth_get();
> +/* Currently AVX512 DPIF dont support recirculation
> + * Once the support will be added the condition would
> + * be removed.
> + */
> +if (recirc_depth) {
I'm just wondering if there is a way to avoid getting and checking the
recirc_depth. This would avoid the need for patch 1/4 in this series. What
about the below check instead?
if (flow_tnl_dst_is_set(>tunnel)) {
This is the check that happens inside the generic miniflow extract
implementation. Do you think this will work instead?
> +miniflow_extract_(packet, key);
> +} else {
> +dpif_miniflow_extract_autovalidator(, key, 1,
> +odp_to_u32(md->in_port.odp_port), NULL);
We need to be careful about the 'NULL' being passed in here. We need to add
checks to the use of the 'struct dp_netdev_pmd_thread *pmd_handle' in
'dpif_miniflow_extract_autovalidator()'.
> +}
> +#else
> miniflow_extract_(packet, key);
> +#endif
> }
> static ovs_be16
> parse_dl_type(const void **datap, size_t *sizep, ovs_be16 *first_vlan_tci_p)
> --
> 2.25.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2 0/4] Miniflow Extract Testing Improvements
> -Original Message- > From: Amber, Kumar > Sent: Sunday 27 March 2022 08:09 > To: ovs-dev@openvswitch.org > Cc: Stokes, Ian ; echau...@redhat.com; > ktray...@redhat.com; i.maxim...@ovn.org; > Ferriter, Cian ; f...@sysclose.org; Van Haaren, Harry > ; Amber, Kumar > Subject: [PATCH v2 0/4] Miniflow Extract Testing Improvements > > The patch-set introduces changes which would improve > the testing of miniflow_extract for AVX512 based > miniflow_extract optimizations whithout affecting scalar > code path. > > Kumar Amber (4): > dpif-netdev: Refactor per thread recirc data allocation. > dpif-netdev: Refactor hashing function. > Miniflow_extract: Refactor miniflow_extract into api. > miniflow_extract: Add autovalidator support to miniflow_extract. > > lib/dpif-netdev-avx512.c | 2 +- > lib/dpif-netdev-private-dpcls.h | 23 ++ > lib/dpif-netdev-private-dpif.c| 2 ++ > lib/dpif-netdev-private-dpif.h| 5 +++ > lib/dpif-netdev-private-extract.c | 6 ++-- > lib/dpif-netdev.c | 27 +--- > lib/flow.c| 52 +-- > lib/flow.h| 6 +++- > ofproto/ofproto.c | 10 +++--- > 9 files changed, 87 insertions(+), 46 deletions(-) > > -- > 2.25.1 Adding Ilya to the To: line since I looked at his feedback from the v1 of this patch set to see if it was addressed. Hi Amber and Ilya, I've been looking at and testing this patch set to see what improvements it brings to MFEX testing. I intentionally introduced errors to both generic and AVX512 implementations of miniflow extract to check what the testing was picking up on. Thanks to the patch set, when I build with the '--enable-mfex-default-autovalidator' at configure time, the intentionally introduced errors in the AVX512 implementations of miniflow extract are found. The 'make check' tests return 98 failures. I looked closely at the 'flow extractor' unit test that Ilya mentioned previously. It's in 'tests/test-flows.c', uses flows and packets generated from 'tests/flowgen.py' and can be run 2 ways: make check TESTSUITEFLAGS='-k extractor -v' OR $OVS_DIR/tests/ovstest test-flows flows pcap The intentionally introduced errors in the AVX512 implementations of miniflow extract are found. All packets used in 'test/test-flows.c' are passed through the autovalidator. The errors are reported by the miniflow extract autovalidator, not the actual 'test/test-flows.c' checks. I did have to fix one segfault happening in autovalidator because of the new way in which this patch set uses it. I'll highlight the cause of this segfault in the actual patch (patch 4/4). So with these changes, we are testing the avx512 versions of the miniflow_extract with all the packets we're testing the generic implementation with. However, since we are not removing the MFEX related .at tests, we still test the avx512 versions of miniflow extract with more packets than the generic implementation. @Ilya, I can see in your feedback on the v1, you would like to see the .at miniflow extract traffic tests to be removed as well as the 'mfex_fuzzy.py' file. I see value in what this patch set is doing, the non-fuzzy .at miniflow extract test is covered by the 'test-flows.c' test since the same types of packets are covered. The fuzz testing .at miniflow extract test still provides additional value and removing it would reduce testing coverage. This patch set adds value as is but I can see 2 shortcomings with the test coverage of this patch set if we want to remove the .at miniflow extract traffic tests and the 'mfex_fuzzy.py' file: 1. If the eventual goal is to remove the use of 'mfex_fuzzy.py', is 'flowgen.py' viable for all the AVX512 miniflow extract testing? I'm just thinking about testing IPv6 in the future when AVX512 implementations are added for it. 'flowgen.py' doesn't currently generate IPv6 packets. Is adding that support easy? I think it's a great idea because we can benefit all implementations of miniflow extract with improvements to 'flowgen.py', but I find Scapy is the easiest tool for generating PCAPs. 2. We lose fuzzy style testing of the AVX512 miniflow extract implementations. @Amber and Ilya, thoughts on the above shortcomings? Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCHv2] IPv6: Add IPv6 extension header support
> -Original Message- > From: Ilya Maximets > Sent: Monday 21 March 2022 16:56 > To: d...@openvswitch.org; Stokes, Ian ; Van Haaren, > Harry > ; Ferriter, Cian ; > Amber, Kumar > > Cc: i.maxim...@ovn.org; Toms Atteka > Subject: Re: [ovs-dev] [PATCHv2] IPv6: Add IPv6 extension header support > > On 3/18/22 22:34, Toms Atteka wrote: > > IPv6 extension headers carry optional internet layer information > > and are placed between the fixed header and the upper-layer > > protocol header. > > > > This change adds a new OpenFlow field OFPXMT_OFB_IPV6_EXTHDR and > > packets can be filtered using ipv6_ext flag. > > > > Some spacing style issues fixed. > > > > Tested-at: https://github.com/TomCodeLV/ovs/actions/runs/504185214 > > Signed-off-by: Toms Atteka > > --- > > build-aux/extract-odp-netlink-macros-h| 1 + > > build-aux/extract-ofp-fields | 4 +- > > datapath/flow.c | 192 +++--- > > datapath/flow.h | 14 ++ > > datapath/flow_netlink.c | 25 ++- > > .../linux/compat/include/linux/openvswitch.h | 6 + > > include/openvswitch/flow.h| 8 +- > > include/openvswitch/match.h | 2 + > > include/openvswitch/meta-flow.h | 18 ++ > > lib/dpif-netdev-extract-avx512.c | 2 +- > > lib/flow.c| 188 - > > lib/flow.h| 6 +- > > lib/match.c | 25 ++- > > lib/meta-flow.c | 80 +++- > > lib/meta-flow.xml | 14 ++ > > lib/nx-match.c| 13 +- > > lib/odp-execute.c | 2 + > > lib/odp-util.c| 111 +- > > lib/odp-util.h| 11 +- > > lib/ofp-match.c | 2 +- > > lib/packets.c | 28 +++ > > lib/packets.h | 12 ++ > > ofproto/ofproto-dpif-rid.h| 2 +- > > ofproto/ofproto-dpif-sflow.c | 1 + > > ofproto/ofproto-dpif-xlate.c | 2 +- > > ofproto/ofproto-dpif.c| 47 + > > tests/ofproto.at | 4 +- > > tests/system-traffic.at | 31 +++ > > 28 files changed, 789 insertions(+), 62 deletions(-) > > > > > > > diff --git a/lib/dpif-netdev-extract-avx512.c > > b/lib/dpif-netdev-extract-avx512.c > > index c1c1fefb6..3cbe04ca9 100644 > > --- a/lib/dpif-netdev-extract-avx512.c > > +++ b/lib/dpif-netdev-extract-avx512.c > > @@ -289,7 +289,7 @@ BUILD_ASSERT_DECL((OFFSETOFEND(struct dp_packet, l4_ofs) > > BUILD_ASSERT_DECL(FLOWMAP_UNITS == 2); > > > > /* Ensure the miniflow-struct ABI is the expected version. */ > > -BUILD_ASSERT_DECL(FLOW_WC_SEQ == 42); > > +BUILD_ASSERT_DECL(FLOW_WC_SEQ == 43); > > > > /* If the above build assert happens, this means that you might need to > > make > > * some modifications to the AVX512 miniflow extractor code. In general, > > the > > Harry, Cian, others, > > This patch requires some changes in the avx512 miniflow_extract > implementation, due to changes in the layout of the struct flow. > > Could you, please, assist with providing necessary adjustments? > > Best regards, Ilya Maximets. Hi Ilya/all, Thanks again for pointing out the 'struct miniflow' change here. >From a code inspection point of view, I can see that there will be AVX512 MFEX >changes required since 'struct flow' in 'include/openvswitch/flow.h' has been >modified with members being inserted into the middle of the struct. This means >the miniflow bits output by the scalar miniflow_extract will change, so AVX512 >MFEX must change to match. I can also see that a change to AVX512 MFEX is necessary from a testing point of view when I apply this patchset and do some MFEX autovalidator testing. The map or miniflow bits are different: > Autovalidation map failed > Good: 0x18a0 0x80801Test: 0x18a0 0x40401 I'm wondering about the best next steps here. I can see that there's still an open about the best potential placing of the 'uint16_t ipv6_exthdr'. I think it makes sense to wait until the placing of `ipv6_exthdr` has been finalized before sending a patch with the necessary adjustments to AVX512 MFEX. Does this make sense? Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v8 4/4] tests/mfex: Improve pcap script for mfex tests.
> -Original Message- > From: Amber, Kumar > Sent: Friday 1 April 2022 12:24 > To: ovs-dev@openvswitch.org > Cc: Stokes, Ian ; echau...@redhat.com; Ferriter, Cian > ; > f...@sysclose.org; Van Haaren, Harry ; Amber, > Kumar > Subject: [PATCH v8 4/4] tests/mfex: Improve pcap script for mfex tests. > > The mfex pcap generation script is improved for varied length > traffic and also removes the hard coded mfex_pcap and instead uses > the script itself to generate complex traffic patterns for testing. > > Signed-off-by: Kumar Amber > > --- > v8: > - Reduce IO writes. > -- > --- Thanks for improving the mfex_fuzzy.py execution time Amber. I can confirm that it speeds up the unit test execution time. All tests are still passing: 6: OVS-DPDK - MFEX Autovalidator ok 7: OVS-DPDK - MFEX Autovalidator Fuzzy ok 8: OVS-DPDK - MFEX Configuration ok I'll use the same timing method as last time: Before patch: ~/ovs# time make check-dpdk TESTSUITEFLAGS='-k MFEX -d' real0m43.936s user0m26.577s sys 0m0.881s 43 seconds After patch: ~/ovs# time make check-dpdk TESTSUITEFLAGS='-k MFEX -d' real1m3.049s user0m45.487s sys 0m1.297s 1 minute and 3 seconds I think the time increase is acceptable since we are getting rid of a binary PCAP file in the process. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v8 3/4] dpif-netdev/mfex: Avoid hashing when opt mfex called.
> -Original Message- > From: Amber, Kumar > Sent: Friday 1 April 2022 12:24 > To: ovs-dev@openvswitch.org > Cc: Stokes, Ian ; echau...@redhat.com; Ferriter, Cian > ; > f...@sysclose.org; Van Haaren, Harry ; Amber, > Kumar > Subject: [PATCH v8 3/4] dpif-netdev/mfex: Avoid hashing when opt mfex called. > > This patch avoids calculating the software hash of the packet again > if the optimized miniflow-extract hit. In cases of scalar miniflow > extract, the normal hashing calculation is performed. > > Signed-off-by: Kumar Amber > --- Thanks for improving the commit message based on my suggestion Amber. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v8 2/4] dpif-netdev/mfex: Add packet hash check to autovalidator.
> -Original Message-
> From: Amber, Kumar
> Sent: Friday 1 April 2022 12:24
> To: ovs-dev@openvswitch.org
> Cc: Stokes, Ian ; echau...@redhat.com; Ferriter, Cian
> ;
> f...@sysclose.org; Van Haaren, Harry ; Amber,
> Kumar
> Subject: [PATCH v8 2/4] dpif-netdev/mfex: Add packet hash check to
> autovalidator.
>
> This patch adds the scalar hash calls to the autovalidator.
> It also adds checks for comparing the scalar hash against
> the profile based hash calculated as part of AVX512 MFEX implementations.
>
> The per profile AVX512 optimized hash was added to the autovalidator
> in the last commit. The autovalidator was already calling that code,
> we just add the checks and scalar hashing in this commit.
>
> Signed-off-by: Kumar Amber
>
> ---
> v8:
> - Fix hash validation.
> ---
>
> -/* Preserve packet correctness by storing back the good offsets in
> - * packets back. */
> +/* Reset all packet values. */
> DP_PACKET_BATCH_FOR_EACH (i, packet, packets) {
> -packet->l2_5_ofs = good_l2_5_ofs[i];
> -packet->l3_ofs = good_l3_ofs[i];
> -packet->l4_ofs = good_l4_ofs[i];
> -packet->l2_pad_size = good_l2_pad_size[i];
> +dp_packet_reset_offsets(packet);
> +*dp_packet_ol_flags_ptr(packet) &= ~DP_PACKET_OL_RSS_HASH;
> }
Thanks for addressing my comments Amber. I think this change makes sense. Since
we always return 0 from the autovalidator, we want to reset 'struct dp_packet'
values such as offsets and offload flags.
After intentionally introducing errors into the ipv4 profile specific hashing
code added in patch 1/4 of this series, I can verify that the autovalidator
correctly identifies the wrongly calculated hash values.
Acked-by: Cian Ferriter
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v8 1/4] dpif-netdev/mfex: Add ipv4 profile based hashing.
> -Original Message- > From: Amber, Kumar > Sent: Friday 1 April 2022 12:24 > To: ovs-dev@openvswitch.org > Cc: Stokes, Ian ; echau...@redhat.com; Ferriter, Cian > ; > f...@sysclose.org; Van Haaren, Harry ; Amber, > Kumar > Subject: [PATCH v8 1/4] dpif-netdev/mfex: Add ipv4 profile based hashing. > > For packets which don't already have a hash calculated, > miniflow_hash_5tuple() calculates the hash of a packet > using the previously built miniflow. > > This commit adds IPv4 profile specific hashing which > uses fixed offsets into the packet to improve hashing > performance. > > Signed-off-by: Kumar Amber > Signed-off-by: Harry van Haaren > Co-authored-by: Harry van Haaren > > --- > v8: > - Fix comments from cian. > v4: > - Use pre-defined hash length values. > v3: > - Fix check-patch sign-offs. > --- Thanks for addressing my comments Amber. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 4/4] tests/mfex: Improve pcap script for mfex tests.
> -Original Message- > From: Amber, Kumar > Sent: Wednesday 23 March 2022 12:40 > To: ovs-dev@openvswitch.org > Cc: f...@sysclose.org; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; echau...@redhat.com; Van Haaren, Harry > ; Amber, > Kumar > Subject: [PATCH v7 4/4] tests/mfex: Improve pcap script for mfex tests. > > The mfex pcap generation script is improved for varied length > traffic and also removes the hard coded mfex_pcap and instead uses > the script itself to generate complex traffic patterns for testing. > > Signed-off-by: Kumar Amber Hi Amber, I like idea of removing the binary PCAP file and instead generating the PCAPs on the fly. That way we can see what packets are being generated from the mfex_fuzzy.py script. I've run the MFEX unit tests before and after the patch series. They are all passing: 6: OVS-DPDK - MFEX Autovalidator ok 7: OVS-DPDK - MFEX Autovalidator Fuzzy ok 8: OVS-DPDK - MFEX Configuration ok But I see increased MFEX unit test execution time: Before patch: ~/ovs# time make check-dpdk TESTSUITEFLAGS='-k MFEX -d' real0m43.936s user0m26.577s sys 0m0.881s 43 seconds After patch: ~/ovs# time make check-dpdk TESTSUITEFLAGS='-k MFEX -d' real4m24.100s user4m2.374s sys 0m3.014s 4 minutes and 24 seconds Its seems that a lot of time is spent calling mfex_fuzzy.py. Have you seen this behaviour too? I wonder if there's any way to improve the execution of mfex_fuzzy.py. It looks like we write each packet to the file 1 at a time. Maybe we can write packet to file once at the end of the script? Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 2/4] dpif-netdev/mfex: Add packet hash check to autovalidator.
> -Original Message-
> From: Amber, Kumar
> Sent: Wednesday 23 March 2022 12:40
> To: ovs-dev@openvswitch.org
> Cc: f...@sysclose.org; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; echau...@redhat.com; Van Haaren, Harry
> ; Amber,
> Kumar
> Subject: [PATCH v7 2/4] dpif-netdev/mfex: Add packet hash check to
> autovalidator.
>
> This patch adds the per profile AVX512 opt hashing to autovalidator
> for validating the optimized hash values against the scalar hash.
Hi Amber,
Is it more correct to say that this patch adds the scalar hash calls to the
autovalidator. It also adds checks for comparing the scalar hash against the
profile based hash calculated as part of AVX512 MFEX implementations.
The per profile AVX512 optimized hash was added to the autovalidator in the
last commit. The autovalidator was already calling that code, we just add the
checks and scalar hashing in this commit.
>
> Signed-off-by: Kumar Amber
> ---
> lib/dpif-netdev-private-extract.c | 12
> 1 file changed, 12 insertions(+)
>
> diff --git a/lib/dpif-netdev-private-extract.c
> b/lib/dpif-netdev-private-extract.c
> index 4b2f12015..63826b643 100644
> --- a/lib/dpif-netdev-private-extract.c
> +++ b/lib/dpif-netdev-private-extract.c
> @@ -252,6 +252,9 @@ dpif_miniflow_extract_autovalidator(struct
> dp_packet_batch *packets,
> DP_PACKET_BATCH_FOR_EACH (i, packet, packets) {
> pkt_metadata_init(>md, in_port);
> miniflow_extract(packet, [i].mf);
> +keys[i].len = netdev_flow_key_size(miniflow_n_values([i].mf));
> +keys[i].hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
> +[i].mf);
The call to dpif_netdev_packet_get_rss_hash_orig_pkt() updates the ol_flags for
that dp_packet to state that there is now a hash (DP_PACKET_OL_RSS_HASH). To
ensure that the AVX512 hashing is always executed, we have to remove this
'packet has a hash' flag after calling so other implementations can call it
again.
Thanks,
Cian
>
> /* Store known good metadata to compare with optimized metadata. */
> good_l2_5_ofs[i] = packet->l2_5_ofs;
> @@ -335,6 +338,15 @@ dpif_miniflow_extract_autovalidator(struct
> dp_packet_batch *packets,
> failed = 1;
> }
>
> +/* Check hashes are equal. */
> +if ((keys[i].hash != test_keys[i].hash) ||
> +(keys[i].len != test_keys[i].len)) {
> +ds_put_format(_msg, "Good hash: %d len: %d\tTest hash:%d"
> + " len:%d\n", keys[i].hash, keys[i].len,
> + test_keys[i].hash, test_keys[i].len);
> +failed = 1;
> +}
> +
> if (failed) {
> VLOG_ERR("Autovalidation for %s failed in pkt %d,"
> " disabling.", mfex_impls[j].name, i);
> --
> 2.25.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 3/4] dpif-netdev/mfex: Avoid hashing when opt mfex called.
> -Original Message-
> From: Amber, Kumar
> Sent: Wednesday 23 March 2022 12:40
> To: ovs-dev@openvswitch.org
> Cc: f...@sysclose.org; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; echau...@redhat.com; Van Haaren, Harry
> ; Amber,
> Kumar
> Subject: [PATCH v7 3/4] dpif-netdev/mfex: Avoid hashing when opt mfex called.
>
> This patch avoids calculating the software hash of the packet again
> if the optimized miniflow-extract hit and has already calculated the
> packet hash. In cases of scalar miniflow extract, the normal hashing
> calculation is performed.
Hi Amber,
This makes sense as long as all MFEX implementations will provide the hash.
Above you say that the SW hash is only calculated if the optimized MFEX hits
AND has already calculated the packet hash, but in the below code, we only
check if the optimized MFEX impl has hit, right?
I think this is OK, since the goal for any future AVX512 MFEX impls would be to
also use profile specific hashing and also provide the SW hash when it's not
present in the dp_packet. Maybe just take out the "and has already calculated
the packet hash" part of the commit message.
Thanks,
Cian
>
> Signed-off-by: Kumar Amber
> ---
> lib/dpif-netdev-avx512.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c
> index b7131ba3f..c68b79f6b 100644
> --- a/lib/dpif-netdev-avx512.c
> +++ b/lib/dpif-netdev-avx512.c
> @@ -212,15 +212,15 @@ dp_netdev_input_outer_avx512(struct
> dp_netdev_pmd_thread *pmd,
> if (!mfex_hit) {
> /* Do a scalar miniflow extract into keys. */
> miniflow_extract(packet, >mf);
> +key->len = netdev_flow_key_size(miniflow_n_values(>mf));
> +key->hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
> + >mf);
> }
>
> /* Cache TCP and byte values for all packets. */
> pkt_meta[i].bytes = dp_packet_size(packet);
> pkt_meta[i].tcp_flags = miniflow_get_tcp_flags(>mf);
>
> -key->len = netdev_flow_key_size(miniflow_n_values(>mf));
> -key->hash = dpif_netdev_packet_get_rss_hash_orig_pkt(packet,
> >mf);
> -
> if (emc_enabled) {
> f = emc_lookup(>emc_cache, key);
>
> --
> 2.25.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 1/4] dpif-netdev/mfex: Add ipv4 profile based hashing.
> -Original Message-
> From: Amber, Kumar
> Sent: Wednesday 23 March 2022 12:40
> To: ovs-dev@openvswitch.org
> Cc: f...@sysclose.org; i.maxim...@ovn.org; Ferriter, Cian
> ; Stokes, Ian
> ; echau...@redhat.com; Van Haaren, Harry
> ; Amber,
> Kumar
> Subject: [PATCH v7 1/4] dpif-netdev/mfex: Add ipv4 profile based hashing.
>
> This commit adds IPv4 profile specific hashing which
> uses fixed offsets into the packet to improve hashing
> performance.
Hi Amber,
The scalar code retrieves the packet values from the already built miniflow of
the packet to calculate the hash, correct?
That difference helps me understand what this optimization does. So how about
adding this line:
For packets which don't already have a hash calculated, miniflow_hash_5tuple()
calculates the hash of a packet using the previously built miniflow.
This commit adds IPv4 profile specific hashing which
uses fixed offsets into the packet to improve hashing
performance.
>
> Signed-off-by: Kumar Amber
> Signed-off-by: Harry van Haaren
> Co-authored-by: Harry van Haaren
>
> ---
> v4:
> - Use pre-defined hash length values.
> v3:
> - Fix check-patch sign-offs.
> ---
> ---
> NEWS | 2 +
> lib/dpif-netdev-extract-avx512.c | 65
> 2 files changed, 67 insertions(+)
>
> diff --git a/NEWS b/NEWS
> index df633e8e2..2090498bb 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -4,6 +4,8 @@ Post-v2.17.0
> * 'relay' service model now supports transaction history, i.e. honors
> the
> 'last-txn-id' field in 'monitor_cond_since' requests from clients.
>
> + - Userspace datapath:
> + * Add IPv4 profile based 5tuple hashing optimizations.
>
I don't see a newline between any of the NEWS items in a release. I think your
NEWS item should move up a line with 2 newlines after it like others.
> v2.17.0 - xx xxx
> -
> diff --git a/lib/dpif-netdev-extract-avx512.c
> b/lib/dpif-netdev-extract-avx512.c
> index c1c1fefb6..e0db86629 100644
> --- a/lib/dpif-netdev-extract-avx512.c
> +++ b/lib/dpif-netdev-extract-avx512.c
> @@ -278,6 +278,10 @@ struct mfex_profile {
> uint64_t mf_bits[FLOWMAP_UNITS];
> uint16_t dp_pkt_offs[4];
> uint16_t dp_pkt_min_size;
> +
> +/* Constant data offsets for Hashing. */
Hashing -> hashing
> +uint8_t hash_pkt_offs[6];
Why are there 6 elements in this array? From HASH_IPV4 and HASH_DT1Q_IPV4
below, I can see that only 4 are used. Can we make this a 4 element array?
> +uint32_t hash_len;
> };
>
> /* Ensure dp_pkt_offs[4] is the correct size as in struct dp_packet. */
> @@ -327,6 +331,13 @@ enum MFEX_PROFILES {
> PROFILE_COUNT,
> };
>
> +/* Packet offsets for 5 tuple Hash function. */
Hash -> hash
> +#define HASH_IPV4 \
> +26, 30, 23, 34, 0, 0
> +
> +#define HASH_DT1Q_IPV4 \
> +30, 34, 27, 38, 0, 0
> +
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7 0/4] IPv4 Hashing AVX512 Optimizations
> -Original Message- > From: Amber, Kumar > Sent: Wednesday 23 March 2022 12:40 > To: ovs-dev@openvswitch.org > Cc: f...@sysclose.org; i.maxim...@ovn.org; Ferriter, Cian > ; Stokes, Ian > ; echau...@redhat.com; Van Haaren, Harry > ; Amber, > Kumar > Subject: [PATCH v7 0/4] IPv4 Hashing AVX512 Optimizations > > Hashing Optimization are also included which can further > improve performance by approximately 10%. > Hi Amber, I'm taking a look at this series. I can see a 9% improvement in my performance testing. I'm using the DPDK PCAP PMD with the infinite_rx=1 which means OVS has to calculate the 5tuple hash. I can also see in perf top that before the patchset, miniflow_hash_5tuple() takes ~8% of cycles, and afterwards it's not used and mfex_avx512_ip_udp() takes more cycles since the newly introduced mfex_5tuple_hash_ipv4() function is inlined under it. I'll leave more comments on the actual patches in the series. Thanks, Cian > --- > v7: > - Split Ipv4 Hahsing to separate Patchset > v6: > - Reorder Patches in the Patchset. > v5: > - Add Ipv6 and TCP packet length checks. > v4: > - rebase to master. > - use static key lenghts for different packet types. > v3: > - rebase to master. > v2: > - fix the CI build. > - fix check-patch for co-author. > --- > > Kumar Amber (4): > dpif-netdev/mfex: Add ipv4 profile based hashing. > dpif-netdev/mfex: Add packet hash check to autovalidator. > dpif-netdev/mfex: Avoid hashing when opt mfex called. > tests/mfex: Improve pcap script for mfex tests. > > NEWS | 2 + > lib/dpif-netdev-avx512.c | 6 +-- > lib/dpif-netdev-extract-avx512.c | 65 ++ > lib/dpif-netdev-private-extract.c | 12 ++ > tests/automake.mk | 1 - > tests/mfex_fuzzy.py | 55 - > tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes > tests/system-dpdk.at | 23 +++ > 8 files changed, 134 insertions(+), 30 deletions(-) > delete mode 100644 tests/pcap/mfex_test.pcap > > -- > 2.25.1 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v6 6/6] tests/mfex: Improve pcap script for complex testing traffic
Hi Amber, I am seeing a failure in one of the unit tests. Test: 7: OVS-DPDK - MFEX Autovalidator Fuzzy FAILED (system-dpdk.at:289) More details about the failure: 2022-03-16T16:07:05.234Z|2|dpif_netdev_extract(pmd-c21/id:101)|ERR|Autovalidation for avx512_dot1q_ipv6_tcp failed in pkt 24, disabling. 2022-03-16T16:07:05.235Z|3|dpif_netdev_extract(pmd-c21/id:101)|ERR|Autovalidation failure details: MFEX autovalidator pkt 24 Autovalidation blocks failed Good hex: 00 00 00 00 02 00 00 00-00 00 00 00 00 00 00 00 0010 e0 ab ce d7 79 62 32 23-2e 12 11 fb 86 dd 00 00 0020 81 00 3d f3 00 00 00 00-b5 c5 84 90 33 93 17 1d 0030 24 85 28 05 e3 ce 7a 7d-9b 89 00 4c 4a 1c b9 70 0040 5a 30 80 14 75 97 a7 ce-00 0a 5e 3c 00 a5 bc 06 0050 00 00 00 00 04 02 00 00-60 70 d0 2f 00 00 00 00 Test hex: 00 00 00 00 02 00 00 00-00 00 00 00 00 00 00 00 0010 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 0020 00 00 3d 00 00 00 00 00-00 00 00 00 00 00 00 00 0030 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 0040 00 00 00 00 00 00 00 00-00 0a 5e 3c 00 a5 bc 06 0050 00 00 00 00 04 02 00 00-60 70 d0 2f 00 00 00 00 Autovalidation packet offsets failed Good offsets: l2_pad_size: 0, l2_5_ofs: 65535, l3_ofs: 18, l4_ofs: 58 Test offsets: l2_pad_size: 40, l2_5_ofs: 65535, l3_ofs: 18, l4_ofs: 58 Thanks, Cian > -Original Message- > From: Amber, Kumar > Sent: Tuesday 15 March 2022 04:12 > To: ovs-dev@openvswitch.org > Cc: i.maxim...@ovn.org; Ferriter, Cian ; Stokes, Ian > ; > f...@sysclose.org; echau...@redhat.com; Van Haaren, Harry > ; Amber, Kumar > > Subject: [PATCH v6 6/6] tests/mfex: Improve pcap script for complex testing > traffic > > The mfex pcap generation script is improved for varied length > traffic and also removes the hard coded mfex_pcap instead uses > the script to generate complex traffic for testing. > > Signed-off-by: Kumar Amber > --- > tests/automake.mk | 1 - > tests/mfex_fuzzy.py | 57 +- > tests/pcap/mfex_test.pcap | Bin 416 -> 0 bytes > tests/system-dpdk.at | 23 ++- > 4 files changed, 54 insertions(+), 27 deletions(-) > delete mode 100644 tests/pcap/mfex_test.pcap > > diff --git a/tests/automake.mk b/tests/automake.mk > index 8a9151f81..507da2ee8 100644 > --- a/tests/automake.mk > +++ b/tests/automake.mk > @@ -145,7 +145,6 @@ $(srcdir)/tests/fuzz-regression-list.at: tests/automake.mk > > EXTRA_DIST += $(MFEX_AUTOVALIDATOR_TESTS) > MFEX_AUTOVALIDATOR_TESTS = \ > - tests/pcap/mfex_test.pcap \ > tests/mfex_fuzzy.py > > OVSDB_CLUSTER_TESTSUITE_AT = \ > diff --git a/tests/mfex_fuzzy.py b/tests/mfex_fuzzy.py > index 3efe1152d..d2314d003 100755 > --- a/tests/mfex_fuzzy.py > +++ b/tests/mfex_fuzzy.py > @@ -3,30 +3,49 @@ > import sys > > from scapy.all import RandMAC, RandIP, PcapWriter, RandIP6, RandShort, fuzz > -from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP > +from scapy.all import IPv6, Dot1Q, IP, Ether, UDP, TCP, random > > path = str(sys.argv[1]) + "/pcap/fuzzy.pcap" > +size = int(sys.argv[2]) > +traffic_opt = str(sys.argv[3]) > + > pktdump = PcapWriter(path, append=False, sync=True) > > -for i in range(0, 2000): > +for i in range(0, size): > > -# Generate random protocol bases, use a fuzz() over the combined packet > -# for full fuzzing. > eth = Ether(src=RandMAC(), dst=RandMAC()) > vlan = Dot1Q() > -ipv4 = IP(src=RandIP(), dst=RandIP()) > -ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) > udp = UDP(dport=RandShort(), sport=RandShort()) > -tcp = TCP(dport=RandShort(), sport=RandShort()) > - > -# IPv4 packets with fuzzing > -pktdump.write(fuzz(eth / ipv4 / udp)) > -pktdump.write(fuzz(eth / ipv4 / tcp)) > -pktdump.write(fuzz(eth / vlan / ipv4 / udp)) > -pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) > - > -# IPv6 packets with fuzzing > -pktdump.write(fuzz(eth / ipv6 / udp)) > -pktdump.write(fuzz(eth / ipv6 / tcp)) > -pktdump.write(fuzz(eth / vlan / ipv6 / udp)) > -pktdump.write(fuzz(eth / vlan / ipv6 / tcp)) > +tcp = TCP(dport=RandShort(), sport=RandShort(), flags='S', dataofs=(0, > 20)) > + > +if traffic_opt == "fuzzy": > + > +ipv4 = IP(src=RandIP(), dst=RandIP()) > +ipv6 = IPv6(src=RandIP6(), dst=RandIP6()) > +# IPv4 packets with fuzzing > +pktdump.write(fuzz(eth / ipv4 / udp)) > +pktdump.write(fuzz(eth / ipv4 / tcp)) > +pktdump.write(fuzz(eth / vlan / ipv4 / udp)) > +pktdump.write(fuzz(eth / vlan / ipv4 / tcp)) > + > +# IPv6 packets with fuzzing > +pktdump.write
Re: [ovs-dev] [PATCH v6 1/6] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
Hi Amber,
As part of reviewing this patch series, I wanted to first run/test the code to
familiarize myself with it.
As part of this, I ran in to some problems where the IPv6 packets I use aren't
parsed by the AVX512 implementations unless they are 106B or larger.
I had a look at why this is and can see that some length checks are failing in
my case. Please see my points below about these length checks.
Hopefully my understanding is correct and makes sense. I wanted to reply with
this feedback now before looking at the series in more detail.
Thanks for the patches.
Cian
> -Original Message-
> From: Amber, Kumar
> Sent: Tuesday 15 March 2022 04:12
> To: ovs-dev@openvswitch.org
> Cc: i.maxim...@ovn.org; Ferriter, Cian ; Stokes, Ian
> ;
> f...@sysclose.org; echau...@redhat.com; Van Haaren, Harry
> ; Amber, Kumar
>
> Subject: [PATCH v6 1/6] dpif-netdev/mfex: Add AVX512 ipv6 traffic profiles
>
> Add AVX512 Ipv6 optimized profile for vlan/IPv6/UDP and
> vlan/IPv6/TCP, IPv6/UDP and IPv6/TCP.
>
> MFEX autovalidaton test-case already has the IPv6 support for
> validating against the scalar mfex.
>
> Signed-off-by: Kumar Amber
> Signed-off-by: Harry van Haaren
> Co-authored-by: Harry van Haaren
>
> +
> +/* IPv6 specific helper functions, for calculating offsets/lengths. */
> +static int
> +mfex_ipv6_set_l2_pad_size(struct dp_packet *pkt,
> + struct ovs_16aligned_ip6_hdr *nh,
> + uint32_t payload_size_ipv6,
> + uint32_t next_hdr_size)
> +{
> +/* Handle dynamic l2_pad_size. */
> +uint16_t p_len = ntohs(nh->plen);
> +
> +/* Error if IP total length is greater than remaining packet size. */
> +bool err_ipv6_len_too_high = p_len >= payload_size_ipv6;
I think this above check is wrong. p_len is the IPv6 payload length. For
example, for an ETH/IPv6/UDP/Payload(len=10B), the p_len will be 18B (8B for
UDP + 10B for payload).
payload_size_ipv6 is like the len_from_ipv4 or size_from_ipv4 field used below
in the mfex_ipv4_set_l2_pad_size() function. It's the actual packet size minus
sizeof(struct eth_header).
I think payload_size_ipv6 should be called len_from_ipv6, so it won't be
confused with the actual payload length IPv6 field.
What are we trying to accomplish with the below check? I think we want to check
if the p_len value in the packet is too big, right?
p_len is too big when:
p_len > payload_size_ipv6 - IPv6_HEADER_LEN
or
plen + IPV6_HEADER_LEN > payload_size_ipv6
This 2nd statement above is the same as what's in the ipv6_sanity_check()
function which carries out similar checks for the scalar MFEX.
Is my understanding correct here?
> +
> +/* Error if IP total length is less than the size of the IP header
> + * itself, and the size of the next-protocol this profile matches on.
> + */
> +bool err_ipv6_len_too_low = (IPV6_HEADER_LEN + next_hdr_size) > p_len;
I think this is wrong too. p_len doesn't include the actual IPv6 header in its
length (see my example above).
> +
> +/* Ensure the l2 pad size will not overflow. */
> +bool err_len_u16_overflow = (payload_size_ipv6 - p_len) > UINT16_MAX;
I think this overflow check is wrong, I think we should follow what's done in
ipv6_sanity_check() for overflow (the last check in the function).
> +
> +if (OVS_UNLIKELY(err_ipv6_len_too_high || err_ipv6_len_too_low ||
> + err_len_u16_overflow)) {
> +return -1;
> +}
> +dp_packet_set_l2_pad_size(pkt, payload_size_ipv6 - p_len);
> +return 0;
> +}
>
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2 4/5] acinclude: Add seperate check for AVX512VBMI ISA.
> -Original Message-
> From: Van Haaren, Harry
> Sent: Wednesday 2 March 2022 14:36
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Subject: RE: [PATCH v2 4/5] acinclude: Add seperate check for AVX512VBMI ISA.
>
> > -Original Message-----
> > From: Ferriter, Cian
> > Sent: Wednesday, March 2, 2022 12:18 PM
> > To: ovs-dev@openvswitch.org
> > Cc: Ferriter, Cian ; Van Haaren, Harry
> >
> > Subject: [PATCH v2 4/5] acinclude: Add seperate check for AVX512VBMI ISA.
> >
> > Only use the "avx512vbmi" compiler target when it is actually supported
> > by the compiler.
> >
> > The order of mfex_impls and the 'dpif_miniflow_extract_impl_idx' enum
> > have to be changed to keep the start index and size of the impl list
> > correct in both VBMI and non VBMI cases.
> >
> > Signed-off-by: Cian Ferriter
>
> Thanks for working on this.
>
>
>
Thanks for the review Harry.
> > return _mm512_maskz_permutexvar_epi8(kmask, idx, a);
> > @@ -481,7 +483,7 @@ mfex_avx512_process(struct dp_packet_batch *packets,
> > odp_port_t in_port,
> > void *pmd_handle OVS_UNUSED,
> > const enum MFEX_PROFILES profile_id,
> > -const uint32_t use_vbmi)
> > +const uint32_t use_vbmi OVS_UNUSED)
> > {
> > uint32_t hitmask = 0;
> > struct dp_packet *packet;
> > @@ -544,7 +546,11 @@ mfex_avx512_process(struct dp_packet_batch *packets,
> > */
> > __m512i v512_zeros = _mm512_setzero_si512();
> > __m512i v_blk0;
> > +#if HAVE_AVX512VBMI
> > if (__builtin_constant_p(use_vbmi) && use_vbmi) {
> > +#else
> > +if (0) {
> > +#endif
> > v_blk0 = _mm512_maskz_permutexvar_epi8_wrap(k_shuf, v_shuf,
> > v_pkt0);
> > } else {
>
> Cian previously wrote (on reply to this patch,
> https://patchwork.ozlabs.org/project/openvswitch/patch/20220302121819.1261928-5-
> cian.ferri...@intel.com/#2850616)
> > The above solution works when the has not support for VBMI but I'm not
> > happy with the solution. It
> makes the code look messier IMO. I'm looking for suggestions on this.
> > I'm thinking to hide all this complexity in a wrapper function which would
> > always have the non VBMI
> permutex2var option and would have the VBMI permutexvar option where
> possible. The run time selecting
> between the two permute impls would remain unchanged. Does this sound good?
>
> There's an equivalent syntax version which might be better or worse. All in
> all, I think this does
> exactly what we want - and I cannot see drastically simpler version?
>
> if (
> #if HAVE_AVX512VBMI
> __builtin_constant_p(use_vbmi) && use_vbmi
> #else
> 0
> #endif
> )
>
>
>
>
Agreed. I'll keep what I have in that case. Thanks for giving your opinion here.
> > diff --git a/lib/dpif-netdev-private-extract.c
> > b/lib/dpif-netdev-private-extract.c
> > index 43b8b824e..ea2b03e5c 100644
> > --- a/lib/dpif-netdev-private-extract.c
> > +++ b/lib/dpif-netdev-private-extract.c
> > @@ -56,45 +56,46 @@ static struct dpif_miniflow_extract_impl mfex_impls[] =
> > {
> > /* Compile in implementations only if the compiler ISA checks pass. */
> > #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD &&
> > HAVE_AVX512BW_DQ \
> > && __SSE4_2__)
> > -[MFEX_IMPL_VBMI_IPv4_UDP] = {
> > -.probe = mfex_avx512_vbmi_probe,
> > -.extract_func = mfex_avx512_vbmi_ip_udp,
> > -.name = "avx512_vbmi_ipv4_udp", },
> > -
> > [MFEX_IMPL_IPv4_UDP] = {
> > .probe = mfex_avx512_probe,
> > .extract_func = mfex_avx512_ip_udp,
> > .name = "avx512_ipv4_udp", },
> >
> > -[MFEX_IMPL_VBMI_IPv4_TCP] = {
> > -.probe = mfex_avx512_vbmi_probe,
> > -.extract_func = mfex_avx512_vbmi_ip_tcp,
> > -.name = "avx512_vbmi_ipv4_tcp", },
> > -
> > [MFEX_IMPL_IPv4_TCP] = {
> > .probe = mfex_avx512_probe,
> > .extract_func = mfex_avx512_ip_tcp,
> > .name = "avx512_ipv4_tcp", },
> >
> > -[MFEX_IMPL_VBMI_DOT1Q_IPv4_UDP] = {
> > -.probe = mfex_avx512_vbmi_probe,
> > -.extract_func = mfex_avx512_vbmi_dot1q_ip_udp,
> > -.name = "avx512_vbmi_dot1q_ipv4_udp", },
Re: [ovs-dev] [PATCH v2] system-dpdk: Fix mfex autovalidator tests.
> -Original Message-
> From: Amber, Kumar
> Sent: Tuesday 1 March 2022 15:08
> To: ovs-dev@openvswitch.org
> Cc: Ferriter, Cian ; echau...@redhat.com; Stokes,
> Ian ;
> Van Haaren, Harry ; Amber, Kumar
>
> Subject: [PATCH v2] system-dpdk: Fix mfex autovalidator tests.
>
> AVX512 DPIF must be active in order for the MFEX AutoValidator to be executed.
> If the DPIF-AVX512 is not available, the unit test is skipped, as the
> scalar DPIF does not use the MFEX function-pointer based optimizations.
>
> Fixes: 50be6715c0 ("test/sytem-dpdk: Add unit test for mfex autovalidator")
> Suggested-by: Cian Ferriter
> Signed-off-by: Kumar Amber
Hi Amber,
Thanks for addressing the changes.
This patch LGTM. I have also tested that the unit tests are being run by
introducing an error in the MFEX C code and running the unit tests. The errors
are caught and the MFEX autovalidator tests fail.
Acked-by: Cian Ferriter
More information about my testing:
Before introducing errors in the C code the tests pass:
~/ovs# make check-dpdk TESTSUITEFLAGS='-k MFEX -d'
OVS-DPDK unit tests
6: OVS-DPDK - MFEX Autovalidator ok
7: OVS-DPDK - MFEX Autovalidator Fuzzy ok
8: OVS-DPDK - MFEX Configuration ok
I introduce the following error:
lib/dpif-netdev-extract-avx512.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-avx512.c
index c1c1fefb6..f31d64889 100644
--- a/lib/dpif-netdev-extract-avx512.c
+++ b/lib/dpif-netdev-extract-avx512.c
@@ -172,7 +172,7 @@ _mm512_maskz_permutexvar_epi8_wrap(__mmask64 kmask, __m512i
idx, __m512i a)
#define NU 0
#define PATTERN_IPV4_UDP_SHUFFLE \
- 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, NU, NU, /* Ether */ \
+ 26, 27, 28, 29, 30, 31, 32, 33, NU, NU, NU, NU, 20, 15, 22, 23, /* IPv4 */ \
26, 27, 28, 29, 30, 31, 32, 33, NU, NU, NU, NU, 20, 15, 22, 23, /* IPv4 */ \
34, 35, 36, 37, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, /* UDP */ \
NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, NU, /* Unused. */
And the tests fail as expected:
~/ovs# make check-dpdk TESTSUITEFLAGS='-k MFEX -d'
OVS-DPDK unit tests
6: OVS-DPDK - MFEX Autovalidator FAILED (system-dpdk.at:252)
7: OVS-DPDK - MFEX Autovalidator Fuzzy FAILED (system-dpdk.at:284)
8: OVS-DPDK - MFEX Configuration ok
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2 4/5] acinclude: Add seperate check for AVX512VBMI ISA.
> -Original Message-
> From: Ferriter, Cian
> Sent: Wednesday 2 March 2022 12:18
> To: ovs-dev@openvswitch.org
> Cc: Ferriter, Cian ; Van Haaren, Harry
>
> Subject: [PATCH v2 4/5] acinclude: Add seperate check for AVX512VBMI ISA.
>
> Only use the "avx512vbmi" compiler target when it is actually supported
> by the compiler.
>
> The order of mfex_impls and the 'dpif_miniflow_extract_impl_idx' enum
> have to be changed to keep the start index and size of the impl list
> correct in both VBMI and non VBMI cases.
>
> Signed-off-by: Cian Ferriter
>
> ---
>
> v2:
> * Don't register vbmi specialized mfex impls unless VBMI is actually
> available.
> * This required some re-ordering of the mfex impl lists.
> ---
> @@ -544,7 +546,11 @@ mfex_avx512_process(struct dp_packet_batch *packets,
> */
> __m512i v512_zeros = _mm512_setzero_si512();
> __m512i v_blk0;
> +#if HAVE_AVX512VBMI
> if (__builtin_constant_p(use_vbmi) && use_vbmi) {
> +#else
> +if (0) {
> +#endif
> v_blk0 = _mm512_maskz_permutexvar_epi8_wrap(k_shuf, v_shuf,
> v_pkt0);
> } else {
The above solution works when the has not support for VBMI but I'm not happy
with the solution. It makes the code look messier IMO. I'm looking for
suggestions on this.
I'm thinking to hide all this complexity in a wrapper function which would
always have the non VBMI permutex2var option and would have the VBMI
permutexvar option where possible. The run time selecting between the two
permute impls would remain unchanged. Does this sound good?
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v1] system-dpdk.at: Improve mfex test to include dpif avx512.
Hi Amber, Thanks for the patch. I've left some responses below. Cian > -Original Message- > From: Amber, Kumar > Sent: Monday 31 January 2022 10:52 > To: ovs-dev@openvswitch.org > Cc: ktray...@redhat.com; i.maxim...@ovn.org; Stokes, Ian > ; Van Haaren, Harry > ; Amber, Kumar ; Ferriter, > Cian > > Subject: [PATCH v1] system-dpdk.at: Improve mfex test to include dpif avx512. For the commit title, how about: system-dpdk: Fix mfex autovalidator tests. And adding a "Fixes:" tag? > > AVX512 DPIF must be active in order for the MFEX AutoValidator to be executed. > If the DPIF-AVX512 is not available, the unit test is skipped, as the > scalar DPIF does not use the MFEX function-pointer based optimizations. > > Suggested-by: Cian Ferriter > Signed-off-by: Kumar Amber > --- > tests/system-dpdk.at | 8 > 1 file changed, 8 insertions(+) > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > index 9384cf7f0..3d89384a3 100644 > --- a/tests/system-dpdk.at > +++ b/tests/system-dpdk.at > @@ -237,6 +237,10 @@ AT_CHECK([ovs-vsctl show], [], [stdout]) > AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > ]) > > +AT_SKIP_IF([! ovs-appctl dpif-netdev/dpif-impl-set dpif_avx512], [], [dnl > +DPIF implementation set to dpif_avx512. > +]) > + For the AT_SKIP_IF statements, only one argument should be given (one set of []). Please search "AT_SKIP_IF" on https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Writing-Testsuites.html for more information. Also, having a look at how AT_SKIP_IF is supposed to be used and how it is actually used in OVS, I don't see anywhere that a command to change an OVS setting is run inside an AT_SKIP_IF. I think we should just use an AT_CHECK like I have below because we will only get this far if the above AT_SKIP_IF passes which means CPU ISA is available for AVX512 MFEX and DPIF impls. AT_CHECK([ovs-appctl dpif-netdev/dpif-impl-set dpif_avx512], [0], [dnl DPIF implementation set to dpif_avx512. ]) What do you think? > AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > Miniflow extract implementation set to autovalidator. > ]) > @@ -265,6 +269,10 @@ AT_CHECK([ovs-vsctl show], [], [stdout]) > AT_SKIP_IF([! ovs-appctl dpif-netdev/miniflow-parser-get | sed 1,4d | grep > "True"], [], [dnl > ]) > > +AT_SKIP_IF([! ovs-appctl dpif-netdev/dpif-impl-set dpif_avx512], [], [dnl > +DPIF implementation set to dpif_avx512. > +]) > + The above points apply for this AT_SKIP_IF too. > AT_CHECK([ovs-appctl dpif-netdev/miniflow-parser-set autovalidator], [0], > [dnl > Miniflow extract implementation set to autovalidator. > ]) > -- > 2.25.1 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] dpif-netdev: Simplify atomic function pointer stores
Hi Amber,
Thanks for reviewing and Acking!
> -Original Message-
> From: Amber, Kumar
> Sent: Tuesday 22 February 2022 06:15
> To: ovs-dev@openvswitch.org
> Cc: i.maxim...@ovn.org; Ferriter, Cian
> Subject: RE: [PATCH] dpif-netdev: Simplify atomic function pointer stores
>
> Hi Cian,
>
> Thanks for the Patch.
>
> I have tested the patch and reviewed as well.
> One small minor comment .
>
> > +atomic_store_relaxed(>miniflow_extract_opt,
> > +miniflow_funcs[MFEX_IMPL_SCALAR].extract_func);
> > VLOG_INFO("Not enough packets matched (%u/%u), disabling"
> >" optimized MFEX.", max_hits, stats->pkt_count);
>
> Alignment of the comment is off.
>
Yes, unfortunately I couldn't wrap the atomic_store_relaxed line and keep the
alignment as we'd like. If I do that, the line is 80 characters long which is
just over the 79 character limit.
> Acked-by: Kumar Amber
>
> Regards
> Amber
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v3] system-dpdk.at: Add warning log in mfex fuzzy test.
> -Original Message- > From: Stokes, Ian > Sent: Friday 11 February 2022 10:49 > To: Ferriter, Cian ; Amber, Kumar > ; ovs- > d...@openvswitch.org > Cc: Amber, Kumar ; david.march...@redhat.com > Subject: RE: [ovs-dev] [PATCH v3] system-dpdk.at: Add warning log in mfex > fuzzy test. > > > > -Original Message- > > > From: dev On Behalf Of Kumar Amber > > > Sent: Wednesday 9 February 2022 09:50 > > > To: ovs-dev@openvswitch.org > > > Cc: Amber, Kumar ; david.march...@redhat.com > > > Subject: [ovs-dev] [PATCH v3] system-dpdk.at: Add warning log in mfex > > > fuzzy > > test. > > > > > > Some specific warning are seen on various systems > > > which may not be visible on others but good to add > > > such logs to test to avoid test-case failure. > > > > > > Thw warning only effects the fuzzy tests due to > > > more than 1000+ flows being offloading simultanously. > > > > > > Wilcarding flow count number as for different systems > > > under test the number could vary in the warning log. > > > > > > Suggested-by: Eelco Chaudron > > > Signed-off-by: Kumar Amber > > > > > > > I didn't see the issue that this patch fixes initially. However, if I make > > the below > > change, I can see the error during the test. > > > > Change: > > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > > index 9384cf7f0..165b68f0e 100644 > > --- a/tests/system-dpdk.at > > +++ b/tests/system-dpdk.at > > @@ -256,6 +256,8 @@ AT_KEYWORDS([dpdk]) > > AT_SKIP_IF([! $PYTHON3 -c "import scapy"], [], []) > > AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) > > OVS_DPDK_START() > > +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:flow- > > limit=13500]) > > + > > > > dnl Add userspace bridge and attach it to OVS > > AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) > > > > Error: > > > 2022-02-10T09:42:11.350Z|6|ofproto_dpif_upcall(pmd- > > c21/id:101)|WARN|upcall: datapath reached the dynamic limit of 13500 flows. > > > > Which causes a fail: > > 7. system-dpdk.at:254: FAILED (system-dpdk.at:280) > > > > Applying this patch while keeping the above diff to generate the flow limit > > error > > results in the test passing. I can see the "reached the dynamic limit" > > warning in > > the ovs-vswitchd.log but it doesn't cause a test failure. > > > > Finding the value of "13500" as a flow-limit took some experimentation and > > would probably vary based on platform. > > > > With that, and looking at the .at changes, LGTM. > > > > Acked-by: Cian Ferriter > > Thanks for testing and reviewing folks. > > So does it make sense to apply this to both 2.17 and 2.16 also? > > Thanks > ian I think it makes sense to backport this patch. Thanks, Cian ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v3] system-dpdk.at: Add warning log in mfex fuzzy test.
> -Original Message- > From: dev On Behalf Of Kumar Amber > Sent: Wednesday 9 February 2022 09:50 > To: ovs-dev@openvswitch.org > Cc: Amber, Kumar ; david.march...@redhat.com > Subject: [ovs-dev] [PATCH v3] system-dpdk.at: Add warning log in mfex fuzzy > test. > > Some specific warning are seen on various systems > which may not be visible on others but good to add > such logs to test to avoid test-case failure. > > Thw warning only effects the fuzzy tests due to > more than 1000+ flows being offloading simultanously. > > Wilcarding flow count number as for different systems > under test the number could vary in the warning log. > > Suggested-by: Eelco Chaudron > Signed-off-by: Kumar Amber > I didn't see the issue that this patch fixes initially. However, if I make the below change, I can see the error during the test. Change: diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 9384cf7f0..165b68f0e 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -256,6 +256,8 @@ AT_KEYWORDS([dpdk]) AT_SKIP_IF([! $PYTHON3 -c "import scapy"], [], []) AT_CHECK([$PYTHON3 $srcdir/mfex_fuzzy.py $srcdir], [], [stdout]) OVS_DPDK_START() +AT_CHECK([ovs-vsctl --no-wait set Open_vSwitch . other_config:flow-limit=13500]) + dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev]) Error: > 2022-02-10T09:42:11.350Z|6|ofproto_dpif_upcall(pmd-c21/id:101)|WARN|upcall: > datapath reached the dynamic limit of 13500 flows. Which causes a fail: 7. system-dpdk.at:254: FAILED (system-dpdk.at:280) Applying this patch while keeping the above diff to generate the flow limit error results in the test passing. I can see the "reached the dynamic limit" warning in the ovs-vswitchd.log but it doesn't cause a test failure. Finding the value of "13500" as a flow-limit took some experimentation and would probably vary based on platform. With that, and looking at the .at changes, LGTM. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] dpif-netdev-dpcls: Make subtable reprobe thread-safe.
> -Original Message-
> From: Ilya Maximets
> Sent: Friday 4 February 2022 12:11
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Cc: i.maxim...@ovn.org; Van Haaren, Harry
> Subject: Re: [PATCH] dpif-netdev-dpcls: Make subtable reprobe thread-safe.
>
> On 2/2/22 15:51, Cian Ferriter wrote:
> > The subtable search function can be used at any time by a PMD thread.
> > Setting the subtable search function should be done atomically to
> > prevent garbage data from being read.
> >
> > A dpcls_subtable_lookup_reprobe() call can happen at the same time that
> > DPCLS subtables are being sorted. This could lead to modifications done
> > by the reprobe() to be lost. Prevent this from happening by locking on
> > pmd->flow_mutex. After this change both the reprobe function and a
> > subtable sort will share the flow_mutex preventing modifications by
> > either one from being lost.
> >
> > Fixes: 3d018c3ea79d ("dpif-netdev: add subtable lookup prio set command.")
> > Reported-by: Ilya Maximets
> > Reported-at:
> > https://mail.openvswitch.org/pipermail/ovs-dev/2022-January/390757.html
> > Signed-off-by: Cian Ferriter
>
> Hi, Cian. Thanks for working on this!
> I didn't test that, but see some comments inline.
>
> Best regards, Ilya Maximets.
>
Hi Ilya, thanks for the review. The explanations around pvector_publish were
very helpful!
I sent a new version of the patch:
http://patchwork.ozlabs.org/project/openvswitch/patch/20220208103038.2213498-1-cian.ferri...@intel.com/
> > ---
> > lib/dpif-netdev-private-dpcls.h | 2 +-
> > lib/dpif-netdev.c | 23 +++
> > 2 files changed, 20 insertions(+), 5 deletions(-)
> >
> > diff --git a/lib/dpif-netdev-private-dpcls.h
> > b/lib/dpif-netdev-private-dpcls.h
> > index 7c4a840cb..1b4d69605 100644
> > --- a/lib/dpif-netdev-private-dpcls.h
> > +++ b/lib/dpif-netdev-private-dpcls.h
> > @@ -84,7 +84,7 @@ struct dpcls_subtable {
> > * property of the subtable (eg: only 3 bits of miniflow metadata is
> > * used for the lookup) then this can point at an optimized version of
> > * the lookup function for this particular subtable. */
> > -dpcls_subtable_lookup_func lookup_func;
> > +ATOMIC(dpcls_subtable_lookup_func) lookup_func;
>
> Maybe a comment update saying why it's atomic?
>
Good idea, it's good to document the reason.
Updated this in the v2.
> >
> > /* Caches the masks to match a packet to, reducing runtime
> > calculations. */
> > uint64_t *mf_masks;
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index e28e0b554..9ca6d9842 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -1074,7 +1074,9 @@ dpif_netdev_subtable_lookup_set(struct unixctl_conn
> > *conn, int argc
> OVS_UNUSED,
> > if (!cls) {
> > continue;
> > }
> > +ovs_mutex_lock(>flow_mutex);
> > uint32_t subtbl_changes =
> > dpcls_subtable_lookup_reprobe(cls);
> > +ovs_mutex_unlock(>flow_mutex);
> > if (subtbl_changes) {
> > lookup_dpcls_changed++;
> > lookup_subtable_changed += subtbl_changes;
> > @@ -9736,9 +9738,14 @@ dpcls_create_subtable(struct dpcls *cls, const
> > struct netdev_flow_key *mask)
> >
> > /* Get the preferred subtable search function for this (u0,u1)
> > subtable.
> > * The function is guaranteed to always return a valid implementation,
> > and
> > - * possibly an ISA optimized, and/or specialized implementation.
> > + * possibly an ISA optimized, and/or specialized implementation.
> > Initialize
> > + * the subtable search function atomically.
>
> Again, the comment 'why?' would be helpful. OTOH, the comment in the
> structure
> itself might be enough.
>
I think you're right. The WHY is more important than me just saying WHAT we are
doing.
Hopefully the code says WHAT we are doing, and we say WHY! :)
Updated this in the v2.
> > */
> > -subtable->lookup_func = dpcls_subtable_get_best_impl(unit0, unit1);
> > +dpcls_subtable_lookup_func best_func =
> > dpcls_subtable_get_best_impl(unit0,
> > +
> > unit1);
> > +atomic_uintptr_t *subtable_func = (void *) >lookup_func;
> > +atomic_init(subtable_func, (uintptr_t) best_func);
>
> This pattern looks a bit weird, but for som
Re: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler support.
> -Original Message-
> From: Ilya Maximets
> Sent: Wednesday 2 February 2022 13:57
> To: Stokes, Ian ; Ferriter, Cian
> ; William Tu
> ; d...@openvswitch.org
> Cc: i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> support.
>
> On 2/2/22 13:57, Stokes, Ian wrote:
> >>> -----Original Message-
> >>> From: Ferriter, Cian
> >>> Sent: Friday 28 January 2022 16:32
> >>> To: William Tu ; d...@openvswitch.org
> >>> Subject: RE: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> >> support.
> >>>
> >>>
> >>>> -Original Message-
> >>>> From: dev On Behalf Of William Tu
> >>>> Sent: Thursday 27 January 2022 03:45
> >>>> To: d...@openvswitch.org
> >>>> Subject: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> >> support.
> >>>>
> >>>> Ubuntu Xenial 16.04 is using GCC 5.4 and it does not support
> >>>> target "-mavx512vpopcntdq" and cuases error
> >>>> lib/dpif-netdev-lookup-avx512-gather.c:356:47:
> >>>> error: attribute(target("avx512vpopcntdq")) is unknown
> >>>> GCC 7+ supports vpopcntdq:
> >>>> https://gcc.gnu.org/gcc-7/changes.html
> >>>> The patch detects vpopcntdq and disables AVX512 when not found.
> >>>>
> >>>> Reported-by: Greg Rose
> >>>> Signed-off-by: William Tu
> >>>> ---
> >>>> acinclude.m4 | 2 +-
> >>>> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/acinclude.m4 b/acinclude.m4
> >>>> index 5c971e98ce91..0c360fd1ef73 100644
> >>>> --- a/acinclude.m4
> >>>> +++ b/acinclude.m4
> >>>> @@ -77,7 +77,7 @@ dnl Checks if compiler and binutils supports AVX512.
> >>>> AC_DEFUN([OVS_CHECK_AVX512], [
> >>>>OVS_CHECK_BINUTILS_AVX512
> >>>>OVS_CHECK_CC_OPTION(
> >>>> -[-mavx512f], [ovs_have_cc_mavx512f=yes],
> >> [ovs_have_cc_mavx512f=no])
> >>>> +[-mavx512f -mavx512vpopcntdq], [ovs_have_cc_mavx512f=yes],
> >> [ovs_have_cc_mavx512f=no])
> >>>>AM_CONDITIONAL([HAVE_AVX512F], [test $ovs_have_cc_mavx512f =
> >> yes])
> >>>>if test "$ovs_have_cc_mavx512f" = yes; then
> >>>> AC_DEFINE([HAVE_AVX512F], [1],
> >>>> --
> >>>
> >>> Hi William,
> >>>
> >>> Thanks for this fix, I can verify that this fixes the below error for GCC
> >>> 5 (it will
> >> work for GCC 4.9
> >>> - GCC 6):
> >>> make[3]: Entering directory '/root/cian/ovs/datapath'
> >>> lib/dpif-netdev-lookup-avx512-gather.c:90:1: error:
> >> attribute(target("avx512vpopcntdq")) is unknown
> >>>
> >>> We would like to re-spin a new version of this patch that fixes the
> >>> compile
> >> error for the relevant GCC
> >>> versions (GCC 4.9 - GCC 6) but allows some AVX512 code to be build.
> >>> For setups with GCC 6 for example, we still have support for most of the
> >> AVX512 ISA used in OVS, so we
> >>> can still enable building of this ISA, while still correctly avoiding the
> >> avx512vpopcntdq error that
> >>> your patch does.
> >>>
> >>> Please allow us a few days to test and re-spin a new version of this
> >>> patch.
> >>> Thanks,
> >>> Cian
> >>
> >> Hi again William,
> >>
> >> I have been looking into a new version of this patch to add partial
> >> support to
> >> GCC versions that don't have full support for all the AVX512 ISA we use in
> >> OVS.
> >> This is still a work in progress.
> >>
> >> While I'm looking into this more, I think it doesn't make sense to hold
> >> back this
> >> patch from being merged. It fixes build issues.
> >>
> >> I think the patch should be applied as is. We can add further improvements
> >> later.
> >>
> >> Acked-by: Cian Ferriter
> >>
> >> More details about what I tested:
> >> I switched to GCC 5 on my system and can see the compile issue below before
> >> applying the patch.
> >> After applying the patch, the build is successful.
> >> Before patch
> >
Re: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler support.
> -Original Message-
> From: Ferriter, Cian
> Sent: Friday 28 January 2022 16:32
> To: William Tu ; d...@openvswitch.org
> Subject: RE: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> support.
>
>
> > -Original Message-
> > From: dev On Behalf Of William Tu
> > Sent: Thursday 27 January 2022 03:45
> > To: d...@openvswitch.org
> > Subject: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> > support.
> >
> > Ubuntu Xenial 16.04 is using GCC 5.4 and it does not support
> > target "-mavx512vpopcntdq" and cuases error
> > lib/dpif-netdev-lookup-avx512-gather.c:356:47:
> > error: attribute(target("avx512vpopcntdq")) is unknown
> > GCC 7+ supports vpopcntdq:
> > https://gcc.gnu.org/gcc-7/changes.html
> > The patch detects vpopcntdq and disables AVX512 when not found.
> >
> > Reported-by: Greg Rose
> > Signed-off-by: William Tu
> > ---
> > acinclude.m4 | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/acinclude.m4 b/acinclude.m4
> > index 5c971e98ce91..0c360fd1ef73 100644
> > --- a/acinclude.m4
> > +++ b/acinclude.m4
> > @@ -77,7 +77,7 @@ dnl Checks if compiler and binutils supports AVX512.
> > AC_DEFUN([OVS_CHECK_AVX512], [
> >OVS_CHECK_BINUTILS_AVX512
> >OVS_CHECK_CC_OPTION(
> > -[-mavx512f], [ovs_have_cc_mavx512f=yes], [ovs_have_cc_mavx512f=no])
> > +[-mavx512f -mavx512vpopcntdq], [ovs_have_cc_mavx512f=yes],
> > [ovs_have_cc_mavx512f=no])
> >AM_CONDITIONAL([HAVE_AVX512F], [test $ovs_have_cc_mavx512f = yes])
> >if test "$ovs_have_cc_mavx512f" = yes; then
> > AC_DEFINE([HAVE_AVX512F], [1],
> > --
>
> Hi William,
>
> Thanks for this fix, I can verify that this fixes the below error for GCC 5
> (it will work for GCC 4.9
> - GCC 6):
> make[3]: Entering directory '/root/cian/ovs/datapath'
> lib/dpif-netdev-lookup-avx512-gather.c:90:1: error:
> attribute(target("avx512vpopcntdq")) is unknown
>
> We would like to re-spin a new version of this patch that fixes the compile
> error for the relevant GCC
> versions (GCC 4.9 - GCC 6) but allows some AVX512 code to be build.
> For setups with GCC 6 for example, we still have support for most of the
> AVX512 ISA used in OVS, so we
> can still enable building of this ISA, while still correctly avoiding the
> avx512vpopcntdq error that
> your patch does.
>
> Please allow us a few days to test and re-spin a new version of this patch.
> Thanks,
> Cian
Hi again William,
I have been looking into a new version of this patch to add partial support to
GCC versions that don't have full support for all the AVX512 ISA we use in OVS.
This is still a work in progress.
While I'm looking into this more, I think it doesn't make sense to hold back
this patch from being merged. It fixes build issues.
I think the patch should be applied as is. We can add further improvements
later.
Acked-by: Cian Ferriter
More details about what I tested:
I switched to GCC 5 on my system and can see the compile issue below before
applying the patch.
After applying the patch, the build is successful.
Before patch
Configure message:
checking whether gcc accepts -mavx512f... yes
Build error:
make[3]: Entering directory '/root/cian/ovs/datapath'
lib/dpif-netdev-lookup-avx512-gather.c:90:1: error:
attribute(target("avx512vpopcntdq")) is unknown
After patch
Configure message:
checking whether gcc accepts -mavx512f -mavx512vpopcntdq... no
On GCC 4.8 before the patch is applied, the build is actually successful, since
the "avx512f" only check fails, so the AVX512 build is disabled.
On GCC 4.8 after the patch is applied, the build still works as expected.
Before patch
Configure message:
checking whether gcc -std=gnu99 accepts -mavx512f... no
After patch
Configure message:
checking whether gcc -std=gnu99 accepts -mavx512f -mavx512vpopcntdq... no
I also checked that all was working as expected with the compile time AVX512
flags:
--enable-dpif-default-avx512 --enable-autovalidator
--enable-mfex-default-autovalidator
I can confirm that these compile time flags don't cause issues.
Finally, I tested and confirmed that OVS was still building and running with
AVX512 support when built with GCC 9.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler support.
> -Original Message-
> From: dev On Behalf Of William Tu
> Sent: Thursday 27 January 2022 03:45
> To: d...@openvswitch.org
> Subject: [ovs-dev] [PATCH] acinclude: Detect avx512 vpopcntdq compiler
> support.
>
> Ubuntu Xenial 16.04 is using GCC 5.4 and it does not support
> target "-mavx512vpopcntdq" and cuases error
> lib/dpif-netdev-lookup-avx512-gather.c:356:47:
> error: attribute(target("avx512vpopcntdq")) is unknown
> GCC 7+ supports vpopcntdq:
> https://gcc.gnu.org/gcc-7/changes.html
> The patch detects vpopcntdq and disables AVX512 when not found.
>
> Reported-by: Greg Rose
> Signed-off-by: William Tu
> ---
> acinclude.m4 | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/acinclude.m4 b/acinclude.m4
> index 5c971e98ce91..0c360fd1ef73 100644
> --- a/acinclude.m4
> +++ b/acinclude.m4
> @@ -77,7 +77,7 @@ dnl Checks if compiler and binutils supports AVX512.
> AC_DEFUN([OVS_CHECK_AVX512], [
>OVS_CHECK_BINUTILS_AVX512
>OVS_CHECK_CC_OPTION(
> -[-mavx512f], [ovs_have_cc_mavx512f=yes], [ovs_have_cc_mavx512f=no])
> +[-mavx512f -mavx512vpopcntdq], [ovs_have_cc_mavx512f=yes],
> [ovs_have_cc_mavx512f=no])
>AM_CONDITIONAL([HAVE_AVX512F], [test $ovs_have_cc_mavx512f = yes])
>if test "$ovs_have_cc_mavx512f" = yes; then
> AC_DEFINE([HAVE_AVX512F], [1],
> --
Hi William,
Thanks for this fix, I can verify that this fixes the below error for GCC 5 (it
will work for GCC 4.9 - GCC 6):
make[3]: Entering directory '/root/cian/ovs/datapath'
lib/dpif-netdev-lookup-avx512-gather.c:90:1: error:
attribute(target("avx512vpopcntdq")) is unknown
We would like to re-spin a new version of this patch that fixes the compile
error for the relevant GCC versions (GCC 4.9 - GCC 6) but allows some AVX512
code to be build.
For setups with GCC 6 for example, we still have support for most of the AVX512
ISA used in OVS, so we can still enable building of this ISA, while still
correctly avoiding the avx512vpopcntdq error that your patch does.
Please allow us a few days to test and re-spin a new version of this patch.
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [BUG] dpcls_subtable_lookup_reprobe is not thread-safe.
> -Original Message-
> From: dev On Behalf Of Ilya Maximets
> Sent: Friday 14 January 2022 11:44
> To: ovs-dev
> Cc: i.maxim...@ovn.org
> Subject: [ovs-dev] [BUG] dpcls_subtable_lookup_reprobe is not thread-safe.
>
> The function currently looks like this:
>
> lib/dpif-netdev.c:
>
> static uint32_t
> dpcls_subtable_lookup_reprobe(struct dpcls *cls)
> {
> struct pvector *pvec = >subtables;
> uint32_t subtables_changed = 0;
> struct dpcls_subtable *subtable = NULL;
>
> PVECTOR_FOR_EACH (subtable, pvec) {
> uint32_t u0_bits = subtable->mf_bits_set_unit0;
> uint32_t u1_bits = subtable->mf_bits_set_unit1;
> void *old_func = subtable->lookup_func;
> subtable->lookup_func = dpcls_subtable_get_best_impl(u0_bits,
> u1_bits);
> subtables_changed += (old_func != subtable->lookup_func);
> }
> pvector_publish(pvec);
>
> return subtables_changed;
> }
>
> The problem is that it only pretends to be thread-safe:
>
> 1. PVECTOR_FOR_EACH() iterates over the *current* implementation of a priority
>vector. Hence, the function is changing the pointers that are currently in
>use by a PMD thread. A few possible consequences:
>
>a. Pointer store may be not atomic. In that case PMD thread may read
> garbage and use it as a function pointer leading to a crash on some
> platforms.
>
>b. If PMD thread is currently re-sorting subtables, it already cloned the
> current pvector implementation and will replace it with a new version.
> Therefore modifications done here will be lost.
>
> 2. At best, pvector_publish(pvec) is useless here, because there were no
> pvector
>modifications. At worst, it will publish changes that the other thread is
>working on at the same time. That will likely break the vector.
>
> 3. Even if there were actual pvector modifications and correct use of
>pvector_publish(), these calls should be done under the pmd->flow_mutex
>in order to be safe.
>
> Best regards, Ilya Maximets.
Hi Ilya,
Thanks for finding and reporting this. I'm looking into the details and your
suggestions.
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] system-dpdk: Fix MFEX logs check.
> -Original Message-
> From: David Marchand
> Sent: Wednesday 19 January 2022 17:26
> To: d...@openvswitch.org
> Cc: Stokes, Ian ; Phelan, Michael
> ; Ferriter, Cian
> ; i.maxim...@ovn.org; ktray...@redhat.com
> Subject: [PATCH] system-dpdk: Fix MFEX logs check.
>
> Some warning logs must be waived when using the net/pcap DPDK driver.
> Those logs can affect different DPDK drivers (like mlx5) and the tests in
> system-dpdk are not testing MTU and Rx checksum, we might as well ignore
> those warnings from OVS.
>
> Fixes: d446dcb7e03f ("system-dpdk: Refactor common logs matching.")
> Signed-off-by: David Marchand
Reviewed and tested. This fixes the errors I see prior to applying the patch.
The MFEX tests now pass:
OVS-DPDK unit tests
6: OVS-DPDK - MFEX Autovalidator ok
7: OVS-DPDK - MFEX Autovalidator Fuzzy ok
8: OVS-DPDK - MFEX Configuration ok
Acked-by: Cian Ferriter
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS DPDK MFEX Unit Tests Failing
> -Original Message- > From: David Marchand > Sent: Wednesday 12 January 2022 17:05 > To: Ferriter, Cian > Cc: Phelan, Michael ; d...@openvswitch.org; Ilya > Maximets > Subject: Re: [ovs-dev] OVS DPDK MFEX Unit Tests Failing > > On Wed, Jan 12, 2022 at 5:54 PM Ferriter, Cian > wrote: > > I tested your fix and it works, but I had to modify the port number from 0 > > to 1. If I leave it at 0, > the tests still fail. > > It depends on what DPDK ports get initialised on your system. > On mine, I had no pci device bound to vfio-pci (and unloaded mlx5 > drivers), so the net/pcap port got the 0 dpdk portid. > > > > > > I put the modifications I made inline below. > > > > Perhaps we need to wildcard this number? > > Yes, like what I did for mlx5 testing: > https://patchwork.ozlabs.org/project/openvswitch/patch/20220103141552.27060-2- > david.march...@redhat.com/ > > I updated my repo and I'll send a fix on the ml. > https://github.com/david-marchand/ovs/commit/system-dpdk > > > -- > David Marchand The fix on your updated repo works for me. I'll give my ack here even though this isn't the patch. Happy to test the patch once it's on the mailing list and ack it too. Acked-by: Cian Ferriter ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] OVS DPDK MFEX Unit Tests Failing
Hi David,
I tested your fix and it works, but I had to modify the port number from 0 to
1. If I leave it at 0, the tests still fail.
I put the modifications I made inline below.
Perhaps we need to wildcard this number?
Thanks,
Cian
> -Original Message-
> From: dev On Behalf Of David Marchand
> Sent: Wednesday 12 January 2022 16:17
> To: Phelan, Michael
> Cc: d...@openvswitch.org; Ilya Maximets
> Subject: Re: [ovs-dev] OVS DPDK MFEX Unit Tests Failing
>
> On Wed, Jan 12, 2022 at 4:48 PM Phelan, Michael
> wrote:
> >
> > Hi,
> >
> > During internal testing of the AVX-512 CI, a bug was picked up on the OVS
> > master branch which
> resulted in the MFEX unit tests consistently failing. I believe the bug was
> introduced by commit
> d446dcb7e03fc7bd4e3050c83c22233b0a46d364 “system-dpdk: Refactor common logs
> matching”. It looks like
> this commit changed the logs which in turn meant that the expected output
> became outdated causing the
> unit tests to fail. The unit tests are still failing with the most recent
> updates to master so this
> bug does not seem to have been fixed with any commits that have come after
> the one specified.
>
> Thanks for reporting.
> I did not recheck my series after the dpdk upgrade to 21.11 (with only
> the first patch applied and the whole series does not have the issue
> since it does not rely on net/pcap anymore).
>
> Could you try the following diff:
>
> diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at
> index 1dd7aae1b..922f27032 100644
> --- a/tests/system-dpdk.at
> +++ b/tests/system-dpdk.at
> @@ -243,7 +243,10 @@ OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1
> statistics | grep -oP 'rx_packe
>
> dnl Clean up
> AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> -OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]")
> +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [
> +\@Interface p1 does not support MTU configuration, max packet size
> supported is 1500.@d
> +\@Rx checksum offload is not supported on port 0@d
Here, I have:
\@Rx checksum offload is not supported on port 1@d
> +])")
> AT_CLEANUP
> dnl
> --
>
> @@ -271,7 +274,10 @@ OVS_WAIT_UNTIL([test `ovs-vsctl get interface p1
> statistics | grep -oP 'rx_packe
>
> dnl Clean up
> AT_CHECK([ovs-vsctl del-port br0 p1], [], [stdout], [stderr])
> -OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]")
> +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [
> +\@Interface p1 does not support MTU configuration, max packet size
> supported is 1500.@d
> +\@Rx checksum offload is not supported on port 0@d
Here I have:
\@Rx checksum offload is not supported on port 1@d
> +])")
> AT_CLEANUP
> dnl
> --
>
> @@ -389,6 +395,8 @@ OVS_VSWITCHD_STOP("m4_join([],
> [SYSTEM_DPDK_ALLOWED_LOGS], [
> \@Error: no miniflow extract name provided. Output of
> miniflow-parser-get shows implementation list.@d
> \@Error: unknown miniflow extract implementation superstudy.@d
> \@Error: invalid study_pkt_cnt value: -pmd.@d
> +\@Interface p1 does not support MTU configuration, max packet size
> supported is 1500.@d
> +\@Rx checksum offload is not supported on port 0@d
Here I have:
\@Rx checksum offload is not supported on port 1@d
> ])")
> AT_CLEANUP dnl
> dnl
> --
>
>
> --
> David Marchand
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] Documentation: Change the address in userspace-tunneling.rst
> -Original Message-
> From: dev On Behalf Of Paolo Valerio
> Sent: Wednesday 29 September 2021 21:43
> To: d...@openvswitch.org
> Cc: f...@redhat.com; i.maxim...@ovn.org
> Subject: [ovs-dev] [PATCH] Documentation: Change the address in
> userspace-tunneling.rst
>
> Fixes: b438493e1b03 ("doc: Add DPDK to userspace tunneling guide")
> Signed-off-by: Paolo Valerio
> ---
> Documentation/howto/userspace-tunneling.rst |2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Documentation/howto/userspace-tunneling.rst
> b/Documentation/howto/userspace-tunneling.rst
> index 4e23b2e0c..e96b79985 100644
> --- a/Documentation/howto/userspace-tunneling.rst
> +++ b/Documentation/howto/userspace-tunneling.rst
> @@ -175,7 +175,7 @@ If the tunnel route is missing, adding it now::
>
> $ ovs-appctl ovs/route/add 172.168.1.1/24 br-phy
>
> -Repeat these steps if necessary for `host2`, but using ``192.168.1.1`` and
> +Repeat these steps if necessary for `host2`, but using ``192.168.1.2`` and
> ``172.168.1.2`` for the VM and tunnel interface IP addresses, respectively.
>
> Testing
>
Hi Paolo,
I'm just wondering if you've seen my patch changing these lines in the docs:
http://patchwork.ozlabs.org/project/openvswitch/patch/20210922120033.39221-1-cian.ferri...@intel.com/
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] dpif-netdev: Mention SMC in 2 pmd_thread comments
> -Original Message-
> From: Kevin Traynor
> Sent: Thursday 9 September 2021 14:39
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Subject: Re: [ovs-dev] [PATCH] dpif-netdev: Mention SMC in 2 pmd_thread
> comments
>
> pmd_thread seems like it is referencing a struct so better to just use a
> natural description (also, style is usually to have summary in sentence
> case including ".")
>
> How about:
>
> dpif-netdev: Fix pmd thread comments to include SMC.
>
Sounds better, I'll use this and send a v2.
> On 27/08/2021 16:52, Cian Ferriter wrote:
> > These comments are relevant to SMC too.
> >
>
> Fixes: 60d8ccae135f ("dpif-netdev: Add SMC cache after EMC cache")
>
I'll add the fixes tag.
> (though it's not worth backporting they have moved files)
>
> Very minor comments, so i'll add ack:
>
> Acked-by: Kevin Traynor
>
Thanks for the review Kevin, I'll add your Acked-by tag in the v2.
> > Signed-off-by: Cian Ferriter
> > ---
> > lib/dpif-netdev-private-dfc.h| 3 ++-
> > lib/dpif-netdev-private-thread.h | 8
> > 2 files changed, 6 insertions(+), 5 deletions(-)
> >
> > diff --git a/lib/dpif-netdev-private-dfc.h b/lib/dpif-netdev-private-dfc.h
> > index 92092ebec..3dfc91f0f 100644
> > --- a/lib/dpif-netdev-private-dfc.h
> > +++ b/lib/dpif-netdev-private-dfc.h
> > @@ -59,7 +59,8 @@ extern "C" {
> > * Thread-safety
> > * =
> > *
> > - * Each pmd_thread has its own private exact match cache.
> > + * Each pmd_thread has its own private exact match cache and signature
> > match
> > + * cache.
> > * If dp_netdev_input is not called from a pmd thread, a mutex is used.
> > */
> >
> > diff --git a/lib/dpif-netdev-private-thread.h
> > b/lib/dpif-netdev-private-thread.h
> > index a782d9678..ac4885538 100644
> > --- a/lib/dpif-netdev-private-thread.h
> > +++ b/lib/dpif-netdev-private-thread.h
> > @@ -78,10 +78,10 @@ struct dp_netdev_pmd_thread {
> > struct ovs_refcount ref_cnt;/* Every reference must be
> > refcount'ed. */
> > struct cmap_node node; /* In 'dp->poll_threads'. */
> >
> > -/* Per thread exact-match cache. Note, the instance for cpu core
> > - * NON_PMD_CORE_ID can be accessed by multiple threads, and thusly
> > - * need to be protected by 'non_pmd_mutex'. Every other instance
> > - * will only be accessed by its own pmd thread. */
> > +/* Per thread exact match cache and signature match cache. Note, the
> > + * instance for cpu core NON_PMD_CORE_ID can be accessed by multiple
> > + * threads, and thusly need to be protected by 'non_pmd_mutex'. Every
> > + * other instance will only be accessed by its own pmd thread. */
> > OVS_ALIGNED_VAR(CACHE_LINE_SIZE) struct dfc_cache flow_cache;
> >
> > /* Flow-Table and classifiers
> >
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 3/3] docs/dpdk/bridge: Fix dpif-netdev/miniflow-parser-set formatting
> -Original Message-
> From: Stokes, Ian
> Sent: Thursday 12 August 2021 15:58
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Cc: f...@sysclose.org
> Subject: RE: [ovs-dev] [PATCH 3/3] docs/dpdk/bridge: Fix
> dpif-netdev/miniflow-parser-set formatting
>
> > The "name" parameter isn't optional so don't use brackets around it.
> >
>
> Thanks for the patch Cian, comment below.
>
> > Fixes: 5c5c98cec21b ("docs/dpdk/bridge: Add miniflow extract section.")
> > Signed-off-by: Cian Ferriter
> > ---
> > Documentation/topics/dpdk/bridge.rst | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/topics/dpdk/bridge.rst
> > b/Documentation/topics/dpdk/bridge.rst
> > index f422eb5e9..1aa41958c 100644
> > --- a/Documentation/topics/dpdk/bridge.rst
> > +++ b/Documentation/topics/dpdk/bridge.rst
> > @@ -318,8 +318,8 @@ command also shows whether the CPU supports each
> > implementation ::
> >
> > An implementation can be selected manually by the following command ::
> >
> > -$ ovs-appctl dpif-netdev/miniflow-parser-set [-pmd core_id] [name]
> > - [study_cnt]
> > +$ ovs-appctl dpif-netdev/miniflow-parser-set [-pmd core_id] name \
> > +[study_cnt]
>
> I would keep [study_cnt] aligned under ovs-appctl above, currently seems to
> be extra white space
> before it.
>
The white space before [study_cnt] was intentional. I was following the
convention for wrapping shell commands shown in Documentation/howto/dpdk.rst:
https://github.com/openvswitch/ovs/blob/master/Documentation/howto/dpdk.rst#ports-and-bridges
You are right though, the convention in Documentation/topics/dpdk/bridge.rst is
not to indent after wrapping the shall command. I'm fine with either way, so
I'll align under ovs-appctl in the next version.
> BR
> Ian
> >
> > The above command has two optional parameters: study_cnt and core_id.
> > The core_id sets a particular miniflow extract function to a specific
> > --
> > 2.32.0
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 2/3] dpif-netdev-unixctl.man: Document miniflow-parser-* CMDs
> -Original Message-
> From: Stokes, Ian
> Sent: Thursday 12 August 2021 15:46
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Cc: f...@sysclose.org
> Subject: RE: [ovs-dev] [PATCH 2/3] dpif-netdev-unixctl.man: Document
> miniflow-parser-* CMDs
>
> > Fixes: 3d8f47bc041a ("dpif-netdev: Add command line and function pointer for
> > miniflow extract")
> > Signed-off-by: Cian Ferriter
>
> Hi Cian, as before a one line summary line in the message body would be
> appreciated.
>
Added in the next version.
> > ---
> > lib/dpif-netdev-unixctl.man | 10 ++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man
> > index c44c7eaa0..ceb107f90 100644
> > --- a/lib/dpif-netdev-unixctl.man
> > +++ b/lib/dpif-netdev-unixctl.man
> > @@ -247,3 +247,13 @@ Lists the DPIF implementations that are available.
> > .
> > .IP "\fBdpif-netdev/dpif-impl-set\fR \fIdpif_impl\fR"
> > Sets the DPIF to be used to \fIdpif_impl\fR. By default "dpif_scalar" is
> > used.
> > +.
> > +.IP "\fBdpif-netdev/miniflow-parser-get\fR
> > +Lists the miniflow extract implementations that are available.
> > +.
> > +.IP "\fBdpif-netdev/miniflow-parser-set\fR [\fB-pmd\fR \fIcore\fR] \
> > +\fIminiflow_impl\fR [\fIstudy_cnt\fR]"
> > +Sets the miniflow extract to be used for one or all pmd threads to
>
> Would maybe re-word this to sets the miniflow extract to be used for a
> specified PMD or all PMDs in
> the case where no value is specified.
>
> Just to make it clear the behavior if the argument is not provided.
>
Fixed in the next version.
> BR
> Ian
>
Thanks for the review Ian.
> > +\fIminiflow_impl\fR. By default "scalar" is used. \fIstudy_cnt\fR
> > defaults to
> > +128 and indicates the number of packets that the "study" miniflow
> > +implementation must parse before choosing an optimal implementation.
> > --
> > 2.32.0
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 1/3] dpif-netdev-unixctl.man: Document subtable-lookup-* CMDs
> -Original Message-
> From: Stokes, Ian
> Sent: Thursday 12 August 2021 15:43
> To: Ferriter, Cian ; ovs-dev@openvswitch.org
> Cc: f...@sysclose.org
> Subject: RE: [ovs-dev] [PATCH 1/3] dpif-netdev-unixctl.man: Document
> subtable-lookup-* CMDs
>
> > Fixes: 9ff7cabfd78d ("dpif-netdev: add subtable-lookup-prio-get command.")
> > Fixes: 3d018c3ea79d ("dpif-netdev: add subtable lookup prio set command.")
> > Signed-off-by: Cian Ferriter
>
> Thanks for the patch Cian. Can you add a one line summary to the commit
> message above also?
>
Added in the next version
> Few comments below.
> > ---
> > lib/dpif-netdev-unixctl.man | 10 ++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man
> > index 80304ad35..c44c7eaa0 100644
> > --- a/lib/dpif-netdev-unixctl.man
> > +++ b/lib/dpif-netdev-unixctl.man
> > @@ -232,6 +232,16 @@ When this is the case, the above command prints the
> > load-balancing information
> > of the bonds configured in datapath \fIdp\fR showing the interface
> > associated
> > with each bucket (hash).
> > .
> > +.IP "\fBdpif-netdev/subtable-lookup-prio-get\fR"
> > +Lists the DPCLS implementations or lookup functions that are available as
> > well
> > +as their priorities.
> > +.
> > +.IP "\fBdpif-netdev/subtable-lookup-prio-set\fR \fIlookup_function\fR \
> > +\fIprio\fR"
> > +Sets the priority of a lookup function by name, \fIlookup_function\fR, and
> > +priority, \fIprio\fR, which should be a positive integer value. The
> > highest
> > +priority lookup function is used for classification.
>
> Is it worth mentioning that the number of dpcls ports and subtables affected
> are returned?
>
Added in the next version
>
> BR
> Ian
Thanks for the review Ian.
> > +.
> > .IP "\fBdpif-netdev/dpif-impl-get\fR
> > Lists the DPIF implementations that are available.
> > .
> > --
> > 2.32.0
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] acinclude: Don't set AVX512-related configuration via CFLAGS.
> -Original Message-
> From: Ilya Maximets
> Sent: Tuesday 3 August 2021 18:37
> To: ovs-dev@openvswitch.org; Stokes, Ian
> Cc: Flavio Leitner ; Amber, Kumar ;
> Ferriter, Cian
> ; Van Haaren, Harry ;
> Ilya Maximets
>
> Subject: [PATCH] acinclude: Don't set AVX512-related configuration via CFLAGS.
>
> The correct way to pass configuration options is to define them
> inside the config.h. Additionally, few long lines wrapped and
> fixed the unnecessary double check for -mavx512f.
>
> Fixes: abb807e27dd4 ("dpif-netdev: Add command to switch dpif
> implementation.")
> Fixes: 5324b54e606a ("dpif-netdev: Add configure to enable autovalidator at
> build time.")
> Fixes: e90e115a01af ("dpif-netdev: implement subtable lookup validation.")
> Fixes: 352b6c7116cd ("dpif-lookup: add avx512 gather implementation.")
> Signed-off-by: Ilya Maximets
> ---
>
> Only tested that it builds. Didn't run any runtime checks
> since I have no HW for that.
>
> acinclude.m4 | 36 ++--
> configure.ac | 4 +---
> m4/openvswitch.m4 | 5 -
> 3 files changed, 35 insertions(+), 10 deletions(-)
>
Hi Ilya,
Thanks for the changes, much appreciated.
I've tested out this patch on a system with AVX512 and everything works as
expected.
I can verify that:
* I see an additional output line in the ./configure step like below:
checking whether gcc accepts -mavx512f... yes
* When I add " --enable-autovalidator --enable-dpif-default-avx512
--enable-mfex-default-autovalidator" options to configure, these options still
work as expected, enabling the respective features at compile time.
* I can see the configuration options in config.h after the patch is applied:
~# diff config.h_before_patch config.h_after_patch
26a27,30
> /* Autovalidator for the userspace datapath classifier is a default
>implementation. */
> #define DPCLS_AUTOVALIDATOR_DEFAULT 1
>
29a34,37
> /* DPIF AVX512 is a default implementation of the userspace datapath
>interface. */
> #define DPIF_AVX512_DEFAULT 1
>
35a44,46
> /* Define to 1 if compiler supports AVX512. */
> #define HAVE_AVX512F 1
>
90a102,104
> /* Define to 1 if binutils correctly supports AVX512. */
> #define HAVE_LD_AVX512_GOOD 1
>
263a278,280
>
> /* Autovalidator for miniflow_extract is a default implementation. */
> #define MFEX_AUTOVALIDATOR_DEFAULT 1
Acked-by: Cian Ferriter
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 1/3] dpif-netdev-unixctl.man: Document subtable-lookup-* CMDs
> -Original Message-
> From: Ferriter, Cian
> Sent: Wednesday 28 July 2021 14:20
> To: ovs-dev@openvswitch.org
> Cc: f...@sysclose.org; Ferriter, Cian
> Subject: [PATCH 1/3] dpif-netdev-unixctl.man: Document subtable-lookup-* CMDs
>
> Fixes: 9ff7cabfd78d ("dpif-netdev: add subtable-lookup-prio-get command.")
> Fixes: 3d018c3ea79d ("dpif-netdev: add subtable lookup prio set command.")
> Signed-off-by: Cian Ferriter
> ---
> lib/dpif-netdev-unixctl.man | 10 ++
> 1 file changed, 10 insertions(+)
>
> diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man
> index 80304ad35..c44c7eaa0 100644
> --- a/lib/dpif-netdev-unixctl.man
> +++ b/lib/dpif-netdev-unixctl.man
> @@ -232,6 +232,16 @@ When this is the case, the above command prints the
> load-balancing information
> of the bonds configured in datapath \fIdp\fR showing the interface associated
> with each bucket (hash).
> .
> +.IP "\fBdpif-netdev/subtable-lookup-prio-get\fR"
> +Lists the DPCLS implementations or lookup functions that are available as
> well
> +as their priorities.
> +.
> +.IP "\fBdpif-netdev/subtable-lookup-prio-set\fR \fIlookup_function\fR \
> +\fIprio\fR"
> +Sets the priority of a lookup function by name, \fIlookup_function\fR, and
> +priority, \fIprio\fR, which should be a positive integer value. The highest
> +priority lookup function is used for classification.
> +.
> .IP "\fBdpif-netdev/dpif-impl-get\fR
> Lists the DPIF implementations that are available.
> .
> --
> 2.32.0
Hi Flavio,
You had previously acked this patch when it was part of the DPIF AVX512
patchset. I didn't carry the ack over because of a minor formatting change
(double space after a full-stop) and I didn't want to assume.
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH] dpif-netdev: Fix non-atomic read of smc_enable_db.
> Atomic variables must be read atomically. And since we already have
> 'smc_enable_db' in the PMD context, we need to use it from there to
> avoid reading atomically twice.
>
> Also, 'smc_enable_db' is a global configuration, there is no need
> to read it per-port or per-rxq.
>
> CC: Harry van Haaren
> Fixes: 9ac84a1a3698 ("dpif-avx512: Add ISA implementation of dpif.")
> Signed-off-by: Ilya Maximets
> ---
This LGTM too.
Agreed that it doesn't need to be read per-port or per-RXQ.
Acked-by: Cian Ferriter
I also tested this by running a scenario with multiple PMD threads and toggling
the SMC on and off with EMC off always:
$OVS_DIR/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:smc-enable=true
$OVS_DIR/utilities/ovs-vsctl --no-wait set Open_vSwitch .
other_config:smc-enable=false
I checked pmd-perf-show to see that SMC was being enabled and disabled
correctly. It was.
watch -d -n 1 $OVS_DIR/utilities/ovs-appctl dpif-netdev/pmd-perf-show
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v15 06/10] dpif-netdev: Add a partial HWOL PMD statistic.
> -Original Message- > From: Flavio Leitner > Sent: Thursday 15 July 2021 19:58 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v15 06/10] dpif-netdev: Add a partial HWOL PMD > statistic. > > On Thu, Jul 15, 2021 at 01:39:04PM +, Ferriter, Cian wrote: > > > > > > > -Original Message- > > > From: Flavio Leitner > > > Sent: Friday 9 July 2021 18:54 > > > To: Ferriter, Cian > > > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > > > Subject: Re: [ovs-dev] [v15 06/10] dpif-netdev: Add a partial HWOL PMD > > > statistic. > > > > > > > > > > > > Hi, > > > > > > After rebasing, the performance of branch master boosted in my env > > > from 12Mpps to 13Mpps. However, this specific patch brings down > > > to 12Mpps. I am using dpif_scalar and generic lookup (no AVX512). > > > > > > > Thanks for the investigation. Always great seeing perf numbers and details! > > > > I just want to check my understanding here with what you're seeing: > > > > Performance before DPIF patchset > > 12Mpps > > > > Performance at this patch > > 12Mpps > > > > Performance after DPIF patchset > > 13Mpps > > > > So the performance recovers somewhere else in the patchset? > > > Interesting, which flags are you passing to build OVS? > > Thanks for following up! > fbl > > My flags: ./configure CFLAGS="-g -Ofast -march=native" --with-dpdk=static This is how I build OVS to get the performance numbers below. > > > > I've checked the performance behaviour in my case. I'm going to report > > relative performance numbers. > They are relative to master branch before AVX512 DPIF was applied (c36c8e3). > > I tried to run a similar testcase, I can see you are using EMC from the > > memcmp in perf top output. I > am also using the scalar DPIF in all the below testcases. > > > > Master before AVX512 DPIF (c36c8e3) > > 1.000x (0.0%) > > DPIF patch 3 - dpif-avx512: Add ISA implementation of dpif. > > 1.010x (1.0%) > > DPIF patch 4 - dpif-netdev: Add command to switch dpif implementation. > > 1.042x (4.2%) > > DPIF patch 5 - dpif-netdev: Add command to get dpif implementations. > > 1.063x (6.3%) > > DPIF patch 6 - dpif-netdev: Add a partial HWOL PMD statistic. > > 1.069x (6.9%) > > Latest master which has AVX512 DPIF patches (d2e9703) > > 1.075x (7.5%) > > Master before AVX512 DPIF (c36c8e3), with prefetch change > > 0.983x (-1.7%) > > Latest master which has AVX512 DPIF patches (d2e9703), with prefetch change > > 1.080x (8.0%) > > > > > (I don't think this report should block the patch because the > > > counter are interesting and the analysis below doesn't point > > > directly to the proposed changes.) > > > > > > This is a diff using all patches applied versus this patch reverted: > > > 21.44% +6.08% ovs-vswitchd[.] miniflow_extract > > > 8.94% -1.92% libc-2.28.so[.] __memcmp_avx2_movbe > > > 14.62% +1.44% ovs-vswitchd[.] dp_netdev_input__ > > > 2.80% -1.08% ovs-vswitchd[.] > > > dp_netdev_pmd_flush_output_on_port > > > 3.44% -0.91% ovs-vswitchd[.] netdev_send > > > > > > This is the code side by side, patch applied on the right side: > > > (sorry, long lines) > > > > > > > My mail client has wrapped the below lines, sorry for mangling the output! > > > > > > Please find it here: > > https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385448.html > > > > > > > > > > > I don't see any relevant optimization difference in the code > > > above, but the "mov %r15w,-0x2(%r13)" on the right side accounts > > > for almost all the difference, though on the left side it seems > > > a bit more spread. > > > > > > I applied the patch below and it helped to get to 12.7Mpps, so > > > almost at the same levels. I wonder if you see the same result. > > > > > > > Since I don't see the drop that you see with this patch, when I apply the > > below patch to the latest > master, I see a smaller benefit. > > The relative performance after adding the below prefetch compared to before > > (latest master): > > 1.005x (0.5%) > > > > When I compare before/after performance (including the prefetch code, on > > latest master), the
Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache
> -Original Message- > From: Eli Britstein > Sent: Thursday 15 July 2021 15:10 > To: Ferriter, Cian ; Ilya Maximets > ; Gaëtan Rivet > ; d...@openvswitch.org; Van Haaren, Harry > > Cc: Majd Dibbiny ; Stokes, Ian ; > Flavio Leitner > > Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache > > > On 7/15/2021 4:35 PM, Ferriter, Cian wrote: > > External email: Use caution opening links or attachments > > > > > >> -Original Message- > >> From: Eli Britstein > >> Sent: Wednesday 14 July 2021 16:21 > >> To: Ferriter, Cian ; Ilya Maximets > >> ; Gaëtan Rivet > >> ; d...@openvswitch.org; Van Haaren, Harry > >> > >> Cc: Majd Dibbiny ; Stokes, Ian ; > >> Flavio Leitner > >> > >> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >> cache > >> > >> > >> On 7/14/2021 5:58 PM, Ferriter, Cian wrote: > >>> External email: Use caution opening links or attachments > >>> > >>> > >>>> -Original Message- > >>>> From: Ilya Maximets > >>>> Sent: Friday 9 July 2021 21:53 > >>>> To: Ferriter, Cian ; Gaëtan Rivet > >>>> ; Eli Britstein > >>>> ; d...@openvswitch.org; Van Haaren, Harry > >>>> > >>>> Cc: Majd Dibbiny ; Ilya Maximets ; > >>>> Stokes, Ian > >>>> ; Flavio Leitner > >>>> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >>>> cache > >>>> > >>>> On 7/8/21 6:43 PM, Ferriter, Cian wrote: > >>>>> Hi Gaetan, Eli and all, > >>>>> > >>>>> Thanks for the patch and the info on how it affects performance in your > >>>>> case. I just wanted to > >> post > >>>> the performance we are seeing. > >>>>> I've posted the numbers inline. Please note, I'll be away on leave till > >>>>> Tuesday. > >>>>> Thanks, > >>>>> Cian > >>>>> > >>>>>> -Original Message- > >>>>>> From: Gaëtan Rivet > >>>>>> Sent: Wednesday 7 July 2021 17:36 > >>>>>> To: Eli Britstein ; > >>>>>> ; Van Haaren, > >>>> Harry > >>>>>> ; Ferriter, Cian > >>>>>> Cc: Majd Dibbiny ; Ilya Maximets > >>>>>> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >>>>>> cache > >>>>>> > >>>>>> On Wed, Jul 7, 2021, at 17:05, Eli Britstein wrote: > >>>>>>> Port numbers are usually small. Maintain an array of netdev handles > >>>>>>> indexed > >>>>>>> by port numbers. It accelerates looking up for them for > >>>>>>> netdev_hw_miss_packet_recover(). > >>>>>>> > >>>>>>> Reported-by: Cian Ferriter > >>>>>>> Signed-off-by: Eli Britstein > >>>>>>> Reviewed-by: Gaetan Rivet > >>>>>>> --- > >>>>> > >>>>> > >>>>>>> ___ > >>>>>>> dev mailing list > >>>>>>> d...@openvswitch.org > >>>>>>> > >> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flis > >> tinfo%2Fovs- > >> > devdata=04%7C01%7Celibr%40nvidia.com%7C7ca0caf9434e429e4ffd08d946d7cf2f%7C43083d15727340c1b7db39e > >> > fd9ccc17a%7C0%7C0%7C637618715041410254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ > >> > BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=rv%2FdenANxrcTGxBBbRvhhlNioyswL7ieFr8AGcGtCs8%3Drese > >> rved=0 > >>>>>> Hello, > >>>>>> > >>>>>> I tested the performance impact of this patch with a partial offload > >>>>>> setup. > >>>>>> As reported by pmd-stats-show, in average cycles per packet: > >>>>>> > >>>>>> Before vxlan-decap: 525 c/p > >>>>>> After vxlan-decap: 542 c/p > >>>>>> After this fix: 530 c/p > >>>>>> > >>>>>> Without those fixes, vxlan-decap has a 3.2% negative impact
Re: [ovs-dev] [v15 06/10] dpif-netdev: Add a partial HWOL PMD statistic.
> -Original Message-
> From: Flavio Leitner
> Sent: Friday 9 July 2021 18:54
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v15 06/10] dpif-netdev: Add a partial HWOL PMD
> statistic.
>
>
>
> Hi,
>
> After rebasing, the performance of branch master boosted in my env
> from 12Mpps to 13Mpps. However, this specific patch brings down
> to 12Mpps. I am using dpif_scalar and generic lookup (no AVX512).
>
Thanks for the investigation. Always great seeing perf numbers and details!
I just want to check my understanding here with what you're seeing:
Performance before DPIF patchset
12Mpps
Performance at this patch
12Mpps
Performance after DPIF patchset
13Mpps
So the performance recovers somewhere else in the patchset?
I've checked the performance behaviour in my case. I'm going to report relative
performance numbers. They are relative to master branch before AVX512 DPIF was
applied (c36c8e3).
I tried to run a similar testcase, I can see you are using EMC from the memcmp
in perf top output. I am also using the scalar DPIF in all the below testcases.
Master before AVX512 DPIF (c36c8e3)
1.000x (0.0%)
DPIF patch 3 - dpif-avx512: Add ISA implementation of dpif.
1.010x (1.0%)
DPIF patch 4 - dpif-netdev: Add command to switch dpif implementation.
1.042x (4.2%)
DPIF patch 5 - dpif-netdev: Add command to get dpif implementations.
1.063x (6.3%)
DPIF patch 6 - dpif-netdev: Add a partial HWOL PMD statistic.
1.069x (6.9%)
Latest master which has AVX512 DPIF patches (d2e9703)
1.075x (7.5%)
Master before AVX512 DPIF (c36c8e3), with prefetch change
0.983x (-1.7%)
Latest master which has AVX512 DPIF patches (d2e9703), with prefetch change
1.080x (8.0%)
> (I don't think this report should block the patch because the
> counter are interesting and the analysis below doesn't point
> directly to the proposed changes.)
>
> This is a diff using all patches applied versus this patch reverted:
> 21.44% +6.08% ovs-vswitchd[.] miniflow_extract
> 8.94% -1.92% libc-2.28.so[.] __memcmp_avx2_movbe
> 14.62% +1.44% ovs-vswitchd[.] dp_netdev_input__
> 2.80% -1.08% ovs-vswitchd[.]
> dp_netdev_pmd_flush_output_on_port
> 3.44% -0.91% ovs-vswitchd[.] netdev_send
>
> This is the code side by side, patch applied on the right side:
> (sorry, long lines)
>
My mail client has wrapped the below lines, sorry for mangling the output!
Please find it here:
https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385448.html
>
>
> I don't see any relevant optimization difference in the code
> above, but the "mov %r15w,-0x2(%r13)" on the right side accounts
> for almost all the difference, though on the left side it seems
> a bit more spread.
>
> I applied the patch below and it helped to get to 12.7Mpps, so
> almost at the same levels. I wonder if you see the same result.
>
Since I don't see the drop that you see with this patch, when I apply the below
patch to the latest master, I see a smaller benefit.
The relative performance after adding the below prefetch compared to before
(latest master):
1.005x (0.5%)
When I compare before/after performance (including the prefetch code, on latest
master), the overall performance difference is 0.5% here.
> diff --git a/lib/flow.c b/lib/flow.c
> index 729d59b1b..4572e356b 100644
> --- a/lib/flow.c
> +++ b/lib/flow.c
> @@ -746,6 +746,9 @@ miniflow_extract(struct dp_packet *packet, struct
> miniflow *dst)
> uint8_t *ct_nw_proto_p = NULL;
> ovs_be16 ct_tp_src = 0, ct_tp_dst = 0;
>
> +/* dltype will be updated later. */
> +OVS_PREFETCH_WRITE(miniflow_pointer(mf, dl_type));
> +
> /* Metadata. */
> if (flow_tnl_dst_is_set(>tunnel)) {
> miniflow_push_words(mf, tunnel, >tunnel,
>
>
> fbl
>
Thanks,
Cian
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache
> -Original Message- > From: Eli Britstein > Sent: Wednesday 14 July 2021 16:21 > To: Ferriter, Cian ; Ilya Maximets > ; Gaëtan Rivet > ; d...@openvswitch.org; Van Haaren, Harry > > Cc: Majd Dibbiny ; Stokes, Ian ; > Flavio Leitner > > Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache > > > On 7/14/2021 5:58 PM, Ferriter, Cian wrote: > > External email: Use caution opening links or attachments > > > > > >> -Original Message- > >> From: Ilya Maximets > >> Sent: Friday 9 July 2021 21:53 > >> To: Ferriter, Cian ; Gaëtan Rivet > >> ; Eli Britstein > >> ; d...@openvswitch.org; Van Haaren, Harry > >> > >> Cc: Majd Dibbiny ; Ilya Maximets ; > >> Stokes, Ian > >> ; Flavio Leitner > >> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >> cache > >> > >> On 7/8/21 6:43 PM, Ferriter, Cian wrote: > >>> Hi Gaetan, Eli and all, > >>> > >>> Thanks for the patch and the info on how it affects performance in your > >>> case. I just wanted to > post > >> the performance we are seeing. > >>> I've posted the numbers inline. Please note, I'll be away on leave till > >>> Tuesday. > >>> Thanks, > >>> Cian > >>> > >>>> -Original Message- > >>>> From: Gaëtan Rivet > >>>> Sent: Wednesday 7 July 2021 17:36 > >>>> To: Eli Britstein ; > >>>> ; Van Haaren, > >> Harry > >>>> ; Ferriter, Cian > >>>> Cc: Majd Dibbiny ; Ilya Maximets > >>>> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >>>> cache > >>>> > >>>> On Wed, Jul 7, 2021, at 17:05, Eli Britstein wrote: > >>>>> Port numbers are usually small. Maintain an array of netdev handles > >>>>> indexed > >>>>> by port numbers. It accelerates looking up for them for > >>>>> netdev_hw_miss_packet_recover(). > >>>>> > >>>>> Reported-by: Cian Ferriter > >>>>> Signed-off-by: Eli Britstein > >>>>> Reviewed-by: Gaetan Rivet > >>>>> --- > >>> > >>> > >>>>> ___ > >>>>> dev mailing list > >>>>> d...@openvswitch.org > >>>>> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.openvswitch.org%2Fmailman%2Flis > tinfo%2Fovs- > devdata=04%7C01%7Celibr%40nvidia.com%7C7ca0caf9434e429e4ffd08d946d7cf2f%7C43083d15727340c1b7db39e > fd9ccc17a%7C0%7C0%7C637618715041410254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ > BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=rv%2FdenANxrcTGxBBbRvhhlNioyswL7ieFr8AGcGtCs8%3Drese > rved=0 > >>>>> > >>>> Hello, > >>>> > >>>> I tested the performance impact of this patch with a partial offload > >>>> setup. > >>>> As reported by pmd-stats-show, in average cycles per packet: > >>>> > >>>> Before vxlan-decap: 525 c/p > >>>> After vxlan-decap: 542 c/p > >>>> After this fix: 530 c/p > >>>> > >>>> Without those fixes, vxlan-decap has a 3.2% negative impact on cycles, > >>>> with the fixes, the impact is reduced to 0.95%. > >>>> > >>>> As I had to force partial offloads for our hardware, it would be better > >>>> with an outside confirmation on a proper setup. > >>>> > >>>> Kind regards, > >>>> -- > >>>> Gaetan Rivet > >>> I'm showing the performance relative to what we measured on OVS master > >>> directly before the VXLAN > >> HWOL changes went in. All of the below results are using the scalar DPIF > >> and partial HWOL. > >>> Link to "Fixup patches": > https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpatchwork.ozlabs.org%2Fproject%2Fopen > vswitch%2Flist%2F%3Fseries%3D252356data=04%7C01%7Celibr%40nvidia.com%7C7ca0caf9434e429e4ffd08d946 > d7cf2f%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637618715041410254%7CUnknown%7CTWFpbGZsb3d8eyJWIjo > iMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=Y62OrCRyS00vJHHPQvAHyhG5C > 4eO%2FSfWMCSPtszn3Is%3Dreserved=0 > >>> > >>> Master before VXLAN HWOL change
Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache
> -Original Message- > From: Ilya Maximets > Sent: Friday 9 July 2021 21:53 > To: Ferriter, Cian ; Gaëtan Rivet ; > Eli Britstein > ; d...@openvswitch.org; Van Haaren, Harry > > Cc: Majd Dibbiny ; Ilya Maximets ; > Stokes, Ian > ; Flavio Leitner > Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache > > On 7/8/21 6:43 PM, Ferriter, Cian wrote: > > Hi Gaetan, Eli and all, > > > > Thanks for the patch and the info on how it affects performance in your > > case. I just wanted to post > the performance we are seeing. > > > > I've posted the numbers inline. Please note, I'll be away on leave till > > Tuesday. > > Thanks, > > Cian > > > >> -Original Message- > >> From: Gaëtan Rivet > >> Sent: Wednesday 7 July 2021 17:36 > >> To: Eli Britstein ; > >> ; Van Haaren, > Harry > >> ; Ferriter, Cian > >> Cc: Majd Dibbiny ; Ilya Maximets > >> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array > >> cache > >> > >> On Wed, Jul 7, 2021, at 17:05, Eli Britstein wrote: > >>> Port numbers are usually small. Maintain an array of netdev handles > >>> indexed > >>> by port numbers. It accelerates looking up for them for > >>> netdev_hw_miss_packet_recover(). > >>> > >>> Reported-by: Cian Ferriter > >>> Signed-off-by: Eli Britstein > >>> Reviewed-by: Gaetan Rivet > >>> --- > > > > > > > >>> ___ > >>> dev mailing list > >>> d...@openvswitch.org > >>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev > >>> > >> > >> Hello, > >> > >> I tested the performance impact of this patch with a partial offload setup. > >> As reported by pmd-stats-show, in average cycles per packet: > >> > >> Before vxlan-decap: 525 c/p > >> After vxlan-decap: 542 c/p > >> After this fix: 530 c/p > >> > >> Without those fixes, vxlan-decap has a 3.2% negative impact on cycles, > >> with the fixes, the impact is reduced to 0.95%. > >> > >> As I had to force partial offloads for our hardware, it would be better > >> with an outside confirmation on a proper setup. > >> > >> Kind regards, > >> -- > >> Gaetan Rivet > > > > I'm showing the performance relative to what we measured on OVS master > > directly before the VXLAN > HWOL changes went in. All of the below results are using the scalar DPIF and > partial HWOL. > > > > Link to "Fixup patches": > > http://patchwork.ozlabs.org/project/openvswitch/list/?series=252356 > > > > Master before VXLAN HWOL changes (f0e4a73) > > 1.000x > > > > Latest master after VXLAN HWOL changes (b780911) > > 0.918x (-8.2%) > > > > After fixup patches on OVS ML are applied (with ALLOW_EXPERIMENTAL_API=off) > > 0.973x (-2.7%) > > > > After fixup patches on OVS ML are applied and after ALLOW_EXPERIMENTAL_API > > is removed. > > 0.938x (-6.2%) > > > > I ran the last set of results by applying the below diff. I did this > > because I'm assuming the plan > is to remove the ALLOW_EXPERIMENTAL_API '#ifdef's at some point? > > Yes, that is the plan. > Thanks for confirming this. > And thanks for testing, Gaetan and Cian! > > Could you also provide more details on your test environment, > so someone else can reproduce? > Good idea, I'll add the details inline below. These details apply to the performance measured previously by me, and the performance in this mail. > What is important to know: > - Test configuration: P2P, V2V, PVP, etc. P2P 1 PHY port 1 RXQ > - Test type: max. throughput, zero packet loss. Max throughput. > - OVS config: EMC, SMC, HWOL, AVX512 - on/off/type In all tests, all packets hit a single datapath flow with "offloaded:partial". So all packets are partially offloaded, skipping miniflow_extract() and EMC/SMC/DPCLS lookups. AVX512 is off. > - Installed OF rules. $ $OVS_DIR/utilities/ovs-ofctl dump-flows br0 cookie=0x0, duration=253.691s, table=0, n_packets=2993867136, n_bytes=179632028160, in_port=phy0 actions=IN_PORT > - Traffic pattern: Packet size, number of flows, packet type. 64B, 1 flow, ETH/IP packets. > > This tests also didn't include the fix from Balazs, IIUC, because > they were performed a bit before that patch got accepted. > Correct, the above tests didn't include the optimization from Bal
Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache
Hi Gaetan, Eli and all,
Thanks for the patch and the info on how it affects performance in your case. I
just wanted to post the performance we are seeing.
I've posted the numbers inline. Please note, I'll be away on leave till Tuesday.
Thanks,
Cian
> -Original Message-
> From: Gaëtan Rivet
> Sent: Wednesday 7 July 2021 17:36
> To: Eli Britstein ;
> ; Van Haaren, Harry
> ; Ferriter, Cian
> Cc: Majd Dibbiny ; Ilya Maximets
> Subject: Re: [ovs-dev] [PATCH 2/2] dpif-netdev: Introduce netdev array cache
>
> On Wed, Jul 7, 2021, at 17:05, Eli Britstein wrote:
> > Port numbers are usually small. Maintain an array of netdev handles indexed
> > by port numbers. It accelerates looking up for them for
> > netdev_hw_miss_packet_recover().
> >
> > Reported-by: Cian Ferriter
> > Signed-off-by: Eli Britstein
> > Reviewed-by: Gaetan Rivet
> > ---
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >
>
> Hello,
>
> I tested the performance impact of this patch with a partial offload setup.
> As reported by pmd-stats-show, in average cycles per packet:
>
> Before vxlan-decap: 525 c/p
> After vxlan-decap: 542 c/p
> After this fix: 530 c/p
>
> Without those fixes, vxlan-decap has a 3.2% negative impact on cycles,
> with the fixes, the impact is reduced to 0.95%.
>
> As I had to force partial offloads for our hardware, it would be better
> with an outside confirmation on a proper setup.
>
> Kind regards,
> --
> Gaetan Rivet
I'm showing the performance relative to what we measured on OVS master directly
before the VXLAN HWOL changes went in. All of the below results are using the
scalar DPIF and partial HWOL.
Link to "Fixup patches":
http://patchwork.ozlabs.org/project/openvswitch/list/?series=252356
Master before VXLAN HWOL changes (f0e4a73)
1.000x
Latest master after VXLAN HWOL changes (b780911)
0.918x (-8.2%)
After fixup patches on OVS ML are applied (with ALLOW_EXPERIMENTAL_API=off)
0.973x (-2.7%)
After fixup patches on OVS ML are applied and after ALLOW_EXPERIMENTAL_API is
removed.
0.938x (-6.2%)
I ran the last set of results by applying the below diff. I did this because
I'm assuming the plan is to remove the ALLOW_EXPERIMENTAL_API '#ifdef's at some
point?
Diff:
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index accb23a1a..0e29c609f 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -7132,7 +7132,6 @@ dp_netdev_hw_flow(const struct dp_netdev_pmd_thread *pmd,
struct netdev *netdev OVS_UNUSED;
uint32_t mark;
-#ifdef ALLOW_EXPERIMENTAL_API /* Packet restoration API required. */
/* Restore the packet if HW processing was terminated before completion. */
netdev = pmd_netdev_cache_lookup(pmd, port_no);
if (OVS_LIKELY(netdev)) {
@@ -7143,7 +7142,6 @@ dp_netdev_hw_flow(const struct dp_netdev_pmd_thread *pmd,
return -1;
}
}
-#endif
/* If no mark, no flow to find. */
if (!dp_packet_has_flow_mark(packet, )) {
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v14 06/11] dpif-netdev: Add command to get dpif implementations.
Hi Flavio,
Thanks for your comments. My responses are inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Wednesday 7 July 2021 18:29
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v14 06/11] dpif-netdev: Add command to get dpif
> implementations.
>
>
> Hello,
>
> Please find my comments below.
>
> On Thu, Jul 01, 2021 at 04:06:14PM +0100, Cian Ferriter wrote:
> > From: Harry van Haaren
> >
> > This commit adds a new command to retrieve the list of available
> > DPIF implementations. This can be used by to check what implementations
> > of the DPIF are available in any given OVS binary. It also returns which
> > implementations are in use by the OVS PMD threads.
> >
> > Usage:
> > $ ovs-appctl dpif-netdev/dpif-impl-get
> >
> > Signed-off-by: Harry van Haaren
> > Co-authored-by: Cian Ferriter
> > Signed-off-by: Cian Ferriter
> >
> > ---
> >
> > v14:
> > - Rename command to dpif-impl-get.
> > - Hide more of the dpif impl details from lib/dpif-netdev.c. Pass a
> > dynamic_string to return the dpif-impl-get CMD output.
> > - Add information about which DPIF impl is currently in use by each PMD
> > thread.
> >
> > v13:
> > - Add NEWS item about DPIF get and set commands here rather than in a
> > later commit.
> > - Add documentation items about DPIF set commands here rather than in a
> > later commit.
> > ---
> > Documentation/topics/dpdk/bridge.rst | 8 +++
> > NEWS | 1 +
> > lib/dpif-netdev-private-dpif.c | 33
> > lib/dpif-netdev-private-dpif.h | 8 +++
> > lib/dpif-netdev-unixctl.man | 3 +++
> > lib/dpif-netdev.c| 30 +
> > 6 files changed, 83 insertions(+)
> >
> > diff --git a/Documentation/topics/dpdk/bridge.rst
> > b/Documentation/topics/dpdk/bridge.rst
> > index 06d1f943c..2d0850836 100644
> > --- a/Documentation/topics/dpdk/bridge.rst
> > +++ b/Documentation/topics/dpdk/bridge.rst
> > @@ -226,6 +226,14 @@ stats associated with the datapath.
> > Just like with the SIMD DPCLS feature above, SIMD can be applied to the
> > DPIF to
> > improve performance.
> >
> > +OVS provides multiple implementations of the DPIF. The available
> > +implementations can be listed with the following command ::
> > +
> > +$ ovs-appctl dpif-netdev/dpif-impl-get
> > +Available DPIF implementations:
> > + dpif_scalar (pmds: none)
> > + dpif_avx512 (pmds: 1,2,6,7)
> > +
> > By default, dpif_scalar is used. The DPIF implementation can be selected by
> > name ::
> >
> > diff --git a/NEWS b/NEWS
> > index e23506225..cf0987a24 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -13,6 +13,7 @@ Post-v2.15.0
> > * Refactor lib/dpif-netdev.c to multiple header files.
> > * Add avx512 implementation of dpif which can process non recirculated
> > packets. It supports partial HWOL, EMC, SMC and DPCLS lookups.
> > + * Add commands to get and set the dpif implementations.
> > - ovs-ctl:
> > * New option '--no-record-hostname' to disable hostname configuration
> > in ovsdb on startup.
> > diff --git a/lib/dpif-netdev-private-dpif.c b/lib/dpif-netdev-private-dpif.c
> > index da3511f51..4eaefb291 100644
> > --- a/lib/dpif-netdev-private-dpif.c
> > +++ b/lib/dpif-netdev-private-dpif.c
> > @@ -92,6 +92,39 @@ dp_netdev_impl_set_default_by_name(const char *name)
> >
> > }
> >
> > +uint32_t
> > +dp_netdev_impl_get(struct ds *reply, struct dp_netdev_pmd_thread
> > **pmd_list,
> > + size_t n)
> > +{
> > +/* Add all dpif functions to reply string. */
> > +ds_put_cstr(reply, "Available DPIF implementations:\n");
> > +
> > +for (uint32_t i = 0; i < ARRAY_SIZE(dpif_impls); i++) {
> > +ds_put_format(reply, " %s (pmds: ", dpif_impls[i].name);
> > +
> > +for (size_t j = 0; j < n; j++) {
> > +struct dp_netdev_pmd_thread *pmd = pmd_list[j];
> > +if (pmd->core_id == NON_PMD_CORE_ID) {
> > +continue;
> > +}
> > +
> > +if (pmd->netdev_input_func == dpif_impls[i].input_func) {
> > +ds_put_format(reply, "%u,", pmd->core_id);
> > +}
> > +}
> > +
> &g
Re: [ovs-dev] [v14 05/11] dpif-netdev: Add command to switch dpif implementation.
Hi Flavio,
Thanks for your comments. My responses are inline.
Thanks,
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Wednesday 7 July 2021 15:40
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v14 05/11] dpif-netdev: Add command to switch dpif
> implementation.
>
> Hi,
>
> Please see my comments below.
>
> > diff --git a/lib/dpif-netdev-private-dpif.c b/lib/dpif-netdev-private-dpif.c
> > new file mode 100644
> > index 0..da3511f51
> > --- /dev/null
> > +++ b/lib/dpif-netdev-private-dpif.c
> > @@ -0,0 +1,122 @@
> > +/*
> > + * Copyright (c) 2021 Intel Corporation.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + * http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#include
> > +
> > +#include "dpif-netdev-private-dpif.h"
> > +#include "dpif-netdev-private-thread.h"
> > +
> > +#include
> > +#include
> > +
> > +#include "openvswitch/dynamic-string.h"
> > +#include "openvswitch/vlog.h"
> > +#include "util.h"
> > +
> > +VLOG_DEFINE_THIS_MODULE(dpif_netdev_impl);
> > +
> > +enum dpif_netdev_impl_info_idx {
> > +DPIF_NETDEV_IMPL_SCALAR,
> > +DPIF_NETDEV_IMPL_AVX512
> > +};
> > +
> > +/* Actual list of implementations goes here. */
> > +static struct dpif_netdev_impl_info_t dpif_impls[] = {
> > +/* The default scalar C code implementation. */
> > +[DPIF_NETDEV_IMPL_SCALAR] = { .input_func = dp_netdev_input,
> > + .probe = NULL,
> > + .name = "dpif_scalar", },
> > +
> > +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__)
> > +/* Only available on x86_64 bit builds with SSE 4.2 used for OVS core.
> > */
> > +[DPIF_NETDEV_IMPL_AVX512] = { .input_func =
> > dp_netdev_input_outer_avx512,
> > + .probe = dp_netdev_input_outer_avx512_probe,
> > + .name = "dpif_avx512", },
> > +#endif
> > +};
> > +
> > +static dp_netdev_input_func default_dpif_func;
> > +
> > +dp_netdev_input_func
> > +dp_netdev_impl_get_default(void)
> > +{
> > +/* For the first call, this will be NULL. Compute the compile time
> > default.
> > + */
> > +if (!default_dpif_func) {
> > +int dpif_idx = 0;
>
> That should be DPIF_NETDEV_IMPL_SCALAR.
>
Agreed, I'll fix this.
> > +
> > +/* Configure-time overriding to run test suite on all implementations. */
> > +#if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__)
> > +#ifdef DPIF_AVX512_DEFAULT
> > +ovs_assert(dpif_impls[DPIF_NETDEV_IMPL_AVX512].input_func
> > + == dp_netdev_input_outer_avx512);
>
> This assert() makes little sense now. It's not possible to change
> the dpif_impls at runtime, and if we change the code we will notice
> the problem only at runtime. Wouldn't it make more sense to make it
> generic like below?
>
> #ifdef DPIF_AVX512_DEFAULT
> dp_netdev_input_func_probe probe;
>
> /* Check if the compiled default is compatible. */
> probe = dpif_impls[DPIF_NETDEV_IMPL_AVX512].probe;
> if (!probe || !probe()) {
> dpif_idx = DPIF_NETDEV_IMPL_AVX512;
> }
> #endif
>
>
Makes sense. Your implementation looks clean. I'll use it in the next version.
Thanks!
> > +if (!dp_netdev_input_outer_avx512_probe()) {
> > +dpif_idx = DPIF_NETDEV_IMPL_AVX512;
> > +};
> > +#endif
> > +#endif
> > +
> > +VLOG_INFO("Default DPIF implementation is %s.\n",
> > + dpif_impls[dpif_idx].name);
> > +default_dpif_func = dpif_impls[dpif_idx].input_func;
> > +}
> > +
> > +return default_dpif_func;
> > +}
> > +
> > +int32_t
> > +dp_netdev_impl_set_default_by_name(const char *nam
Re: [ovs-dev] [v14 02/11] dpif-netdev: Split HWOL out to own header file.
Hi Flavio,
Thanks for your comment. My response is inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Wednesday 7 July 2021 00:06
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v14 02/11] dpif-netdev: Split HWOL out to own header
> file.
>
>
> Hi,
>
> After the refactoring and rebasing, this patch doesn't seem
> necessary anymore. I don't see value in keeping it.
> Can we drop it? What do you think?
>
> fbl
>
>
Good catch and agreed! This is no longer necessary. I'll drop it for the next
version.
> On Thu, Jul 01, 2021 at 04:06:10PM +0100, Cian Ferriter wrote:
> > From: Harry van Haaren
> >
> > This commit moves the datapath lookup functions required for
> > hardware offload to a separate file. This allows other DPIF
> > implementations to access the lookup functions, encouraging
> > code reuse.
> >
> > Signed-off-by: Harry van Haaren
> >
> > ---
> >
> > Cc: Gaetan Rivet
> > Cc: Sriharsha Basavapatna
> >
> > v14:
> > - Fix spelling mistake in commit message.
> >
> > v13:
> > - Minor code refactor to address review comments.
> > ---
> > lib/automake.mk| 1 +
> > lib/dpif-netdev-private-hwol.h | 63 ++
> > lib/dpif-netdev-private.h | 1 +
> > lib/dpif-netdev.c | 38 ++--
> > 4 files changed, 67 insertions(+), 36 deletions(-)
> > create mode 100644 lib/dpif-netdev-private-hwol.h
> >
> > diff --git a/lib/automake.mk b/lib/automake.mk
> > index fdba3c6c0..3a33cdd5c 100644
> > --- a/lib/automake.mk
> > +++ b/lib/automake.mk
> > @@ -115,6 +115,7 @@ lib_libopenvswitch_la_SOURCES = \
> > lib/dpif-netdev-private-dfc.h \
> > lib/dpif-netdev-private-dpcls.h \
> > lib/dpif-netdev-private-flow.h \
> > + lib/dpif-netdev-private-hwol.h \
> > lib/dpif-netdev-private-thread.h \
> > lib/dpif-netdev-private.h \
> > lib/dpif-netdev-perf.c \
> > diff --git a/lib/dpif-netdev-private-hwol.h b/lib/dpif-netdev-private-hwol.h
> > new file mode 100644
> > index 0..b93297a74
> > --- /dev/null
> > +++ b/lib/dpif-netdev-private-hwol.h
> > @@ -0,0 +1,63 @@
> > +/*
> > + * Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013, 2015 Nicira, Inc.
> > + * Copyright (c) 2021 Intel Corporation.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + * http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#ifndef DPIF_NETDEV_PRIVATE_HWOL_H
> > +#define DPIF_NETDEV_PRIVATE_HWOL_H 1
> > +
> > +#include "dpif-netdev-private-flow.h"
> > +
> > +#define MAX_FLOW_MARK (UINT32_MAX - 1)
> > +#define INVALID_FLOW_MARK 0
> > +/* Zero flow mark is used to indicate the HW to remove the mark. A packet
> > + * marked with zero mark is received in SW without a mark at all, so it
> > + * cannot be used as a valid mark.
> > + */
> > +
> > +struct megaflow_to_mark_data {
> > +const struct cmap_node node;
> > +ovs_u128 mega_ufid;
> > +uint32_t mark;
> > +};
> > +
> > +struct flow_mark {
> > +struct cmap megaflow_to_mark;
> > +struct cmap mark_to_flow;
> > +struct id_pool *pool;
> > +};
> > +
> > +/* allocated in dpif-netdev.c */
> > +extern struct flow_mark flow_mark;
> > +
> > +static inline struct dp_netdev_flow *
> > +mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd,
> > + const uint32_t mark)
> > +{
> > +struct dp_netdev_flow *flow;
> > +
> > +CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0),
> > + _mark.mark_to_flow) {
> > +if (flow->mark == mark && flow->pmd_id == pmd->core_id &&
> > +flow->dead == false) {
> > +return flow;
> > +}
> > +}
> > +
> > +return NULL;
Re: [ovs-dev] [v14 01/11] dpif-netdev: Refactor to multiple header files.
Hi Flavio, Thanks for the info. My response is inline. Cian > -Original Message- > From: Flavio Leitner > Sent: Tuesday 6 July 2021 20:36 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v14 01/11] dpif-netdev: Refactor to multiple header > files. > > On Tue, Jul 06, 2021 at 04:20:59PM -0300, Flavio Leitner wrote: > > > > Hi, > > > > I was reviewing the patch while testing and I can consistently > > loss 1Mpps (or more) on a P2P scenario with this flow table: > > ovs-ofctl add-flow br0 in_port=dpdk0,actions=output:dpdk1 > > > > TX: 14Mpps > > RX without patch: +12.6Mpps > > RX with patch: 11.67Mpps > > FYI: the performance is consistently recovered with patch 03. > fbl > Good that the performance is recovered. This is probably a compiler-inlining behaviour change due to files moving around (and compiler seeing "re-use" of functions, resulting in not being inlined). As the movement of inline functions takes its final shape, the compiler identifies better inlining again, resulting in the same performance as before patch 01. (Note that using -O3 instead of e.g. -O2 causes more aggressive inlining, and may continually have best performance). Since the performance is recovered in patch 03, this should be acceptable. > > CPU: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz > > > > Perf diff: > > # Event 'cycles' > > # > > # Baseline Delta Abs Shared Object Symbol > > # . .. > > .. > > # > > 8.32% -2.64% libc-2.28.so[.] __memcmp_avx2_movbe > >+2.04% ovs-vswitchd[.] > > dp_netdev_pmd_flush_output_packets.part.41 > > 14.60% -1.78% ovs-vswitchd[.] mlx5_rx_burst_vec > > 2.78% +1.78% ovs-vswitchd[.] non_atomic_ullong_add > > 23.95% +1.60% ovs-vswitchd[.] miniflow_extract > > 2.02% +1.54% ovs-vswitchd[.] netdev_dpdk_rxq_recv > > 14.77% -0.82% ovs-vswitchd[.] dp_netdev_input__ > > 5.46% +0.79% ovs-vswitchd[.] mlx5_tx_burst_none_empw > > 2.79% -0.77% ovs-vswitchd[.] > > dp_netdev_pmd_flush_output_on_port > > 3.70% -0.58% ovs-vswitchd[.] dp_execute_output_action > > 3.34% -0.58% ovs-vswitchd[.] netdev_send > > 2.92% +0.41% ovs-vswitchd[.] dp_netdev_process_rxq_port > > 0.36% +0.39% ovs-vswitchd[.] netdev_dpdk_vhost_rxq_recv > > 4.25% +0.38% ovs-vswitchd[.] > > mlx5_tx_handle_completion.isra.49 > > 0.82% +0.30% ovs-vswitchd[.] pmd_perf_end_iteration > > 1.53% -0.12% ovs-vswitchd[.] netdev_dpdk_filter_packet_len > > 0.53% -0.11% ovs-vswitchd[.] netdev_is_flow_api_enabled > > 0.54% -0.10% [vdso] [.] 0x09c0 > > 1.72% +0.10% ovs-vswitchd[.] netdev_rxq_recv > > 0.61% +0.09% ovs-vswitchd[.] pmd_thread_main > > 0.08% +0.07% ovs-vswitchd[.] userspace_tso_enabled > > 0.45% -0.07% ovs-vswitchd[.] memcmp@plt > > 0.22% -0.05% ovs-vswitchd[.] dp_execute_cb > > > > > > > > On Thu, Jul 01, 2021 at 04:06:09PM +0100, Cian Ferriter wrote: > > > From: Harry van Haaren > > > > > > Split the very large file dpif-netdev.c and the datastructures > > > it contains into multiple header files. Each header file is > > > responsible for the datastructures of that component. > > > > > > This logical split allows better reuse and modularity of the code, > > > and reduces the very large file dpif-netdev.c to be more managable. > > > > > > Due to dependencies between components, it is not possible to > > > move component in smaller granularities than this patch. > > > > > > To explain the dependencies better, eg: > > > > > > DPCLS has no deps (from dpif-netdev.c file) > > > FLOW depends on DPCLS (struct dpcls_rule) > > > DFC depends on DPCLS (netdev_flow_key) and FLOW (netdev_flow_key) > > > THREAD depends on DFC (struct dfc_cache) > > > > > > DFC_PROC depends on THREAD (struct pmd_thread) > > > > > > DPCLS lookup.h/c require only DPCLS > > > DPCLS implementations require only dpif-netdev-lookup.h. > > > - This change was made in 2.12 release with function pointers > > > - This commit only refactors the nam
Re: [ovs-dev] [v13 06/12] dpif-netdev: Add command to get dpif implementations.
Hi Eelco, Thanks for you comment. My reply is inline. Cian > -Original Message- > From: Eelco Chaudron > Sent: Tuesday 29 June 2021 09:25 > To: Ferriter, Cian ; Van Haaren, Harry > > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org; Flavio Leitner > > Subject: Re: [ovs-dev] [v13 06/12] dpif-netdev: Add command to get dpif > implementations. > > > > On 17 Jun 2021, at 18:18, Cian Ferriter wrote: > > > From: Harry van Haaren > > > > This commit adds a new command to retrieve the list of available > > DPIF implementations. This can be used by to check what implementations > > of the DPIF are available in any given OVS binary. > > > > Usage: > > $ ovs-appctl dpif-netdev/dpif-get > > > > This command only shows the actual available options, not the currently > selected ones. > > I think from a support perspective, we need this command to also show the > current setting for each > datapath. Having this only in the log will not work in cases where the log is > overwritten. Yes, good idea. More visibility here is a good suggestion, rather than fishing through the logs to see what DPIF is set. > > //Eelco > > Guess this also is true for the MFEX patchset (will reply during my review) > and the already included > “ovs-appctl dpif-netdev/subtable-lookup-prio-get” > > I'll improve the DPIF get command as per your suggestion. I'm following the thread where the DPIF, MFEX and DPCLS get commands are being discussed. I'll take the output of this thread into account for the next revision of the dpif-impl-get command. Thread: https://mail.openvswitch.org/pipermail/ovs-dev/2021-June/384644.html ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif implementation.
Hi Eelco,
Thanks for your comments. My response is inline.
Cian
> -Original Message-
> From: Eelco Chaudron
> Sent: Friday 25 June 2021 16:14
> To: Ferriter, Cian
> Cc: Flavio Leitner ; ovs-dev@openvswitch.org;
> i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif
> implementation.
>
>
>
> On 24 Jun 2021, at 13:53, Ferriter, Cian wrote:
>
> > Hi Flavio,
> >
> > Thanks for the review. My responses are inline.
> >
> > Cian
> >
>
>
>
> >>>
> >>> +static void
> >>> +dpif_netdev_impl_set(struct unixctl_conn *conn, int argc,
> >>> + const char *argv[], void *aux OVS_UNUSED)
> >>> +{
> >>> +/* This function requires just one parameter, the DPIF name.
> >>> + * A second optional parameter can identify the datapath instance.
> >>> + */
> >>> +const char *dpif_name = argv[1];
> >>> +
> >>> +static const char *error_description[2] = {
> >>> +"Unknown DPIF implementation",
> >>> +"CPU doesn't support the required instruction for",
> >>> +};
> >>> +
> >>> +dp_netdev_input_func new_func;
> >>> +int32_t err = dp_netdev_impl_get_by_name(dpif_name, _func);
> >>> +if (err) {
> >>> +struct ds reply = DS_EMPTY_INITIALIZER;
> >>> +ds_put_format(, "DPIF implementation not available: %s
> >>> %s.\n",
> >>> + error_description[ (err == -ENOTSUP) ], dpif_name);
> >>> +const char *reply_str = ds_cstr();
> >>> +unixctl_command_reply(conn, reply_str);
> >>> +VLOG_INFO("%s", reply_str);
> >>> +ds_destroy();
> >>> +return;
> >>> +}
> >>> +
> >>> +/* argv[2] is optional datapath instance. If no datapath name is
> >>> provided
> >>> + * and only one datapath exists, the one existing datapath is
> >>> reprobed.
> >>> + */
> >>> +ovs_mutex_lock(_netdev_mutex);
> >>> +struct dp_netdev *dp = NULL;
> >>> +
> >>> +if (argc == 3) {
> >>> +dp = shash_find_data(_netdevs, argv[2]);
> >>> +} else if (shash_count(_netdevs) == 1) {
> >>> +dp = shash_first(_netdevs)->data;
> >>> +}
> >>> +
> >>> +if (!dp) {
> >>> +ovs_mutex_unlock(_netdev_mutex);
> >>> +unixctl_command_reply_error(conn,
> >>> +"please specify an existing
> >>> datapath");
> >>> +return;
> >>> +}
> >>> +
> >>> +/* Get PMD threads list. */
> >>> +size_t n;
> >>> +struct dp_netdev_pmd_thread **pmd_list;
> >>> +sorted_poll_thread_list(dp, _list, );
> >>> +
> >>> +for (size_t i = 0; i < n; i++) {
> >>> +struct dp_netdev_pmd_thread *pmd = pmd_list[i];
> >>> +if (pmd->core_id == NON_PMD_CORE_ID) {
> >>> +continue;
> >>> +}
> >>> +
> >>> +/* Set PMD threads DPIF implementation to requested one. */
> >>> +pmd->netdev_input_func = *new_func;
> >>> +};
> >>> +
> >>> +ovs_mutex_unlock(_netdev_mutex);
> >>> +
> >>> +/* Set the default implementation for PMD threads created in the
> >>> future. */
> >>> +dp_netdev_impl_set_default(*new_func);
> >>
> >> I checked other patches and it seems this interface could be simplified
> >> and would fix the set_default() above to be more robust.
> >> What do you think of:
> >>
> >>1) lock dp_netdev_mutex
> >>2) Check if the DP argument is valid.
> >>3) Use a new dp_netdev_impl_set_default_by_name()
> >> That fails if the name is not available or set the default input
> >> hiding the function pointer from outside.
> >>4) Loop on each pmd doing the same as in dp_netdev_configure_pmd():
> >> pmd->netdev_input_func = dp_netdev_impl_get_default();
> >>5) unlock dp_netdev_mutex
> >>
> >> It will hold the lock a bit more time but we don't expect to have
> >> several inputs and no frequent input changes, so we should be fine.
> >>
> >
> > Good idea. Hiding the function pointer from here is a nice improvement.
> > I'll rework it to do this.
>
>
> Do we also assume that setting the function pointer happens atomically on all
> supported architectures?
> I would assume this requires an atomic set?
>
This is a good point. We want an atomic set here. I will push the below fix
with the next version of the patchset.
-pmd->netdev_input_func = dp_netdev_impl_get_default();
+dp_netdev_input_func default_func = dp_netdev_impl_get_default();
+atomic_uintptr_t *pmd_func = (void *)>netdev_input_func;
+atomic_store_relaxed(pmd_func, (uintptr_t)default_func);
> //Eelco
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif implementation.
Hi Eelco, Thanks for the comments. My responses are inline. Cian > -Original Message- > From: Eelco Chaudron > Sent: Monday 28 June 2021 15:35 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org; Flavio Leitner > ; Van Haaren, Harry > > Subject: Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif > implementation. > > > > On 17 Jun 2021, at 18:18, Cian Ferriter wrote: > > > From: Harry van Haaren > > > > This commit adds a new command to allow the user to switch > > the active DPIF implementation at runtime. A probe function > > is executed before switching the DPIF implementation, to ensure > > the CPU is capable of running the ISA required. For example, the > > below code will switch to the AVX512 enabled DPIF assuming > > that the runtime CPU is capable of running AVX512 instructions: > > > > $ ovs-appctl dpif-netdev/dpif-set dpif_avx512 > > > > A new configuration flag is added to allow selection of the > > default DPIF. This is useful for running the unit-tests against > > the available DPIF implementations, without modifying each unit test. > > > > The design of the testing & validation for ISA optimized DPIF > > implementations is based around the work already upstream for DPCLS. > > Note however that a DPCLS lookup has no state or side-effects, allowing > > the auto-validator implementation to perform multiple lookups and > > provide consistent statistic counters. > > > > The DPIF component does have state, so running two implementations in > > parallel and comparing output is not a valid testing method, as there > > are changes in DPIF statistic counters (side effects). As a result, the > > DPIF is tested directly against the unit-tests. > > > > Signed-off-by: Harry van Haaren > > Co-authored-by: Cian Ferriter > > Signed-off-by: Cian Ferriter > > > > --- > > > > v13: > > - Add Docs items about the switch DPIF command here rather than in > > later commit. > > - Document operation in manpages as well as rST. > > - Minor code refactoring to address review comments. > > --- > > Documentation/topics/dpdk/bridge.rst | 34 + > > acinclude.m4 | 15 > > configure.ac | 1 + > > lib/automake.mk | 1 + > > lib/dpif-netdev-avx512.c | 14 > > lib/dpif-netdev-private-dpif.c | 103 +++ > > lib/dpif-netdev-private-dpif.h | 49 - > > lib/dpif-netdev-private-thread.h | 11 +-- > > lib/dpif-netdev-unixctl.man | 3 + > > lib/dpif-netdev.c| 89 +-- > > 10 files changed, 304 insertions(+), 16 deletions(-) > > create mode 100644 lib/dpif-netdev-private-dpif.c > > > > diff --git a/Documentation/topics/dpdk/bridge.rst > > b/Documentation/topics/dpdk/bridge.rst > > index 526d5c959..fafa8c821 100644 > > --- a/Documentation/topics/dpdk/bridge.rst > > +++ b/Documentation/topics/dpdk/bridge.rst > > @@ -214,3 +214,37 @@ implementation :: > > > > Compile OVS in debug mode to have `ovs_assert` statements error out if > > there is a mis-match in the DPCLS lookup implementation. > > + > > +Datapath Interface Performance > > +-- > > + > > +The datapath interface (DPIF) or dp_netdev_input() is responsible for > > taking > > +packets through the major components of the userspace datapath; such as > > +miniflow_extract, EMC, SMC and DPCLS lookups, and a lot of the performance > > +stats associated with the datapath. > > + > > +Just like with the SIMD DPCLS feature above, SIMD can be applied to the > > DPIF to > > +improve performance. > > + > > +By default, dpif_scalar is used. The DPIF implementation can be selected by > > +name :: > > + > > +$ ovs-appctl dpif-netdev/dpif-set dpif_avx512 > > +DPIF implementation set to dpif_avx512. > > + > > +$ ovs-appctl dpif-netdev/dpif-set dpif_scalar > > +DPIF implementation set to dpif_scalar. > > + > > +Running Unit Tests with AVX512 DPIF > > +~~~ > > + > > +Since the AVX512 DPIF is disabled by default, a compile time option is > > +available in order to test it with the OVS unit test suite. When building > > with > > +a CPU that supports AVX512, use the following configure option :: > > + > > +$ ./configure --enable-dpif-default-avx512 > > + >
Re: [ovs-dev] [PATCH V7 06/13] dpif-netdev: Add HW miss packet state recover logic.
Hi Eli,
I have some concerns about how we plug the packet state recover logic into
dfc_processing() here. My concerns are inline below.
I'm concerned that we are hurting performance in the partial HWOL case, as this
patchset is introducing new direct (non-inline) function calls per packet to
the software datapath. We can reduce performance impact in this area, see
specific suggestions below.
Thanks,
Cian
> -Original Message-
> From: Eli Britstein
> Sent: Wednesday 23 June 2021 16:53
> To: d...@openvswitch.org; Ilya Maximets
> Cc: Gaetan Rivet ; Majd Dibbiny ;
> Sriharsha Basavapatna
> ; Hemal Shah ;
> Ivan Malov
> ; Ferriter, Cian ; Eli
> Britstein ;
> Finn, Emma ; Kovacevic, Marko
> Subject: [PATCH V7 06/13] dpif-netdev: Add HW miss packet state recover logic.
>
> Recover the packet if it was partially processed by the HW. Fallback to
> lookup flow by mark association.
>
> Signed-off-by: Eli Britstein
> Reviewed-by: Gaetan Rivet
> Acked-by: Sriharsha Basavapatna
> Tested-by: Emma Finn
> Tested-by: Marko Kovacevic
> Signed-off-by: Ilya Maximets
> ---
> lib/dpif-netdev.c | 45 +
> 1 file changed, 41 insertions(+), 4 deletions(-)
>
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 8fa7eb6d4..d5b7d5b73 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -114,6 +114,7 @@ COVERAGE_DEFINE(datapath_drop_invalid_port);
> COVERAGE_DEFINE(datapath_drop_invalid_bond);
> COVERAGE_DEFINE(datapath_drop_invalid_tnl_port);
> COVERAGE_DEFINE(datapath_drop_rx_invalid_packet);
> +COVERAGE_DEFINE(datapath_drop_hw_miss_recover);
>
> /* Protects against changes to 'dp_netdevs'. */
> static struct ovs_mutex dp_netdev_mutex = OVS_MUTEX_INITIALIZER;
> @@ -7062,6 +7063,39 @@ smc_lookup_batch(struct dp_netdev_pmd_thread *pmd,
> pmd_perf_update_counter(>perf_stats, PMD_STAT_SMC_HIT, n_smc_hit);
> }
>
> +static struct tx_port * pmd_send_port_cache_lookup(
> +const struct dp_netdev_pmd_thread *pmd, odp_port_t port_no);
> +
> +static inline int
> +dp_netdev_hw_flow(const struct dp_netdev_pmd_thread *pmd,
> + odp_port_t port_no,
> + struct dp_packet *packet,
> + struct dp_netdev_flow **flow)
> +{
> +struct tx_port *p;
> +uint32_t mark;
> +
Start of full HWOL recovery block
> +/* Restore the packet if HW processing was terminated before completion.
> */
> +p = pmd_send_port_cache_lookup(pmd, port_no);
> +if (OVS_LIKELY(p)) {
> +int err = netdev_hw_miss_packet_recover(p->port->netdev, packet);
> +
> +if (err && err != EOPNOTSUPP) {
> +COVERAGE_INC(datapath_drop_hw_miss_recover);
> +return -1;
> +}
> +}
End of full HWOL recovery block
IIUC, the above is only relevant to full HWOL where the packet is partially
processed by the HW. In a partial HWOL testcase, we see a performance drop as a
result of the above code. The performance after the patchset is applied is
0.94x what the performance was before.
We should branch over this code in the partial HWOL scenario, where we don't
need to get the call to pmd_send_port_cache_lookup() and
netdev_hw_miss_packet_recover(). We want this branch to be transparent to the
user. Since both full and partial HWOL falls under the
other_config:hw-offload=true switch, we might need a configure time check NIC
capabilities solution or something similar to branch over full HWOL packet
recovery code. Does this make sense?
perf top showing cycles spent per function in my partial HWOL scenario. We can
see netdev_hw_miss_packet_recover(),
netdev_offload_dpdk_hw_miss_packet_recover() and netdev_is_flow_api_enabled()
taking cycles:
28.79% pmd-c01/id:8 ovs-vswitchd[.] dp_netdev_input__
13.72% pmd-c01/id:8 ovs-vswitchd[.] parse_tcp_flags
11.07% pmd-c01/id:8 ovs-vswitchd[.] i40e_recv_pkts_vec_avx2
10.94% pmd-c01/id:8 ovs-vswitchd[.] i40e_xmit_fixed_burst_vec_avx2
7.48% pmd-c01/id:8 ovs-vswitchd[.] cmap_find
4.94% pmd-c01/id:8 ovs-vswitchd[.] netdev_hw_miss_packet_recover
4.77% pmd-c01/id:8 ovs-vswitchd[.] dp_execute_output_action
3.81% pmd-c01/id:8 ovs-vswitchd[.]
dp_netdev_pmd_flush_output_on_port
3.40% pmd-c01/id:8 ovs-vswitchd[.] netdev_send
2.49% pmd-c01/id:8 ovs-vswitchd[.] netdev_dpdk_eth_send
1.16% pmd-c01/id:8 ovs-vswitchd[.] netdev_dpdk_rxq_recv
0.90% pmd-c01/id:8 ovs-vswitchd[.] pmd_perf_end_iteration
0.75% pmd-c01/id:8 ovs-vswitchd[.] dp_netdev_process_rxq_port
0.68% pmd-c01/id:8 ovs-vswitchd[.] netdev_is_flow_api_enabled
0.55% pmd-c01/id
Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif implementation.
Hi Flavio,
Just one correction I noticed while implementing your feedback. I've replied
inline.
Thanks,
Cian
> -Original Message-
> From: Ferriter, Cian
> Sent: Thursday 24 June 2021 12:54
> To: Flavio Leitner
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: RE: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif
> implementation.
>
> Hi Flavio,
>
> Thanks for the review. My responses are inline.
>
> Cian
>
> > -Original Message-
> > From: Flavio Leitner
> > Sent: Wednesday 23 June 2021 19:18
> > To: Ferriter, Cian
> > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> > Subject: Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif
> > implementation.
> >
> > On Thu, Jun 17, 2021 at 05:18:18PM +0100, Cian Ferriter wrote:
> > > From: Harry van Haaren
> > >
> > > +
> > > +if (!avx512f_available || !bmi2_available) {
> > > +return -ENOTSUP;
> >
> > Also the callers of the probe function only cares about true or
> > false, so there is no need to return errno here, then the new
> > include can also be removed.
> >
>
> I'll simplify the return and remove the errno include.
>
This return -ENOTSUP is used in dpif_netdev_impl_set() to select the correct
error message to show the user on error.
static const char *error_description[2] = {
"Unknown DPIF implementation",
"CPU doesn't support the required instruction for",
};
I'm going to leave in. I hope that makes sense.
> > > +}
> > > +
> > > +return 0;
> > > +}
> > > +
> > > int32_t
> > > dp_netdev_input_outer_avx512(struct dp_netdev_pmd_thread *pmd,
> > > struct dp_packet_batch *packets,
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif implementation.
Hi Flavio,
Thanks for clarifying. My response is inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Thursday 24 June 2021 13:26
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif
> implementation.
>
> On Thu, Jun 24, 2021 at 11:53:34AM +, Ferriter, Cian wrote:
> [...]
> > > On Thu, Jun 17, 2021 at 05:18:18PM +0100, Cian Ferriter wrote:
> > > > +
> > > > +VLOG_DEFINE_THIS_MODULE(dpif_netdev_impl);
> > > > +
> > > > +/* Actual list of implementations goes here. */
> > > > +static struct dpif_netdev_impl_info_t dpif_impls[] = {
> > > > +/* The default scalar C code implementation. */
> > > > +{ .func = dp_netdev_input,
> > >
> > > The '.func' is too generic. Can we rename this to something that
> > > relates to 'input'?
> > >
> >
> > I'll rename to 'input_func'. Does that sound good to you?
>
> Sounds better, indeed.
>
> [..]
> > > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > > > index 1f15af882..9c234ef3d 100644
> > > > --- a/lib/dpif-netdev.c
> > > > +++ b/lib/dpif-netdev.c
> > > > @@ -479,8 +479,8 @@ static void dp_netdev_execute_actions(struct
> > > > dp_netdev_pmd_thread *pmd,
> > > >const struct flow *flow,
> > > >const struct nlattr *actions,
> > > >size_t actions_len);
> > > > -static int32_t dp_netdev_input(struct dp_netdev_pmd_thread *,
> > > > -struct dp_packet_batch *, odp_port_t
> > > > port_no);
> > > > +int32_t dp_netdev_input(struct dp_netdev_pmd_thread *,
> > > > +struct dp_packet_batch *, odp_port_t port_no);
> > >
> > >
> > > All other functions around are static and this one is now part of
> > > dpif-netdev-private-dpif.h which is included by
> > > dpif-netdev-private-thread.h as part of dpif-netdev-private.h.
> > >
> > > Perhaps fixing that header issue I reported in the first patch
> > > helps to solve this too.
> > >
> >
> > This function can't be static. We need it to be available in
> > lib/dpif-netdev-private-dpif.c to register it as the DPIF function
> > for the dpif_scalar dpif_impl. I hope that makes sense.
>
> Sure, it can't be. What I am saying is that it is declared in two
> different places. One in dpif-netdev.c and another in the header
> dpif-netdev-private-dpif.h which is included as well.
>
My apologies, I understand now. I'll fix this. I'm going to remove the
lib/dpif-netdev.c declaration of dp_netdev_input().
> Thanks,
> --
> fbl
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 06/12] dpif-netdev: Add command to get dpif implementations.
Hi Flavio,
Thanks for the review. My responses are inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Wednesday 23 June 2021 20:21
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 06/12] dpif-netdev: Add command to get dpif
> implementations.
>
> On Thu, Jun 17, 2021 at 05:18:19PM +0100, Cian Ferriter wrote:
> > From: Harry van Haaren
> >
> > This commit adds a new command to retrieve the list of available
> > DPIF implementations. This can be used by to check what implementations
> > of the DPIF are available in any given OVS binary.
> >
> > Usage:
> > $ ovs-appctl dpif-netdev/dpif-get
>
> I didn't mention this in the dpif-set but it would be great
> to have a more targeted command name, like dpif-impl-{get,set}.
>
I'll rename the commands to the above format, thanks for the suggestion.
> >
> > Signed-off-by: Harry van Haaren
> >
> > ---
> >
> > v13:
> > - Add NEWS item about DPIF get and set commands here rather than in a
> > later commit.
> > - Add documentation items about DPIF set commands here rather than in a
> > later commit.
> > ---
> > Documentation/topics/dpdk/bridge.rst | 8
> > NEWS | 1 +
> > lib/dpif-netdev-private-dpif.c | 8
> > lib/dpif-netdev-private-dpif.h | 6 ++
> > lib/dpif-netdev-unixctl.man | 3 +++
> > lib/dpif-netdev.c| 24
> > 6 files changed, 50 insertions(+)
> >
> > diff --git a/Documentation/topics/dpdk/bridge.rst
> > b/Documentation/topics/dpdk/bridge.rst
> > index fafa8c821..f59e26cbe 100644
> > --- a/Documentation/topics/dpdk/bridge.rst
> > +++ b/Documentation/topics/dpdk/bridge.rst
> > @@ -226,6 +226,14 @@ stats associated with the datapath.
> > Just like with the SIMD DPCLS feature above, SIMD can be applied to the
> > DPIF to
> > improve performance.
> >
> > +OVS provides multiple implementations of the DPIF. The available
> > +implementations can be listed with the following command ::
> > +
> > +$ ovs-appctl dpif-netdev/dpif-get
> > +Available DPIF implementations:
> > + dpif_scalar
> > + dpif_avx512
> > +
> > By default, dpif_scalar is used. The DPIF implementation can be selected by
> > name ::
> >
> > diff --git a/NEWS b/NEWS
> > index 6a4a7b76d..c47ab349e 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -12,6 +12,7 @@ Post-v2.15.0
> > * Refactor lib/dpif-netdev.c to multiple header files.
> > * Add avx512 implementation of dpif which can process non recirculated
> > packets. It supports partial HWOL, EMC, SMC and DPCLS lookups.
> > + * Add commands to get and set the dpif implementations.
> > - ovs-ctl:
> > * New option '--no-record-hostname' to disable hostname configuration
> > in ovsdb on startup.
> > diff --git a/lib/dpif-netdev-private-dpif.c b/lib/dpif-netdev-private-dpif.c
> > index d829a7ee5..3649e775d 100644
> > --- a/lib/dpif-netdev-private-dpif.c
> > +++ b/lib/dpif-netdev-private-dpif.c
> > @@ -73,6 +73,14 @@ dp_netdev_impl_set_default(dp_netdev_input_func func)
> > default_dpif_func = func;
> > }
> >
> > +uint32_t
> > +dp_netdev_impl_get(const struct dpif_netdev_impl_info_t **out_impls)
> > +{
> > +ovs_assert(out_impls);
> > +*out_impls = dpif_impls;
> > +return ARRAY_SIZE(dpif_impls);
> > +}
> > +
>
> This could receive struct ds and fill with the internal details
> to keep internal details in private-dpif.c
>
>
Yes, hiding the details is a good suggestion. I'll fix this.
> > /* This function checks all available DPIF implementations, and selects the
> > * returns the function pointer to the one requested by "name".
> > */
> > diff --git a/lib/dpif-netdev-private-dpif.h b/lib/dpif-netdev-private-dpif.h
> > index a6db3c7f2..717e9e2f9 100644
> > --- a/lib/dpif-netdev-private-dpif.h
> > +++ b/lib/dpif-netdev-private-dpif.h
> > @@ -48,6 +48,12 @@ struct dpif_netdev_impl_info_t {
> > const char *name;
> > };
> >
> > +/* This function returns all available implementations to the caller. The
> > + * quantity of implementations is returned by the int return value.
> > + */
> > +uint32_t
> > +dp_netdev_impl_get(const struct dpif_netdev_impl_info_t **out_impls);
> > +
> > /* This function checks all available DPIF implementations, and s
Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif implementation.
Hi Flavio, Thanks for the review. My responses are inline. Cian > -Original Message- > From: Flavio Leitner > Sent: Wednesday 23 June 2021 19:18 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v13 05/12] dpif-netdev: Add command to switch dpif > implementation. > > On Thu, Jun 17, 2021 at 05:18:18PM +0100, Cian Ferriter wrote: > > From: Harry van Haaren > > > > This commit adds a new command to allow the user to switch > > the active DPIF implementation at runtime. A probe function > > is executed before switching the DPIF implementation, to ensure > > the CPU is capable of running the ISA required. For example, the > > below code will switch to the AVX512 enabled DPIF assuming > > that the runtime CPU is capable of running AVX512 instructions: > > > > $ ovs-appctl dpif-netdev/dpif-set dpif_avx512 > > > > A new configuration flag is added to allow selection of the > > default DPIF. This is useful for running the unit-tests against > > the available DPIF implementations, without modifying each unit test. > > > > The design of the testing & validation for ISA optimized DPIF > > implementations is based around the work already upstream for DPCLS. > > Note however that a DPCLS lookup has no state or side-effects, allowing > > the auto-validator implementation to perform multiple lookups and > > provide consistent statistic counters. > > > > The DPIF component does have state, so running two implementations in > > parallel and comparing output is not a valid testing method, as there > > are changes in DPIF statistic counters (side effects). As a result, the > > DPIF is tested directly against the unit-tests. > > > > Signed-off-by: Harry van Haaren > > Co-authored-by: Cian Ferriter > > Signed-off-by: Cian Ferriter > > > > --- > > > > v13: > > - Add Docs items about the switch DPIF command here rather than in > > later commit. > > - Document operation in manpages as well as rST. > > - Minor code refactoring to address review comments. > > --- > > Documentation/topics/dpdk/bridge.rst | 34 + > > acinclude.m4 | 15 > > configure.ac | 1 + > > lib/automake.mk | 1 + > > lib/dpif-netdev-avx512.c | 14 > > lib/dpif-netdev-private-dpif.c | 103 +++ > > lib/dpif-netdev-private-dpif.h | 49 - > > lib/dpif-netdev-private-thread.h | 11 +-- > > lib/dpif-netdev-unixctl.man | 3 + > > lib/dpif-netdev.c| 89 +-- > > 10 files changed, 304 insertions(+), 16 deletions(-) > > create mode 100644 lib/dpif-netdev-private-dpif.c > > > > diff --git a/Documentation/topics/dpdk/bridge.rst > > b/Documentation/topics/dpdk/bridge.rst > > index 526d5c959..fafa8c821 100644 > > --- a/Documentation/topics/dpdk/bridge.rst > > +++ b/Documentation/topics/dpdk/bridge.rst > > @@ -214,3 +214,37 @@ implementation :: > > > > Compile OVS in debug mode to have `ovs_assert` statements error out if > > there is a mis-match in the DPCLS lookup implementation. > > + > > +Datapath Interface Performance > > +-- > > + > > +The datapath interface (DPIF) or dp_netdev_input() is responsible for > > taking > > +packets through the major components of the userspace datapath; such as > > +miniflow_extract, EMC, SMC and DPCLS lookups, and a lot of the performance > > +stats associated with the datapath. > > + > > +Just like with the SIMD DPCLS feature above, SIMD can be applied to the > > DPIF to > > +improve performance. > > + > > +By default, dpif_scalar is used. The DPIF implementation can be selected by > > +name :: > > + > > +$ ovs-appctl dpif-netdev/dpif-set dpif_avx512 > > +DPIF implementation set to dpif_avx512. > > + > > +$ ovs-appctl dpif-netdev/dpif-set dpif_scalar > > +DPIF implementation set to dpif_scalar. > > + > > +Running Unit Tests with AVX512 DPIF > > +~~~ > > + > > +Since the AVX512 DPIF is disabled by default, a compile time option is > > +available in order to test it with the OVS unit test suite. When building > > with > > +a CPU that supports AVX512, use the following configure option :: > > + > > +$ ./configure --enable-dpif-default-avx512 > > + > > +The following line should be seen in the
Re: [ovs-dev] [v13 04/12] dpif-avx512: Add ISA implementation of dpif.
Hi Flavio, Thanks for the testing here. My responses are inline. Cian > -Original Message- > From: Flavio Leitner > Sent: Thursday 24 June 2021 05:06 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; Amber, Kumar ; > i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v13 04/12] dpif-avx512: Add ISA implementation of > dpif. > > On Thu, Jun 17, 2021 at 05:18:17PM +0100, Cian Ferriter wrote: > > From: Harry van Haaren > > > > This commit adds the AVX512 implementation of DPIF functionality, > > specifically the dp_netdev_input_outer_avx512 function. This function > > only handles outer (no re-circulations), and is optimized to use the > > AVX512 ISA for packet batching and other DPIF work. > > > > Sparse is not able to handle the AVX512 intrinsics, causing compile > > time failures, so it is disabled for this file. > > > > Signed-off-by: Harry van Haaren > > Co-authored-by: Cian Ferriter > > Signed-off-by: Cian Ferriter > > Co-authored-by: Kumar Amber > > Signed-off-by: Kumar Amber > > > > --- > > > > v13: > > - Squash "Add HWOL support" commit into this commit. > > - Add NEWS item about this feature here rather than in a later commit. > > - Add #define NUM_U64_IN_ZMM_REG 8. > > - Add comment describing operation of while loop handling HWOL->EMC->SMC > > lookups in dp_netdev_input_outer_avx512(). > > - Add EMC and SMC batch insert functions for better handling of EMC and > > SMC in AVX512 DPIF. > > - Minor code refactor to address review comments. > > --- > > NEWS | 2 + > > lib/automake.mk | 5 +- > > lib/dpif-netdev-avx512.c | 327 +++ > > lib/dpif-netdev-private-dfc.h| 25 +++ > > lib/dpif-netdev-private-dpif.h | 32 +++ > > lib/dpif-netdev-private-thread.h | 11 +- > > lib/dpif-netdev-private.h| 25 +++ > > lib/dpif-netdev.c| 103 -- > > 8 files changed, 514 insertions(+), 16 deletions(-) > > create mode 100644 lib/dpif-netdev-avx512.c > > create mode 100644 lib/dpif-netdev-private-dpif.h > > > > diff --git a/NEWS b/NEWS > > index 96b3a61c8..6a4a7b76d 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -10,6 +10,8 @@ Post-v2.15.0 > > * Auto load balancing of PMDs now partially supports cross-NUMA > > polling > > cases, e.g if all PMD threads are running on the same NUMA node. > > * Refactor lib/dpif-netdev.c to multiple header files. > > + * Add avx512 implementation of dpif which can process non recirculated > > + packets. It supports partial HWOL, EMC, SMC and DPCLS lookups. > > - ovs-ctl: > > * New option '--no-record-hostname' to disable hostname configuration > > in ovsdb on startup. > > diff --git a/lib/automake.mk b/lib/automake.mk > > index 3a33cdd5c..660cd07f0 100644 > > --- a/lib/automake.mk > > +++ b/lib/automake.mk > > @@ -33,11 +33,13 @@ lib_libopenvswitchavx512_la_CFLAGS = \ > > -mavx512f \ > > -mavx512bw \ > > -mavx512dq \ > > + -mbmi \ > > -mbmi2 \ > > -fPIC \ > > $(AM_CFLAGS) > > lib_libopenvswitchavx512_la_SOURCES = \ > > - lib/dpif-netdev-lookup-avx512-gather.c > > + lib/dpif-netdev-lookup-avx512-gather.c \ > > + lib/dpif-netdev-avx512.c > > lib_libopenvswitchavx512_la_LDFLAGS = \ > > -static > > endif > > @@ -114,6 +116,7 @@ lib_libopenvswitch_la_SOURCES = \ > > lib/dpif-netdev-private-dfc.c \ > > lib/dpif-netdev-private-dfc.h \ > > lib/dpif-netdev-private-dpcls.h \ > > + lib/dpif-netdev-private-dpif.h \ > > lib/dpif-netdev-private-flow.h \ > > lib/dpif-netdev-private-hwol.h \ > > lib/dpif-netdev-private-thread.h \ > > diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c > > new file mode 100644 > > index 0..0e55b0be2 > > --- /dev/null > > +++ b/lib/dpif-netdev-avx512.c > > @@ -0,0 +1,327 @@ > > +/* > > + * Copyright (c) 2021 Intel Corporation. > > + * > > + * Licensed under the Apache License, Version 2.0 (the "License"); > > + * you may not use this file except in compliance with the License. > > + * You may obtain a copy of the License at: > > + * > > + * http://www.apache.org/licenses/LICENSE-2.0 > > + * > > + * Unless required by applicable law or agreed to in writing, software > > + * distributed under the License is distributed on an "AS IS" BASIS, &g
Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
> -Original Message-
> From: Ilya Maximets
> Sent: Wednesday 23 June 2021 16:25
> To: Ferriter, Cian ; Ilya Maximets
> ; Eli Britstein
> ; d...@openvswitch.org
> Cc: ivan.ma...@oktetlabs.ru; Ameer Mahagneh ; Majd Dibbiny
>
> Subject: Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
>
> On 6/23/21 5:18 PM, Ferriter, Cian wrote:
> >> -Original Message-----
> >> From: dev On Behalf Of Ferriter, Cian
> >> Sent: Wednesday 23 June 2021 13:38
> >> To: Ilya Maximets ; Eli Britstein ;
> >> d...@openvswitch.org
> >> Cc: ivan.ma...@oktetlabs.ru; Ameer Mahagneh ; Majd
> >> Dibbiny
> >> Subject: Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
> >>
> >> Hi all,
> >>
> >> As part of rebasing our AVX512 DPIF on this patchset, I tested this
> >> patchset with partial HWOL and
> I'm
> >> seeing strange behaviour.
> >>
> >> I'll report back more detailed findings soon, just wanted to mention this
> >> here as soon as I found
> the
> >> issue.
> >>
> >> Thanks,
> >> Cian
> >>
> >
> > More details on the issue I'm seeing:
> > I'm using Ilya's branch from Github:
> > https://github.com/igsilya/ovs/tree/tmp-vxlan-decap
> >
> > ~/ovs_scripts# $OVS_DIR/utilities/ovs-vsctl list Open_vSwitch
> > dpdk_version: "DPDK 20.11.1"
> > other_config: {dpdk-hugepage-dir="/mnt/huge", dpdk-init="true",
> > dpdk-lcore-mask="0x1", dpdk-
> socket-mem="2048,0", emc-insert-inv-prob="0", hw-offload="true",
> pmd-cpu-mask="0x2"}
> >
> > ~/ovs_scripts# $OVS_DIR/utilities/ovs-vsctl show
> > 31584ce5-09c1-44b3-ab27-1a0308d63fff
> > Bridge br0
> > datapath_type: netdev
> > Port br0
> > Interface br0
> > type: internal
> > Port phy0
> > Interface phy0
> > type: dpdk
> > options: {dpdk-devargs="5e:00.0"}
> >
> > ~/ovs_scripts# $OVS_DIR/utilities/ovs-ofctl dump-flows br0
> > cookie=0x0, duration=29.466s, table=0, n_packets=0, n_bytes=0,
> > in_port=phy0 actions=IN_PORT
> >
> > I'm expecting the flow to be partially offloaded, but I get a segfault when
> > using the above branch.
> More info on the segfault below:
> >
> > Thread 13 "pmd-c01/id:8" received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 0x7f9f72734700 (LWP 19327)]
> > 0x56163bf0d825 in set_error (error=0x0, type=RTE_FLOW_ERROR_TYPE_ATTR)
> > at lib/netdev-dpdk.h:84
> > (gdb) bt
> > #0 0x56163bf0d825 in set_error (error=0x0,
> > type=RTE_FLOW_ERROR_TYPE_ATTR) at lib/netdev-
> dpdk.h:84
> > #1 0x56163bf0d8d3 in netdev_dpdk_rte_flow_get_restore_info
> > (netdev=0x1bfc65c80, p=0x19033af00,
> info=0x7f9f72729a20, error=0x0) at lib/netdev-dpdk.h:119
> > #2 0x56163bf14da3 in netdev_offload_dpdk_hw_miss_packet_recover
> > (netdev=0x1bfc65c80,
> packet=0x19033af00) at lib/netdev-offload-dpdk.c:2133
> > #3 0x56163bde3662 in netdev_hw_miss_packet_recover
> > (netdev=0x1bfc65c80, packet=0x19033af00) at
> lib/netdev-offload.c:265
> > #4 0x56163bda19a9 in dp_netdev_hw_flow (pmd=0x7f9f72735010, port_no=2,
> > packet=0x19033af00,
> flow=0x7f9f72729b98) at lib/dpif-netdev.c:7087
> > #5 0x56163bda1c5c in dfc_processing (pmd=0x7f9f72735010,
> > packets_=0x7f9f727310d0,
> keys=0x7f9f7272c480, missed_keys=0x7f9f7272c370, batches=0x7f9f72729f60,
> n_batches=0x7f9f72730f70,
> flow_map=0x7f9f72729c50, n_flows=0x7f9f72730f78, index_map=0x7f9f72729c30 "",
> md_is_valid=false,
> port_no=2) at lib/dpif-netdev.c:7168
> > #6 0x56163bda2f3e in dp_netdev_input__ (pmd=0x7f9f72735010,
> > packets=0x7f9f727310d0,
> md_is_valid=false, port_no=2) at lib/dpif-netdev.c:7475
> > #7 0x56163bda3105 in dp_netdev_input (pmd=0x7f9f72735010,
> > packets=0x7f9f727310d0, port_no=2) at
> lib/dpif-netdev.c:7519
> > #8 0x56163bd9ab04 in dp_netdev_process_rxq_port (pmd=0x7f9f72735010,
> > rxq=0x56163fb3f610,
> port_no=2) at lib/dpif-netdev.c:4774
> > #9 0x56163bd9ee17 in pmd_thread_main (f_=0x7f9f72735010) at
> > lib/dpif-netdev.c:6063
> > #10 0x56163be71c88 in ovsthread_wrapper (aux_=0x56163fb3fe70) at
> > lib/ovs-thread.c:383
> > #11 0x7f9f884cf6db in start_thread (arg=0x7f9f72734700) at
> > pthread_create.c:463
> > #12 0x7f9f862bb71f in clone () at
&
Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
> -Original Message-
> From: dev On Behalf Of Ferriter, Cian
> Sent: Wednesday 23 June 2021 13:38
> To: Ilya Maximets ; Eli Britstein ;
> d...@openvswitch.org
> Cc: ivan.ma...@oktetlabs.ru; Ameer Mahagneh ; Majd Dibbiny
>
> Subject: Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
>
> Hi all,
>
> As part of rebasing our AVX512 DPIF on this patchset, I tested this patchset
> with partial HWOL and I'm
> seeing strange behaviour.
>
> I'll report back more detailed findings soon, just wanted to mention this
> here as soon as I found the
> issue.
>
> Thanks,
> Cian
>
More details on the issue I'm seeing:
I'm using Ilya's branch from Github:
https://github.com/igsilya/ovs/tree/tmp-vxlan-decap
~/ovs_scripts# $OVS_DIR/utilities/ovs-vsctl list Open_vSwitch
dpdk_version: "DPDK 20.11.1"
other_config: {dpdk-hugepage-dir="/mnt/huge", dpdk-init="true",
dpdk-lcore-mask="0x1", dpdk-socket-mem="2048,0", emc-insert-inv-prob="0",
hw-offload="true", pmd-cpu-mask="0x2"}
~/ovs_scripts# $OVS_DIR/utilities/ovs-vsctl show
31584ce5-09c1-44b3-ab27-1a0308d63fff
Bridge br0
datapath_type: netdev
Port br0
Interface br0
type: internal
Port phy0
Interface phy0
type: dpdk
options: {dpdk-devargs="5e:00.0"}
~/ovs_scripts# $OVS_DIR/utilities/ovs-ofctl dump-flows br0
cookie=0x0, duration=29.466s, table=0, n_packets=0, n_bytes=0, in_port=phy0
actions=IN_PORT
I'm expecting the flow to be partially offloaded, but I get a segfault when
using the above branch. More info on the segfault below:
Thread 13 "pmd-c01/id:8" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f9f72734700 (LWP 19327)]
0x56163bf0d825 in set_error (error=0x0, type=RTE_FLOW_ERROR_TYPE_ATTR) at
lib/netdev-dpdk.h:84
(gdb) bt
#0 0x56163bf0d825 in set_error (error=0x0, type=RTE_FLOW_ERROR_TYPE_ATTR)
at lib/netdev-dpdk.h:84
#1 0x56163bf0d8d3 in netdev_dpdk_rte_flow_get_restore_info
(netdev=0x1bfc65c80, p=0x19033af00, info=0x7f9f72729a20, error=0x0) at
lib/netdev-dpdk.h:119
#2 0x56163bf14da3 in netdev_offload_dpdk_hw_miss_packet_recover
(netdev=0x1bfc65c80, packet=0x19033af00) at lib/netdev-offload-dpdk.c:2133
#3 0x56163bde3662 in netdev_hw_miss_packet_recover (netdev=0x1bfc65c80,
packet=0x19033af00) at lib/netdev-offload.c:265
#4 0x56163bda19a9 in dp_netdev_hw_flow (pmd=0x7f9f72735010, port_no=2,
packet=0x19033af00, flow=0x7f9f72729b98) at lib/dpif-netdev.c:7087
#5 0x56163bda1c5c in dfc_processing (pmd=0x7f9f72735010,
packets_=0x7f9f727310d0, keys=0x7f9f7272c480, missed_keys=0x7f9f7272c370,
batches=0x7f9f72729f60, n_batches=0x7f9f72730f70, flow_map=0x7f9f72729c50,
n_flows=0x7f9f72730f78, index_map=0x7f9f72729c30 "", md_is_valid=false,
port_no=2) at lib/dpif-netdev.c:7168
#6 0x56163bda2f3e in dp_netdev_input__ (pmd=0x7f9f72735010,
packets=0x7f9f727310d0, md_is_valid=false, port_no=2) at lib/dpif-netdev.c:7475
#7 0x56163bda3105 in dp_netdev_input (pmd=0x7f9f72735010,
packets=0x7f9f727310d0, port_no=2) at lib/dpif-netdev.c:7519
#8 0x56163bd9ab04 in dp_netdev_process_rxq_port (pmd=0x7f9f72735010,
rxq=0x56163fb3f610, port_no=2) at lib/dpif-netdev.c:4774
#9 0x56163bd9ee17 in pmd_thread_main (f_=0x7f9f72735010) at
lib/dpif-netdev.c:6063
#10 0x56163be71c88 in ovsthread_wrapper (aux_=0x56163fb3fe70) at
lib/ovs-thread.c:383
#11 0x7f9f884cf6db in start_thread (arg=0x7f9f72734700) at
pthread_create.c:463
#12 0x7f9f862bb71f in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
In netdev_offload_dpdk_hw_miss_packet_recover() calls
netdev_dpdk_rte_flow_get_restore_info() with a NULL for the struct
rte_flow_error *error argument:
if (netdev_dpdk_rte_flow_get_restore_info(netdev, packet,
_restore_info, NULL)) {
/* This function is called for every packet, and in most cases there
* will be no restore info from the HW, thus error is expected.
*/
return 0;
}
There are 2 "netdev_dpdk_rte_flow_get_restore_info()" functions. One in
lib/netdev-dpdk.h and one in lib/netdev-dpdk.c.
I don't have the experimental API enabled, so I'm using the function rom
lib/netdev-dpdk.h.
> > -Original Message-
> > From: dev On Behalf Of Ilya Maximets
> > Sent: Tuesday 22 June 2021 16:55
> > To: Eli Britstein ; d...@openvswitch.org; Ilya Maximets
> >
> > Cc: ivan.ma...@oktetlabs.ru; Ameer Mahagneh ; Majd
> > Dibbiny
> > Subject: Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
> >
> > On 4/4/21 11:54 AM, Eli Britstein wrote:
> > > VXLAN decap in OVS-DPDK configuration consists of two flows:
&
Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload
Hi all, As part of rebasing our AVX512 DPIF on this patchset, I tested this patchset with partial HWOL and I'm seeing strange behaviour. I'll report back more detailed findings soon, just wanted to mention this here as soon as I found the issue. Thanks, Cian > -Original Message- > From: dev On Behalf Of Ilya Maximets > Sent: Tuesday 22 June 2021 16:55 > To: Eli Britstein ; d...@openvswitch.org; Ilya Maximets > > Cc: ivan.ma...@oktetlabs.ru; Ameer Mahagneh ; Majd Dibbiny > > Subject: Re: [ovs-dev] [PATCH V6 00/13] Netdev vxlan-decap offload > > On 4/4/21 11:54 AM, Eli Britstein wrote: > > VXLAN decap in OVS-DPDK configuration consists of two flows: > > F1: in_port(ens1f0),eth(),ipv4(),udp(), actions:tnl_pop(vxlan_sys_4789) > > F2: tunnel(),in_port(vxlan_sys_4789),eth(),ipv4(), actions:ens1f0_0 > > > > F1 is a classification flow. It has outer headers matches and it > > classifies the packet as a VXLAN packet, and using tnl_pop action the > > packet continues processing in F2. > > F2 is a flow that has matches on tunnel metadata as well as on the inner > > packet headers (as any other flow). > > > > In order to fully offload VXLAN decap path, both F1 and F2 should be > > offloaded. As there are more than one flow in HW, it is possible that > > F1 is done by HW but F2 is not. Packet is received by SW, and should be > > processed starting from F2 as F1 was already done by HW. > > Rte_flows are applicable only on physical port IDs. Keeping the original > > physical in port on which the packet is received on enables applying > > vport flows (e.g. F2) on that physical port. > > > > This patch-set makes use of [1] introduced in DPDK 20.11, that adds API > > for tunnel offloads. > > > > Note that MLX5 PMD has a bug that the tnl_pop private actions must be > > first. In OVS it is not. > > Fixing this issue is scheduled for 21.05 (and stable 20.11.2). > > Meanwhile, tests were done with a workaround for it [2]. > > > > v2-v1: > > - Tracking original in_port, and applying vport on that physical port > > instead of all PFs. > > v3-v2: > > - Traversing ports using a new API instead of flow_dump. > > - Refactor packet state recover logic, with bug fix for error pop_header. > > - One ref count for netdev in non-tunnel case. > > - Rename variables, comments, rebase. > > v4-v3: > > - Extract orig_in_port from physdev for flow modify. > > - Miss handling fixes. > > v5-v4: > > - Drop refactor offload rule creation commit. > > - Comment about setting in_port in restore. > > - Refactor vports flow offload commit. > > v6-v5: > > - Fixed duplicate netdev ref bug. > > > > Travis: > > v1: https://travis-ci.org/github/elibritstein/OVS/builds/756418552 > > v2: https://travis-ci.org/github/elibritstein/OVS/builds/758382963 > > v3: https://travis-ci.org/github/elibritstein/OVS/builds/761089087 > > v4: https://travis-ci.org/github/elibritstein/OVS/builds/763146966 > > v5: https://travis-ci.org/github/elibritstein/OVS/builds/765271879 > > v6: https://travis-ci.org/github/elibritstein/OVS/builds/765816800 > > > > GitHub Actions: > > v1: https://github.com/elibritstein/OVS/actions/runs/515334647 > > v2: https://github.com/elibritstein/OVS/actions/runs/554986007 > > v3: https://github.com/elibritstein/OVS/actions/runs/613226225 > > v4: https://github.com/elibritstein/OVS/actions/runs/658415274 > > v5: https://github.com/elibritstein/OVS/actions/runs/704357369 > > v6: https://github.com/elibritstein/OVS/actions/runs/716304028 > > > > [1] https://mails.dpdk.org/archives/dev/2020-October/187314.html > > [2] https://github.com/elibritstein/dpdk-stable/pull/1 > > > > > > Eli Britstein (10): > > netdev-offload: Add HW miss packet state recover API > > netdev-dpdk: Introduce DPDK tunnel APIs > > netdev-offload: Introduce an API to traverse ports > > netdev-dpdk: Add flow_api support for netdev vxlan vports > > netdev-offload-dpdk: Implement HW miss packet recover for vport > > dpif-netdev: Add HW miss packet state recover logic > > netdev-offload-dpdk: Change log rate limits > > netdev-offload-dpdk: Support tunnel pop action > > netdev-offload-dpdk: Support vports flows offload > > netdev-dpdk-offload: Add vxlan pattern matching function > > > > Ilya Maximets (2): > > netdev-offload: Allow offloading to netdev without ifindex. > > netdev-offload: Disallow offloading to unrelated tunneling vports. > > > > Sriharsha Basavapatna (1): > > dpif-netdev: Provide orig_in_port in metadata for tunneled packets > > > > Documentation/howto/dpdk.rst | 1 + > > NEWS | 2 + > > lib/dpif-netdev.c | 97 +++-- > > lib/netdev-dpdk.c | 118 ++ > > lib/netdev-dpdk.h | 106 - > > lib/netdev-offload-dpdk.c | 704 +++--- > > lib/netdev-offload-provider.h | 5 + > > lib/netdev-offload-tc.c | 8 + > > lib/netdev-offload.c | 47 ++- > > lib/netdev-offload.h | 10 + > > lib/packets.h
Re: [ovs-dev] [v13 08/12] dpif-netdev-unixctl.man: Document subtable-lookup-* CMDs
Hi all,
Thanks for the feedback. My responses are inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Tuesday 22 June 2021 17:39
> To: Stokes, Ian
> Cc: Ferriter, Cian ; ovs-dev@openvswitch.org;
> i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 08/12] dpif-netdev-unixctl.man: Document
> subtable-lookup-* CMDs
>
> On Tue, Jun 22, 2021 at 03:42:46PM +, Stokes, Ian wrote:
> > > Hi Flavio,
> > >
> > > Thanks for the review. My responses are inline.
> > >
> > > Cian
> > >
> > > > -Original Message-----
> > > > From: Flavio Leitner
> > > > Sent: Monday 21 June 2021 19:22
> > > > To: Ferriter, Cian
> > > > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> > > > Subject: Re: [ovs-dev] [v13 08/12] dpif-netdev-unixctl.man: Document
> > > subtable-lookup-* CMDs
> > > >
> > > >
> > > > Hi,
> > > >
> > > > This commit could be submitted outside of this patch-set as fix
> > > > for commit 9ff7cabfd7 ("dpif-netdev: add subtable-lookup-prio-get
> > > > command") and commit 3d018c3ea79d ("dpif-netdev: add subtable lookup
> > > > prio set command.").
> > > >
> > > > This helps to get it merged sooner and reduce this patch-set size.
> > > >
> > >
> > > I'll remove this patch from the patchset and send to the mailing list
> > > separately.
> > > I'll wait till the DPIF patchset has been merged to send this, since I
> > > don't want
> > > there to be rebase conflicts (the DPIF patchset also modifies this part
> > > of lib/dpif-
> > > netdev-unixctl.man).
> > >
> > > I'll add the appropriate Fixes tags.
> >
> > @Flavio, If there is an aim to reduce the overall patch number of
> > the series then I would recommend the following patches be
> > submitted separately also from this series as there is no
> > dependency on them to enable DPIF with AVX512.
> >
> > [v13 10/12] dpif-netdev/dpcls: Specialize more subtable signatures.
> > [v13 11/12] dpdk: Cache result of CPU ISA checks.
> >
> >
> > These two are quite small and I think could almost be applied now
> > rather than as part of the series as they are modifying existing
> > functionality (DPCLS AVX512 supported traffic types and DPDK flag
> > caching).
> >
> > Thoughts?
>
> The idea is to get unrelated chunks merged sooner, if they make
> sense of course, and then we have less patches to carry on.
>
> If the patches are going to arrive later or it causes more work
> on following up patches, then I see no benefit.
>
> It's a suggestion. I am happy to review either way.
>
> Thanks,
> fbl
I think splitting out the 2 above patches might cause a bit more work on our
side. I'll leave them in the v14 patchset based on no strong preferences above.
Since I've already taken out the "[v13 08/12] dpif-netdev-unixctl.man: Document
subtable-lookup-* CMDs" patch, I'll leave that out and submit separately.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 08/12] dpif-netdev-unixctl.man: Document subtable-lookup-* CMDs
Hi Flavio,
Thanks for the review. My responses are inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Monday 21 June 2021 19:22
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 08/12] dpif-netdev-unixctl.man: Document
> subtable-lookup-* CMDs
>
>
> Hi,
>
> This commit could be submitted outside of this patch-set as fix
> for commit 9ff7cabfd7 ("dpif-netdev: add subtable-lookup-prio-get
> command") and commit 3d018c3ea79d ("dpif-netdev: add subtable lookup
> prio set command.").
>
> This helps to get it merged sooner and reduce this patch-set size.
>
I'll remove this patch from the patchset and send to the mailing list
separately.
I'll wait till the DPIF patchset has been merged to send this, since I don't
want there to be rebase conflicts (the DPIF patchset also modifies this part of
lib/dpif-netdev-unixctl.man).
I'll add the appropriate Fixes tags.
> Thanks for documenting it.
> fbl
>
> On Thu, Jun 17, 2021 at 05:18:21PM +0100, Cian Ferriter wrote:
> > Signed-off-by: Cian Ferriter
> >
> > ---
> >
> > v13:
> > - New commit to update manpages with more commands that are missing.
> > ---
> > lib/dpif-netdev-unixctl.man | 10 ++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man
> > index 45a1bd669..d77f5d9a4 100644
> > --- a/lib/dpif-netdev-unixctl.man
> > +++ b/lib/dpif-netdev-unixctl.man
> > @@ -228,6 +228,16 @@ When this is the case, the above command prints the
> > load-balancing
> information
> > of the bonds configured in datapath \fIdp\fR showing the interface
> > associated
> > with each bucket (hash).
> > .
> > +.IP "\fBdpif-netdev/subtable-lookup-prio-get\fR"
> > +Lists the DPCLS implementations or lookup functions that are available as
> > well
> > +as their priorities.
> > +.
> > +.IP "\fBdpif-netdev/subtable-lookup-prio-set\fR \fIlookup_function\fR \
> > +\fIprio\fR"
> > +Sets the priority of a lookup function by the name, \fIlookup_function\fR,
> > and
> > +the priority, \fIprio\fR, which should be a positive integer value. The
> > highest
> > +priority lookup function is used for classification.
> > +.
> > .IP "\fBdpif-netdev/dpif-get\fR
> > Lists the DPIF implementations that are available.
> > .
> > --
> > 2.32.0
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
> --
> fbl
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 04/12] dpif-avx512: Add ISA implementation of dpif.
Hi Flavio, Thanks for the review. My responses are inline. Cian > -Original Message- > From: Flavio Leitner > Sent: Sunday 20 June 2021 21:09 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; Amber, Kumar ; > i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v13 04/12] dpif-avx512: Add ISA implementation of > dpif. > > > Hi, > > I am still reviewing the patch, but I thought worth to discuss > few items below. > > On Thu, Jun 17, 2021 at 05:18:17PM +0100, Cian Ferriter wrote: > > From: Harry van Haaren > > > > This commit adds the AVX512 implementation of DPIF functionality, > > specifically the dp_netdev_input_outer_avx512 function. This function > > only handles outer (no re-circulations), and is optimized to use the > > AVX512 ISA for packet batching and other DPIF work. > > > > Sparse is not able to handle the AVX512 intrinsics, causing compile > > time failures, so it is disabled for this file. > > > > Signed-off-by: Harry van Haaren > > Co-authored-by: Cian Ferriter > > Signed-off-by: Cian Ferriter > > Co-authored-by: Kumar Amber > > Signed-off-by: Kumar Amber > > > > --- > > > > v13: > > - Squash "Add HWOL support" commit into this commit. > > - Add NEWS item about this feature here rather than in a later commit. > > - Add #define NUM_U64_IN_ZMM_REG 8. > > - Add comment describing operation of while loop handling HWOL->EMC->SMC > > lookups in dp_netdev_input_outer_avx512(). > > - Add EMC and SMC batch insert functions for better handling of EMC and > > SMC in AVX512 DPIF. > > - Minor code refactor to address review comments. > > --- > > NEWS | 2 + > > lib/automake.mk | 5 +- > > lib/dpif-netdev-avx512.c | 327 +++ > > lib/dpif-netdev-private-dfc.h| 25 +++ > > lib/dpif-netdev-private-dpif.h | 32 +++ > > lib/dpif-netdev-private-thread.h | 11 +- > > lib/dpif-netdev-private.h| 25 +++ > > lib/dpif-netdev.c| 103 -- > > 8 files changed, 514 insertions(+), 16 deletions(-) > > create mode 100644 lib/dpif-netdev-avx512.c > > create mode 100644 lib/dpif-netdev-private-dpif.h > > > > diff --git a/NEWS b/NEWS > > index 96b3a61c8..6a4a7b76d 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -10,6 +10,8 @@ Post-v2.15.0 > > * Auto load balancing of PMDs now partially supports cross-NUMA > > polling > > cases, e.g if all PMD threads are running on the same NUMA node. > > * Refactor lib/dpif-netdev.c to multiple header files. > > + * Add avx512 implementation of dpif which can process non recirculated > > + packets. It supports partial HWOL, EMC, SMC and DPCLS lookups. > > - ovs-ctl: > > * New option '--no-record-hostname' to disable hostname configuration > > in ovsdb on startup. > > diff --git a/lib/automake.mk b/lib/automake.mk > > index 3a33cdd5c..660cd07f0 100644 > > --- a/lib/automake.mk > > +++ b/lib/automake.mk > > @@ -33,11 +33,13 @@ lib_libopenvswitchavx512_la_CFLAGS = \ > > -mavx512f \ > > -mavx512bw \ > > -mavx512dq \ > > + -mbmi \ > > -mbmi2 \ > > -fPIC \ > > $(AM_CFLAGS) > > lib_libopenvswitchavx512_la_SOURCES = \ > > - lib/dpif-netdev-lookup-avx512-gather.c > > + lib/dpif-netdev-lookup-avx512-gather.c \ > > + lib/dpif-netdev-avx512.c > > lib_libopenvswitchavx512_la_LDFLAGS = \ > > -static > > endif > > @@ -114,6 +116,7 @@ lib_libopenvswitch_la_SOURCES = \ > > lib/dpif-netdev-private-dfc.c \ > > lib/dpif-netdev-private-dfc.h \ > > lib/dpif-netdev-private-dpcls.h \ > > + lib/dpif-netdev-private-dpif.h \ > > lib/dpif-netdev-private-flow.h \ > > lib/dpif-netdev-private-hwol.h \ > > lib/dpif-netdev-private-thread.h \ > > diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c > > new file mode 100644 > > index 0..0e55b0be2 > > --- /dev/null > > +++ b/lib/dpif-netdev-avx512.c > > @@ -0,0 +1,327 @@ > > +/* > > + * Copyright (c) 2021 Intel Corporation. > > + * > > + * Licensed under the Apache License, Version 2.0 (the "License"); > > + * you may not use this file except in compliance with the License. > > + * You may obtain a copy of the License at: > > + * > > + * http://www.apache.org/licenses/LICENSE-2.0 > > + * > > + * Unless required by applicable law or agreed to in writing
Re: [ovs-dev] [v13 03/12] dpif-netdev: Add function pointer for netdev input.
Hi Flavio,
Thanks for the review. My responses are inline.
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Friday 18 June 2021 13:53
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 03/12] dpif-netdev: Add function pointer for
> netdev input.
>
>
> Hello,
>
> On Thu, Jun 17, 2021 at 05:18:16PM +0100, Cian Ferriter wrote:
> > From: Harry van Haaren
> >
> > This commit adds a function pointer to the pmd thread data structure,
> > giving the pmd thread flexibility in its dpif-input function choice.
> > This allows choosing of the implementation based on ISA capabilities
> > of the runtime CPU, leading to optimizations and higher performance.
> >
> > Signed-off-by: Harry van Haaren
> >
> > ---
> >
> > v13:
> > - Minor code refactor to address review comments.
> > ---
> > lib/dpif-netdev-private-thread.h | 13 +
> > lib/dpif-netdev.c| 7 ++-
> > 2 files changed, 19 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/dpif-netdev-private-thread.h
> > b/lib/dpif-netdev-private-thread.h
> > index 5e5308b96..0d674ab83 100644
> > --- a/lib/dpif-netdev-private-thread.h
> > +++ b/lib/dpif-netdev-private-thread.h
> > @@ -47,6 +47,13 @@ struct dp_netdev_pmd_thread_ctx {
> > uint32_t emc_insert_min;
> > };
> >
> > +/* Forward declaration for typedef. */
> > +struct dp_netdev_pmd_thread;
> > +
> > +typedef void (*dp_netdev_input_func)(struct dp_netdev_pmd_thread *pmd,
> > + struct dp_packet_batch *packets,
> > + odp_port_t port_no);
> > +
> > /* PMD: Poll modes drivers. PMD accesses devices via polling to eliminate
> > * the performance overhead of interrupt processing. Therefore netdev can
> > * not implement rx-wait for these devices. dpif-netdev needs to poll
> > @@ -101,6 +108,12 @@ struct dp_netdev_pmd_thread {
> > /* Current context of the PMD thread. */
> > struct dp_netdev_pmd_thread_ctx ctx;
> >
> > +/* Function pointer to call for dp_netdev_input() functionality. */
> > +dp_netdev_input_func netdev_input_func;
> > +
> > +/* Pointer for per-DPIF implementation scratch space. */
> > +void *netdev_input_func_userdata;
> > +
>
> I see you need to switch the input function and the patch looks fine, but
> the default function doesn't require netdev_input_func_userdata. I think
> it would be better to add that in the next patch where it is actually
> required.
>
Good point, I'll move netdev_input_func_userdata to the next patch.
> fbl
>
> > struct seq *reload_seq;
> > uint64_t last_reload_seq;
> >
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index e913f4efc..e6486417e 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -4231,8 +4231,9 @@ dp_netdev_process_rxq_port(struct
> > dp_netdev_pmd_thread *pmd,
> > }
> > }
> > }
> > +
> > /* Process packet batch. */
> > -dp_netdev_input(pmd, , port_no);
> > +pmd->netdev_input_func(pmd, , port_no);
> >
> > /* Assign processing cycles to rx queue. */
> > cycles = cycle_timer_stop(>perf_stats, );
> > @@ -6029,6 +6030,10 @@ dp_netdev_configure_pmd(struct dp_netdev_pmd_thread
> > *pmd, struct dp_netdev
> *dp,
> > hmap_init(>tnl_port_cache);
> > hmap_init(>send_port_cache);
> > cmap_init(>tx_bonds);
> > +
> > +/* Initialize the DPIF function pointer to the default scalar version.
> > */
> > +pmd->netdev_input_func = dp_netdev_input;
> > +
> > /* init the 'flow_cache' since there is no
> > * actual thread created for NON_PMD_CORE_ID. */
> > if (core_id == NON_PMD_CORE_ID) {
> > --
> > 2.32.0
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
> --
> fbl
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [v13 02/12] dpif-netdev: Split HWOL out to own header file.
Hi Flavio,
Thanks for the review. My comments are inline,
Cian
> -Original Message-
> From: Flavio Leitner
> Sent: Friday 18 June 2021 13:12
> To: Ferriter, Cian
> Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org
> Subject: Re: [ovs-dev] [v13 02/12] dpif-netdev: Split HWOL out to own header
> file.
>
>
> Hello,
>
> I have some comments inline.
>
> On Thu, Jun 17, 2021 at 05:18:15PM +0100, Cian Ferriter wrote:
> > From: Harry van Haaren
> >
> > This commit moves the datapath lookup functions required for
> > hardware offload to a seperate file. This allows other DPIF
>
>
> Spelling error.
>
Good spot, I'll fix this in the next version.
> > implementations to access the lookup functions, encouraging
> > code reuse.
> >
> > Signed-off-by: Harry van Haaren
> >
> > ---
> >
> > Cc: Gaetan Rivet
> > Cc: Sriharsha Basavapatna
> >
> > v13:
> > - Minor code refactor to address review comments.
> > ---
> > lib/automake.mk| 1 +
> > lib/dpif-netdev-private-hwol.h | 63 ++
> > lib/dpif-netdev.c | 38 ++--
> > 3 files changed, 66 insertions(+), 36 deletions(-)
> > create mode 100644 lib/dpif-netdev-private-hwol.h
> >
> > diff --git a/lib/automake.mk b/lib/automake.mk
> > index fdba3c6c0..3a33cdd5c 100644
> > --- a/lib/automake.mk
> > +++ b/lib/automake.mk
> > @@ -115,6 +115,7 @@ lib_libopenvswitch_la_SOURCES = \
> > lib/dpif-netdev-private-dfc.h \
> > lib/dpif-netdev-private-dpcls.h \
> > lib/dpif-netdev-private-flow.h \
> > + lib/dpif-netdev-private-hwol.h \
> > lib/dpif-netdev-private-thread.h \
> > lib/dpif-netdev-private.h \
> > lib/dpif-netdev-perf.c \
> > diff --git a/lib/dpif-netdev-private-hwol.h b/lib/dpif-netdev-private-hwol.h
> > new file mode 100644
> > index 0..b93297a74
> > --- /dev/null
> > +++ b/lib/dpif-netdev-private-hwol.h
> > @@ -0,0 +1,63 @@
> > +/*
> > + * Copyright (c) 2008, 2009, 2010, 2011, 2012, 2013, 2015 Nicira, Inc.
> > + * Copyright (c) 2021 Intel Corporation.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + * http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#ifndef DPIF_NETDEV_PRIVATE_HWOL_H
> > +#define DPIF_NETDEV_PRIVATE_HWOL_H 1
> > +
> > +#include "dpif-netdev-private-flow.h"
> > +
> > +#define MAX_FLOW_MARK (UINT32_MAX - 1)
> > +#define INVALID_FLOW_MARK 0
> > +/* Zero flow mark is used to indicate the HW to remove the mark. A packet
> > + * marked with zero mark is received in SW without a mark at all, so it
> > + * cannot be used as a valid mark.
> > + */
> > +
> > +struct megaflow_to_mark_data {
> > +const struct cmap_node node;
> > +ovs_u128 mega_ufid;
> > +uint32_t mark;
> > +};
> > +
> > +struct flow_mark {
> > +struct cmap megaflow_to_mark;
> > +struct cmap mark_to_flow;
> > +struct id_pool *pool;
> > +};
> > +
> > +/* allocated in dpif-netdev.c */
> > +extern struct flow_mark flow_mark;
> > +
> > +static inline struct dp_netdev_flow *
> > +mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd,
> > + const uint32_t mark)
> > +{
> > +struct dp_netdev_flow *flow;
> > +
> > +CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0),
> > + _mark.mark_to_flow) {
> > +if (flow->mark == mark && flow->pmd_id == pmd->core_id &&
> > +flow->dead == false) {
> > +return flow;
> > +}
> > +}
> > +
> > +return NULL;
> > +}
>
> Wouldn't this be better in a separate .c file? Because although the
> structure flow_mark is here, it is allocated in dpif-netdev.c and
> we have a fairly large function inline. This seems enough to start
Re: [ovs-dev] [v13 01/12] dpif-netdev: Refactor to multiple header files.
Hi Flavio, Thanks for the review. My responses are inline. Thanks, Cian > -Original Message- > From: Flavio Leitner > Sent: Friday 18 June 2021 13:12 > To: Ferriter, Cian > Cc: ovs-dev@openvswitch.org; i.maxim...@ovn.org > Subject: Re: [ovs-dev] [v13 01/12] dpif-netdev: Refactor to multiple header > files. > > > Hello, > > Some comments below. > > On Thu, Jun 17, 2021 at 05:18:14PM +0100, Cian Ferriter wrote: > > From: Harry van Haaren > > > > Split the very large file dpif-netdev.c and the datastructures > > it contains into multiple header files. Each header file is > > responsible for the datastructures of that component. > > > > This logical split allows better reuse and modularity of the code, > > and reduces the very large file dpif-netdev.c to be more managable. > > > > Due to dependencies between components, it is not possible to > > move component in smaller granularities than this patch. > > > > To explain the dependencies better, eg: > > > > DPCLS has no deps (from dpif-netdev.c file) > > FLOW depends on DPCLS (struct dpcls_rule) > > DFC depends on DPCLS (netdev_flow_key) and FLOW (netdev_flow_key) > > THREAD depends on DFC (struct dfc_cache) > > > > DFC_PROC depends on THREAD (struct pmd_thread) > > > > DPCLS lookup.h/c require only DPCLS > > DPCLS implementations require only dpif-netdev-lookup.h. > > - This change was made in 2.12 release with function pointers > > - This commit only refactors the name to "private-dpcls.h" > > > > netdev_flow_key_equal_mf() is renamed to emc_flow_key_equal_mf(). > > > > Rename functions specific to dpcls from netdev_* namespace to the > > dpcls_* namespace, as they are only used by dpcls code. > > > > 'inline' is added to the dp_netdev_flow_hash() when it is moved > > definition to fix a compiler error. > > > > One valid checkpatch issue with the use of the > > EMC_FOR_EACH_POS_WITH_HASH() macro was fixed. > > > > Signed-off-by: Harry van Haaren > > Co-authored-by: Cian Ferriter > > Signed-off-by: Cian Ferriter > > > > --- > > > > Cc: Gaetan Rivet > > Cc: Sriharsha Basavapatna > > > > v13: > > - Add NEWS item in this commit rather than later. > > - Add lib/dpif-netdev-private-dfc.c file and move non fast path dfc > > related functions there. > > - Squash commit which renames functions specific to dpcls from netdev_* > > namespace to the dpcls_* namespace, as they are only used by dpcls > > code into this commit. > > - Minor fixes from review comments. > > --- > > NEWS | 1 + > > lib/automake.mk| 5 + > > lib/dpif-netdev-lookup-autovalidator.c | 1 - > > lib/dpif-netdev-lookup-avx512-gather.c | 1 - > > lib/dpif-netdev-lookup-generic.c | 1 - > > lib/dpif-netdev-lookup.h | 2 +- > > lib/dpif-netdev-private-dfc.c | 110 + > > lib/dpif-netdev-private-dfc.h | 176 > > lib/dpif-netdev-private-dpcls.h| 127 ++ > > lib/dpif-netdev-private-flow.h | 162 > > lib/dpif-netdev-private-thread.h | 206 ++ > > lib/dpif-netdev-private.h | 100 + > > lib/dpif-netdev.c | 539 + > > 13 files changed, 811 insertions(+), 620 deletions(-) > > create mode 100644 lib/dpif-netdev-private-dfc.c > > create mode 100644 lib/dpif-netdev-private-dfc.h > > create mode 100644 lib/dpif-netdev-private-dpcls.h > > create mode 100644 lib/dpif-netdev-private-flow.h > > create mode 100644 lib/dpif-netdev-private-thread.h > > > > diff --git a/NEWS b/NEWS > > index ebba17b22..96b3a61c8 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -9,6 +9,7 @@ Post-v2.15.0 > > - Userspace datapath: > > * Auto load balancing of PMDs now partially supports cross-NUMA > > polling > > cases, e.g if all PMD threads are running on the same NUMA node. > > + * Refactor lib/dpif-netdev.c to multiple header files. > > - ovs-ctl: > > * New option '--no-record-hostname' to disable hostname configuration > > in ovsdb on startup. > > diff --git a/lib/automake.mk b/lib/automake.mk > > index db9017591..fdba3c6c0 100644 > > --- a/lib/automake.mk > > +++ b/lib/automake.mk > > @@ -111,6 +111,11 @@ lib_libopenvswitch_la_SOURCES = \ > > lib/dpif-netdev-lookup-generic.c \ > > lib/dpif-netdev.c \ > > lib
Re: [ovs-dev] [v12 16/16] netdev: Optimize netdev_send_prepare_batch.
> -Original Message-
> From: Stokes, Ian
> Sent: Wednesday 16 June 2021 13:48
> To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van
> Haaren, Harry
> Cc: i.maxim...@ovn.org; Gaetan Rivet ; Sriharsha
> Basavapatna
> Subject: RE: [ovs-dev] [v12 16/16] netdev: Optimize netdev_send_prepare_batch.
>
> > Hi Ian,
> >
> > Thanks for the review. My responses are inline.
> >
> > > -Original Message-
> > > From: Stokes, Ian
> > > Sent: Tuesday 15 June 2021 18:04
> > > To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van
> > Haaren, Harry
> > > Cc: i.maxim...@ovn.org; Gaetan Rivet ; Sriharsha
> > Basavapatna
> > > Subject: RE: [ovs-dev] [v12 16/16] netdev: Optimize
> > netdev_send_prepare_batch.
> > >
> > > > From: Harry van Haaren
> > > >
> > > > Optimize for the best case here where all packets will be compatible
> > > > with 'netdev_flags'.
> > > >
> > > > Signed-off-by: Harry van Haaren
> > > > Co-authored-by: Cian Ferriter
> > > > Signed-off-by: Cian Ferriter
> > >
> > > Thanks for the patch Cian/Harry, comments inline.
> > >
> > > In general I have a concern of the impact on HWOL cases here and how much
> > it would affect that. Have you data on this? Any
> > > thoughts from other HWOL folks here?
> >
> > From our POV, we are optimizing the normal and error free case. We are
> > impacting the error case. When a packet is not compatible with netdev_flags
> > that packet will be deleted from the batch and a " VLOG_WARN_RL()" will be
> > called. This is quite costly as it is.
> >
> > I'm curious to see what other HWOL folks think.
>
> +1, again probably not a blocker for the DPIF framework, could be submitted
> separately from the series if required.
>
I'll submit this patch separately from the DPIF series.
> Regards
> Ian
> >
> > > > ---
> > > > NEWS | 2 ++
> > > > lib/netdev.c | 31 ++-
> > > > 2 files changed, 24 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/NEWS b/NEWS
> > > > index a7ec34af1..8ad3c98db 100644
> > > > --- a/NEWS
> > > > +++ b/NEWS
> > > > @@ -18,6 +18,8 @@ Post-v2.15.0
> > > > CPU supports it. This enhances performance by using the native
> > vpopcount
> > > > instructions, instead of the emulated version of vpopcount.
> > > > * Optimize dp_netdev_output by enhancing compiler optimization
> > potential.
> > > > + * Optimize netdev sending by assuming the happy case, and using
> > fallback
> > > > + for if the netdev doesnt meet the required HWOL needs of a
> > > > packet.
> > > This sounds a bit too low level for a NEWS item. Would suggest changing to
> > single line Optimize netdev sending for best case scenario.
> > >
> >
> > Agreed, your single line sounds better. I've fixed this in the next version.
> >
> > > BR
> > > Ian
> > >
> > > > - ovs-ctl:
> > > > * New option '--no-record-hostname' to disable hostname
> > > > configuration
> > > > in ovsdb on startup.
> > > > diff --git a/lib/netdev.c b/lib/netdev.c
> > > > index 91e91955c..29a5f1aa9 100644
> > > > --- a/lib/netdev.c
> > > > +++ b/lib/netdev.c
> > > > @@ -837,20 +837,33 @@ static void
> > > > netdev_send_prepare_batch(const struct netdev *netdev,
> > > >struct dp_packet_batch *batch)
> > > > {
> > > > -struct dp_packet *packet;
> > > > -size_t i, size = dp_packet_batch_size(batch);
> > > > +struct dp_packet *p;
> > > > +uint32_t i, size = dp_packet_batch_size(batch);
> > > > +char *err_msg = NULL;
> > > >
> > > > -DP_PACKET_BATCH_REFILL_FOR_EACH (i, size, packet, batch) {
> > > > -char *errormsg = NULL;
> > > > +for (i = 0; i < size; i++) {
> > > > +p = batch->packets[i];
> > > > +int pkt_ok = netdev_send_prepare_packet(netdev->ol_flags, p,
> > _msg);
> > > >
> > > > -if (netdev_send_prepare_packet(netdev->ol_flags, packet,
> > > > ))
> > {
> > > > -dp_packet_batch_refill(batch, packet, i);
> > > >
Re: [ovs-dev] [v12 15/16] dpif-netdev: Optimize dp output action.
Hi Ian,
Further responses are inline.
> -Original Message-
> From: Stokes, Ian
> Sent: Wednesday 16 June 2021 13:47
> To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van
> Haaren, Harry ;
> i.maxim...@ovn.org
> Subject: RE: [ovs-dev] [v12 15/16] dpif-netdev: Optimize dp output action.
>
> > Hi Ian,
> >
> > Thanks for the review. My responses are inline.
> >
> > > -Original Message-
> > > From: Stokes, Ian
> > > Sent: Tuesday 15 June 2021 18:04
> > > To: Ferriter, Cian ; ovs-dev@openvswitch.org;
> > Ferriter, Cian ; Van Haaren, Harry
> > >
> > > Cc: i.maxim...@ovn.org
> > > Subject: RE: [ovs-dev] [v12 15/16] dpif-netdev: Optimize dp output action.
> > >
> > > > From: Harry van Haaren
> > > >
> > > > This commit optimizes the output action, by enabling the compiler to
> > > > optimize the code better through reducing code complexity.
> > > >
> > > > The core concept of this optimization is that the array-length checks
> > > > have already been performed above the copying code, so can be removed.
> > > > Removing of the per-packet length checks allows the compiler to auto-
> > vectorize
> > > > the stores using SIMD registers.
> > > >
> > > > Signed-off-by: Harry van Haaren
> > > >
> > >
> > > Thanks for the patch Cian/Harry.
> > >
> > > Comments inline.
> > > > ---
> > > >
> > > > v8: Add NEWS entry.
> > > > ---
> > > > NEWS | 1 +
> > > > lib/dpif-netdev.c | 23 ++-
> > > > 2 files changed, 19 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/NEWS b/NEWS
> > > > index d04dac746..a7ec34af1 100644
> > > > --- a/NEWS
> > > > +++ b/NEWS
> > > > @@ -17,6 +17,7 @@ Post-v2.15.0
> > > > * Enable the AVX512 DPCLS implementation to use VPOPCNT
> > > > instruction
> > if
> > > > the
> > > > CPU supports it. This enhances performance by using the native
> > vpopcount
> > > > instructions, instead of the emulated version of vpopcount.
> > > > + * Optimize dp_netdev_output by enhancing compiler optimization
> > potential.
> > > > - ovs-ctl:
> > > > * New option '--no-record-hostname' to disable hostname
> > > > configuration
> > > > in ovsdb on startup.
> > > > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > > > index 04a4f71cb..f46269ae3 100644
> > > > --- a/lib/dpif-netdev.c
> > > > +++ b/lib/dpif-netdev.c
> > > > @@ -7290,12 +7290,25 @@ dp_execute_output_action(struct
> > > > dp_netdev_pmd_thread *pmd,
> > > > pmd->n_output_batches++;
> > > > }
> > > >
> > > > -struct dp_packet *packet;
> > > > -DP_PACKET_BATCH_FOR_EACH (i, packet, packets_) {
> > > > -p->output_pkts_rxqs[dp_packet_batch_size(>output_pkts)] =
> > > > -pmd->ctx.last_rxq;
> > > > -dp_packet_batch_add(>output_pkts, packet);
> > > > +/* The above checks ensure that there is enough space in the output
> > batch.
> > > > + * Using dp_packet_batch_add() has a branch to check if the batch
> > > > is full.
> > > > + * This branch reduces the compiler's ability to optimize
> > > > efficiently. The
> > > > + * below code implements packet movement between batches without
> > > > checks,
> > > > + * with the required semantics of output batch perhaps containing
> > packets.
> > > > + */
> > >
> > > What is the performance gain recorded with this approach with compiler
> > optimizations?
> > >
> >
> > The test was run using the scalar DPIF, 1 flow and EMC on, where the
> > before/after cycle cost of the patch is 243 cycles before to 230 cycles per
> > packet
> > after. Hence this optimization saves 13 cycles per packet in my
> > measurements.
>
> So I think that info is important and should be added to the commit message
> and will help inform whether to make the change.
>
I added this information to the commit message.
> Ilya has worked in this area extensively so would be interested to hear his
> opinion?
>
> I don't think this is a blocker to the DPIF work so could possibly b
Re: [ovs-dev] [v12 01/16] dpif-netdev: Refactor to multiple header files.
Hi Ian, I've replied on the checkpatch issues inline below. Thanks, Cian > -Original Message- > From: Stokes, Ian > Sent: Tuesday 15 June 2021 18:26 > To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van > Haaren, Harry > Cc: i.maxim...@ovn.org; Gaetan Rivet ; Sriharsha > Basavapatna > Subject: RE: [ovs-dev] [v12 01/16] dpif-netdev: Refactor to multiple header > files. > > > Hi Ian, > > > > Thanks for the review. My responses are inline. > > > > > -Original Message- > > > From: Stokes, Ian > > > Sent: Wednesday 19 May 2021 16:54 > > > To: Ferriter, Cian ; ovs-dev@openvswitch.org > > > Cc: i.maxim...@ovn.org > > > Subject: RE: [ovs-dev] [v12 01/16] dpif-netdev: Refactor to multiple > > > header > > > files. > > > > > > > Split the very large file dpif-netdev.c and the datastructures > > > > it contains into multiple header files. Each header file is > > > > responsible for the datastructures of that component. > > > > > > > > This logical split allows better reuse and modularity of the code, > > > > and reduces the very large file dpif-netdev.c to be more managable. > > > > > > > > Due to dependencies between components, it is not possible to > > > > move component in smaller granularities than this patch. > > > > > > > > To explain the dependencies better, eg: > > > > > > > > DPCLS has no deps (from dpif-netdev.c file) > > > > FLOW depends on DPCLS (struct dpcls_rule) > > > > DFC depends on DPCLS (netdev_flow_key) and FLOW (netdev_flow_key) > > > > THREAD depends on DFC (struct dfc_cache) > > > > > > > > DFC_PROC depends on THREAD (struct pmd_thread) > > > > > > > > DPCLS lookup.h/c require only DPCLS > > > > DPCLS implementations require only dpif-netdev-lookup.h. > > > > - This change was made in 2.12 release with function pointers > > > > - This commit only refactors the name to "private-dpcls.h" > > > > > > > > Signed-off-by: Harry van Haaren > > > > Co-authored-by: Cian Ferriter > > > > Signed-off-by: Cian Ferriter > > > > > > Hi Cian/Harry, > > > > > > Thanks for the patch. > > > > > > One risk I think we need to flag/discuss further with the community is > > > the > > > impact the refactor will have on other patches in progress and that is > > > best > > > dealt with. > > > > > > In an effort to help with this I may Cc folks who have patches that would > > > be > > > affected. > > > > > > Best to come with a plan WRT integration order because although > > > functionally little changes here there would be a wider impact to ongoing > > > patches. > > > > > > I have a few queries below inline below. > > > > > > > If you want me to help here by CC'ing some specific folks in the next > > version of > > the patch set, let me know. > > Sure, I've CCd some folks from Nvidia and Broadcom who may be interested. It > may be worth including them for this aspect of the > patch series going forward. > > > > > > > --- > > > > lib/automake.mk| 4 + > > > > lib/dpif-netdev-lookup-autovalidator.c | 1 - > > > > lib/dpif-netdev-lookup-avx512-gather.c | 1 - > > > > lib/dpif-netdev-lookup-generic.c | 1 - > > > > lib/dpif-netdev-lookup.h | 2 +- > > > > lib/dpif-netdev-private-dfc.h | 244 > > > > lib/dpif-netdev-private-dpcls.h| 129 ++ > > > > lib/dpif-netdev-private-flow.h | 162 > > > > lib/dpif-netdev-private-thread.h | 206 ++ > > > > lib/dpif-netdev-private.h | 100 + > > > > lib/dpif-netdev.c | 519 + > > > > 11 files changed, 760 insertions(+), 609 deletions(-) > > > > create mode 100644 lib/dpif-netdev-private-dfc.h > > > > create mode 100644 lib/dpif-netdev-private-dpcls.h > > > > create mode 100644 lib/dpif-netdev-private-flow.h > > > > create mode 100644 lib/dpif-netdev-private-thread.h > > > > > > > > diff --git a/lib/automake.mk b/lib/automake.mk > > > > index 39901bd6d..9fa8712c3 100644 > > > > --- a/lib/automake.mk > > > > +
Re: [ovs-dev] [v12 04/16] dpif-avx512: Add ISA implementation of dpif.
Hi Ian, Further comments are inline. > -Original Message- > From: Stokes, Ian > Sent: Wednesday 16 June 2021 12:03 > To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van > Haaren, Harry > Cc: i.maxim...@ovn.org > Subject: RE: [ovs-dev] [v12 04/16] dpif-avx512: Add ISA implementation of > dpif. > > > Hi Ian, > > > > Thanks for the review. My responses are inline. > > > > > -Original Message- > > > From: Stokes, Ian > > > Sent: Tuesday 1 June 2021 19:59 > > > To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van > > Haaren, Harry > > > Cc: i.maxim...@ovn.org > > > Subject: RE: [ovs-dev] [v12 04/16] dpif-avx512: Add ISA implementation of > > dpif. > > > > > > > This commit adds the AVX512 implementation of DPIF functionality, > > > > specifically the dp_netdev_input_outer_avx512 function. This function > > > > only handles outer (no re-circulations), and is optimized to use the > > > > AVX512 ISA for packet batching and other DPIF work. > > > > > > > > Sparse is not able to handle the AVX512 intrinsics, causing compile > > > > time failures, so it is disabled for this file. > > > > > > > > Signed-off-by: Harry van Haaren > > > > Co-authored-by: Cian Ferriter > > > > Signed-off-by: Cian Ferriter > > > > > > Thanks for the patch Harry/Cian, still testing this to a degree but > > > questions > > below on initial thoughts. > > > > > > > > > > > --- > > > > > > > > v8: > > > > - Fixup AVX512 mask to uint32_t conversion compilation warning. > > > > --- > > > > lib/automake.mk | 5 +- > > > > lib/dpif-netdev-avx512.c | 265 +++ > > > > lib/dpif-netdev-private-dfc.h| 8 + > > > > lib/dpif-netdev-private-dpif.h | 32 > > > > lib/dpif-netdev-private-thread.h | 11 +- > > > > lib/dpif-netdev-private.h| 25 +++ > > > > lib/dpif-netdev.c| 70 ++-- > > > > 7 files changed, 400 insertions(+), 16 deletions(-) > > > > create mode 100644 lib/dpif-netdev-avx512.c > > > > create mode 100644 lib/dpif-netdev-private-dpif.h > > > > > > > > diff --git a/lib/automake.mk b/lib/automake.mk > > > > index 0bef0cc69..5fab8ba4f 100644 > > > > --- a/lib/automake.mk > > > > +++ b/lib/automake.mk > > > > @@ -33,11 +33,13 @@ lib_libopenvswitchavx512_la_CFLAGS = \ > > > > -mavx512f \ > > > > -mavx512bw \ > > > > -mavx512dq \ > > > > +-mbmi \ > > > > > > Can I ask what's needed in bmi that was not already included in bmi2? Just > > curiosity. > > > > > > > So in the dp_netdev_input_outer_avx512() function (the AVX512 DPIF > > implementation), ' _blsr_u64' is used. > > It's used twice to reset (set to 0) the lowest bit in a variable. > > > > More info on '_blsr_u64': > > https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_blsr_u64; > > expand=463 > > > > > > -mbmi2 \ > > > > -fPIC \ > > > > $(AM_CFLAGS) > > > > lib_libopenvswitchavx512_la_SOURCES = \ > > > > -lib/dpif-netdev-lookup-avx512-gather.c > > > > +lib/dpif-netdev-lookup-avx512-gather.c \ > > > > +lib/dpif-netdev-avx512.c > > > > lib_libopenvswitchavx512_la_LDFLAGS = \ > > > > -static > > > > endif > > > > @@ -113,6 +115,7 @@ lib_libopenvswitch_la_SOURCES = \ > > > > lib/dpif-netdev.h \ > > > > lib/dpif-netdev-private-dfc.h \ > > > > lib/dpif-netdev-private-dpcls.h \ > > > > +lib/dpif-netdev-private-dpif.h \ > > > > lib/dpif-netdev-private-flow.h \ > > > > lib/dpif-netdev-private-hwol.h \ > > > > lib/dpif-netdev-private-thread.h \ > > > > diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c > > > > new file mode 100644 > > > > index 0..91f51c479 > > > > --- /dev/null > > > > +++ b/lib/dpif-netdev-avx512.c > > > > @@ -0,0 +1,265 @@ > > > > +/* > > > > + * Copyright (c) 2021 Intel Corporation. > > > > + * > > > > + * Licensed under the Apache License, Version 2.0 (the "License"); > > > > + * you may not use this file except in compliance with the License. > > > > +
Re: [ovs-dev] [v12 16/16] netdev: Optimize netdev_send_prepare_batch.
Hi Ian,
Thanks for the review. My responses are inline.
> -Original Message-
> From: Stokes, Ian
> Sent: Tuesday 15 June 2021 18:04
> To: Ferriter, Cian ; ovs-dev@openvswitch.org; Van
> Haaren, Harry
> Cc: i.maxim...@ovn.org; Gaetan Rivet ; Sriharsha
> Basavapatna
> Subject: RE: [ovs-dev] [v12 16/16] netdev: Optimize netdev_send_prepare_batch.
>
> > From: Harry van Haaren
> >
> > Optimize for the best case here where all packets will be compatible
> > with 'netdev_flags'.
> >
> > Signed-off-by: Harry van Haaren
> > Co-authored-by: Cian Ferriter
> > Signed-off-by: Cian Ferriter
>
> Thanks for the patch Cian/Harry, comments inline.
>
> In general I have a concern of the impact on HWOL cases here and how much it
> would affect that. Have you data on this? Any
> thoughts from other HWOL folks here?
>From our POV, we are optimizing the normal and error free case. We are
>impacting the error case. When a packet is not compatible with netdev_flags
>that packet will be deleted from the batch and a " VLOG_WARN_RL()" will be
>called. This is quite costly as it is.
I'm curious to see what other HWOL folks think.
> > ---
> > NEWS | 2 ++
> > lib/netdev.c | 31 ++-
> > 2 files changed, 24 insertions(+), 9 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index a7ec34af1..8ad3c98db 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -18,6 +18,8 @@ Post-v2.15.0
> > CPU supports it. This enhances performance by using the native
> > vpopcount
> > instructions, instead of the emulated version of vpopcount.
> > * Optimize dp_netdev_output by enhancing compiler optimization
> > potential.
> > + * Optimize netdev sending by assuming the happy case, and using
> > fallback
> > + for if the netdev doesnt meet the required HWOL needs of a packet.
> This sounds a bit too low level for a NEWS item. Would suggest changing to
> single line Optimize netdev sending for best case scenario.
>
Agreed, your single line sounds better. I've fixed this in the next version.
> BR
> Ian
>
> > - ovs-ctl:
> > * New option '--no-record-hostname' to disable hostname configuration
> > in ovsdb on startup.
> > diff --git a/lib/netdev.c b/lib/netdev.c
> > index 91e91955c..29a5f1aa9 100644
> > --- a/lib/netdev.c
> > +++ b/lib/netdev.c
> > @@ -837,20 +837,33 @@ static void
> > netdev_send_prepare_batch(const struct netdev *netdev,
> >struct dp_packet_batch *batch)
> > {
> > -struct dp_packet *packet;
> > -size_t i, size = dp_packet_batch_size(batch);
> > +struct dp_packet *p;
> > +uint32_t i, size = dp_packet_batch_size(batch);
> > +char *err_msg = NULL;
> >
> > -DP_PACKET_BATCH_REFILL_FOR_EACH (i, size, packet, batch) {
> > -char *errormsg = NULL;
> > +for (i = 0; i < size; i++) {
> > +p = batch->packets[i];
> > +int pkt_ok = netdev_send_prepare_packet(netdev->ol_flags, p,
> > _msg);
> >
> > -if (netdev_send_prepare_packet(netdev->ol_flags, packet,
> > )) {
> > -dp_packet_batch_refill(batch, packet, i);
> > +if (OVS_UNLIKELY(!pkt_ok)) {
> > +goto refill_loop;
> > +}
> > +}
> > +
> > +return;
> > +
> > +refill_loop:
> > +/* Loop through packets from the start of the batch again. This is the
> > + * exceptional case where packets aren't compatible with
> > 'netdev_flags'. */
> > +DP_PACKET_BATCH_REFILL_FOR_EACH (i, size, p, batch) {
> > +if (netdev_send_prepare_packet(netdev->ol_flags, p, _msg)) {
> > +dp_packet_batch_refill(batch, p, i);
> > } else {
> > -dp_packet_delete(packet);
> > +dp_packet_delete(p);
> > COVERAGE_INC(netdev_send_prepare_drops);
> > VLOG_WARN_RL(, "%s: Packet dropped: %s",
> > - netdev_get_name(netdev), errormsg);
> > -free(errormsg);
> > + netdev_get_name(netdev), err_msg);
> > +free(err_msg);
> > }
> > }
> > }
> > --
> > 2.31.1
> >
> > ___
> > dev mailing list
> > d...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
