Re: [ovs-discuss] One question about AVX512 support
On Wed, Jun 8, 2022 at 6:24 PM Van Haaren, Harry wrote: > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:57 PM > > To: Van Haaren, Harry > > Cc: Pai G, Sunil ; ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry > > wrote: > > > > > > Hi Li, Sunil and OVS Discuss mailing list, > > > > > > Answering the direction question inline: > > > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always > > enabled. > > > They are always enabled *assuming* your binutils version does not have > > > bugs. > > > > > > Why are you trying to disable AVX512? It will not run by default, what is > > > the end- > > goal here? > > > Below a large amount of detail around CPU ISA enabling, and how it > > > technically all > > works & why :) > > > > > > > Thanks a lot for your clarification. > > Recently, I encountered a bug. I found that ovs crashes when executing > > avx instructions. > > It's really like this bug: > > https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e > > 00e56d18f19e9911d2370 > > Is the DPDK being built with a buggy (pre-fix) version of binutils? Maybe. I build dpdk/ovs with gcc 7.5.0. Intel(R) Xeon(R) Platinum, Intel(R) Xeon(R) CPU E5-2660. But from the default setting, the AVX512 has been disabled in dpdk. But it fails on: *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 Sometimes, it fails on: 15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c70: 48 83 ea 80 sub$0xff80,%rdx 15c74: 48 83 e9 80 sub$0xff80,%rcx 15c78: 62 f1 7f 08 6f 52 favmovdqu8 -0x60(%rdx),%xmm2 15c7f: c4 e3 65 38 5a 90 01vinserti128 $0x1,-0x70(%rdx),%ymm3,%ymm3 *15c86: 62 f1 7f 08 6f 4a fcvmovdqu8 -0x40(%rdx),%xmm1 15c8d: c4 e3 6d 38 52 b0 01vinserti128 $0x1,-0x50(%rdx),%ymm2,%ymm2 15c94: 62 f1 7f 08 6f 42 fevmovdqu8 -0x20(%rdx),%xmm0 15c9b: c4 e3 75 38 4a d0 01vinserti128 $0x1,-0x30(%rdx),%ymm1,%ymm1 *15ca2: c4 e3 7d 38 42 f0 01vinserti128 $0x1,-0x10(%rdx),%ymm0,%ymm0 15ca9: c5 f8 11 59 80 vmovups %xmm3,-0x80(%rcx) > > > I am not quite sure if avx512 in ovs should be disabled or not. I want > > to make sure that ovs/dpdk disables avx512. > > If it won't run by default, I think I can leave it alone. > > Given the test binutils bug-check passes on the OVS build, it is to be > expected that no issues arise from *OVS* code. If you link OVS against > a DPDK built-with-buggy-binutils, you can still crash in that code. > I see. > For OVS code, you can explicitly disable by modifying the m4/openvswitch.m4 > file at +424 "checks for binutils/assermbler known issue with AVX512". > Always return "=no" (on line 442 here). That will emulate the "bad" binutils, > resulting in full disabling of all AVX512 in the *OVS* build. > I see, thanks. I would like to try to disable avx512 , avx2 or avx. > In the intro email, you mentioned ovs and DPDK versions as follows: > openvswitch 2.14.2, and dpdk-19.11.4 > > There is a 19.11.12 available now, although I didn't find relevant fixes in > the changelog; > https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html# > > To force disable AVX512 in DPDK, pass "-mno-avx512f" as "machine_args", or > modify > /config/x86/meson.build and set "binutils_ok = false" on line 5. > > Then recompile DPDK, and ensure OVS is linking against your newly compiled > DPDK. OK, thanks for your suggestion. I will try it. > > Hope that helps in your root-causing! Regards, -Harry > > > > > -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
> -Original Message- > From: Li Zhang > Sent: Wednesday, June 8, 2022 4:57 PM > To: Van Haaren, Harry > Cc: Pai G, Sunil ; ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] One question about AVX512 support > > On Wed, Jun 8, 2022 at 5:10 PM Van Haaren, Harry > wrote: > > > > Hi Li, Sunil and OVS Discuss mailing list, > > > > Answering the direction question inline: > > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always > enabled. > > They are always enabled *assuming* your binutils version does not have bugs. > > > > Why are you trying to disable AVX512? It will not run by default, what is > > the end- > goal here? > > Below a large amount of detail around CPU ISA enabling, and how it > > technically all > works & why :) > > > > Thanks a lot for your clarification. > Recently, I encountered a bug. I found that ovs crashes when executing > avx instructions. > It's really like this bug: > https://inbox.dpdk.org/dev/31482910.FCGgztJ3Sx@xps/T/#m95492563f7e2819395e > 00e56d18f19e9911d2370 Is the DPDK being built with a buggy (pre-fix) version of binutils? > I am not quite sure if avx512 in ovs should be disabled or not. I want > to make sure that ovs/dpdk disables avx512. > If it won't run by default, I think I can leave it alone. Given the test binutils bug-check passes on the OVS build, it is to be expected that no issues arise from *OVS* code. If you link OVS against a DPDK built-with-buggy-binutils, you can still crash in that code. For OVS code, you can explicitly disable by modifying the m4/openvswitch.m4 file at +424 "checks for binutils/assermbler known issue with AVX512". Always return "=no" (on line 442 here). That will emulate the "bad" binutils, resulting in full disabling of all AVX512 in the *OVS* build. In the intro email, you mentioned ovs and DPDK versions as follows: openvswitch 2.14.2, and dpdk-19.11.4 There is a 19.11.12 available now, although I didn't find relevant fixes in the changelog; https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html# To force disable AVX512 in DPDK, pass "-mno-avx512f" as "machine_args", or modify /config/x86/meson.build and set "binutils_ok = false" on line 5. Then recompile DPDK, and ensure OVS is linking against your newly compiled DPDK. Hope that helps in your root-causing! Regards, -Harry ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
e pkg-config file in OVS's build configuration > stage, as OVS would like control over "-march" explicitly. > > Lastly, as DPDK and OVS could be built with *different* binutils versions (or > even on different compile machines!), both OVS > and DPDK check for the buggy-binutils themselves. As a result, it is possible > to have a DPDK/OVS combo with/without AVX512 > enabled. > > # Concerns around AVX512 enabling > Note that neither DPDK nor OVS enables AVX512 without user input. DPDK > defaults to AVX2/256 bit wide (ymm) registers, > and only uses AVX512 if the --force-max-simd-bitwidth=512 EAL argument is > passed: no reason for concern here. > > OVS will use scalar implementations, unless the appropriate "ovs-appctl > dpif-netdev/*" command is run to enable AVX512 > optimized routines (documentation here; > https://docs.openvswitch.org/en/latest/topics/dpdk/bridge/#datapath-classifier-performance) > > All in all, CPU ISA is compiled in by default, and not enabled. It can be > enabled manually by the user, and will then ensure that > the required CPU ISA is available on the runtime CPU, before actually > executing the optimized instruction sequence. As a result > binaries can happily contain AVX512 optimized code, it just won't be executed > unless the user requests it. > > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 3:51 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org; Van Haaren, Harry > > > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Thanks a lot. > > > > Hi Harry, > > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. > > > > On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > > > > > Adding Harry to help answer these questions on DPDK and OVS building with > > AVX512. > > > > > > Thanks and Regards, > > > Sunil > > > > > > > -Original Message- > > > > From: Li Zhang > > > > Sent: Wednesday, June 8, 2022 4:26 PM > > > > To: Pai G, Sunil > > > > Cc: ovs-discuss@openvswitch.org > > > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > > > > > Hi Pai, > > > > > > > > I have been trying to disable avx512 in OVS for the platform which > > > > doesn't > > > > support avx512. > > > > Building fails and it seems that it is not disabled. Any idea about it? > > > > > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > > > # make > > > > > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused- > > parameter > > > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > > > definition -Wmissing-prototypes -Wmissing-field-initializers > > > > -fno-strict- > > > > aliasing -Wswitch-bool -Wlogical-not-parentheses > > > > -Wsizeof-array-argument - > > > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 > > > > - > > > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F - > > DHAVE_LD_AVX512_GOOD - > > > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c > > > > lib/netdev- > > > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > > > suse-linux/7/include/immintrin.h:41:0, > > > > from /usr/lib64/gcc/x86_64-suse- > > > > linux/7/include/x86intrin.h:48, > > > > from /usr/local/include/rte_vect.h:28, > > > > from /usr/local/include/rte_memcpy.h:17, > > > > from /usr/local/include/rte_ether.h:21, > > > > from /usr/local/include/rte_ethdev.h:159, > > > > from lib/netdev-dpdk.c:39: > > > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > > > specific option mismatch > > > > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A
Re: [ovs-discuss] One question about AVX512 support
CPU ISA is compiled in by default, and not enabled. It can be enabled manually by the user, and will then ensure that the required CPU ISA is available on the runtime CPU, before actually executing the optimized instruction sequence. As a result binaries can happily contain AVX512 optimized code, it just won't be executed unless the user requests it. > -Original Message----- > From: Li Zhang > Sent: Wednesday, June 8, 2022 3:51 PM > To: Pai G, Sunil > Cc: ovs-discuss@openvswitch.org; Van Haaren, Harry > > Subject: Re: [ovs-discuss] One question about AVX512 support > > Thanks a lot. > > Hi Harry, > Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. > > On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > > > Adding Harry to help answer these questions on DPDK and OVS building with > AVX512. > > > > Thanks and Regards, > > Sunil > > > > > -Original Message- > > > From: Li Zhang > > > Sent: Wednesday, June 8, 2022 4:26 PM > > > To: Pai G, Sunil > > > Cc: ovs-discuss@openvswitch.org > > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > > > Hi Pai, > > > > > > I have been trying to disable avx512 in OVS for the platform which doesn't > > > support avx512. > > > Building fails and it seems that it is not disabled. Any idea about it? > > > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > > # make > > > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused- > parameter > > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > > > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F - > DHAVE_LD_AVX512_GOOD - > > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > > suse-linux/7/include/immintrin.h:41:0, > > > from /usr/lib64/gcc/x86_64-suse- > > > linux/7/include/x86intrin.h:48, > > > from /usr/local/include/rte_vect.h:28, > > > from /usr/local/include/rte_memcpy.h:17, > > > from /usr/local/include/rte_ether.h:21, > > > from /usr/local/include/rte_ethdev.h:159, > > > from lib/netdev-dpdk.c:39: > > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > > specific option mismatch > > > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ > In > > > file included from /usr/local/include/rte_ether.h:21:0, > > > from /usr/local/include/rte_ethdev.h:159, > > > from lib/netdev-dpdk.c:39: > > > /usr/local/include/rte_memcpy.h:320:2: note: called from here > > > _mm256_storeu_si256((__m256i *)dst, ymm0); > > > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > > > Hi Li, > > > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > > true. > > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > > helps. > > > > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > > Author: Ian Stokes > > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > > used to build DPDK had broken AVX512 support. > > > > > > > >
Re: [ovs-discuss] One question about AVX512 support
Thanks a lot. Hi Harry, Any ideas? -DHAVE_AVX512F and -DHAVE_LD_AVX512_GOOD are always enabled. On Wed, Jun 8, 2022 at 3:15 PM Pai G, Sunil wrote: > > Adding Harry to help answer these questions on DPDK and OVS building with > AVX512. > > Thanks and Regards, > Sunil > > > -Original Message- > > From: Li Zhang > > Sent: Wednesday, June 8, 2022 4:26 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Hi Pai, > > > > I have been trying to disable avx512 in OVS for the platform which doesn't > > support avx512. > > Building fails and it seems that it is not disabled. Any idea about it? > > > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > > sysconfdir=/etc CFLAGS="-mno-avx512f" > > # make > > > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter > > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD - > > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > > suse-linux/7/include/immintrin.h:41:0, > > from /usr/lib64/gcc/x86_64-suse- > > linux/7/include/x86intrin.h:48, > > from /usr/local/include/rte_vect.h:28, > > from /usr/local/include/rte_memcpy.h:17, > > from /usr/local/include/rte_ether.h:21, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > > specific option mismatch > > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In > > file included from /usr/local/include/rte_ether.h:21:0, > > from /usr/local/include/rte_ethdev.h:159, > > from lib/netdev-dpdk.c:39: > > /usr/local/include/rte_memcpy.h:320:2: note: called from here > > _mm256_storeu_si256((__m256i *)dst, ymm0); > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > Hi Li, > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > true. > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > helps. > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > Author: Ian Stokes > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. But OVS could be > > built > > > with a completely different or fixed toolchain with correct avx512 > > > support. > > > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > > march'. > > > This will allow the OVS to decide if the AVX512 can be used. > > > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > > an > > > option since autotools might reorder them back later and it's very > > > unpredictable. > > > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > > Signed-off-by: Ilya Maximets > > > Co-authored-by: Ilya Maximets > > > Signed-off-by: Ian Stokes > > > > > > commit e9f9104d6a83ce7efd7021
Re: [ovs-discuss] One question about AVX512 support
Adding Harry to help answer these questions on DPDK and OVS building with AVX512. Thanks and Regards, Sunil > -Original Message- > From: Li Zhang > Sent: Wednesday, June 8, 2022 4:26 PM > To: Pai G, Sunil > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] One question about AVX512 support > > Hi Pai, > > I have been trying to disable avx512 in OVS for the platform which doesn't > support avx512. > Building fails and it seems that it is not disabled. Any idea about it? > > # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var -- > sysconfdir=/etc CFLAGS="-mno-avx512f" > # make > > libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I > ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare - > Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter > -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style- > definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict- > aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument - > Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 - > include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include > -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD - > MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev- > dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64- > suse-linux/7/include/immintrin.h:41:0, > from /usr/lib64/gcc/x86_64-suse- > linux/7/include/x86intrin.h:48, > from /usr/local/include/rte_vect.h:28, > from /usr/local/include/rte_memcpy.h:17, > from /usr/local/include/rte_ether.h:21, > from /usr/local/include/rte_ethdev.h:159, > from lib/netdev-dpdk.c:39: > /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: > /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: > inlining failed in call to always_inline ‘_mm256_storeu_si256’: target > specific option mismatch > _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In > file included from /usr/local/include/rte_ether.h:21:0, > from /usr/local/include/rte_ethdev.h:159, > from lib/netdev-dpdk.c:39: > /usr/local/include/rte_memcpy.h:320:2: note: called from here > _mm256_storeu_si256((__m256i *)dst, ymm0); > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > Hi Li, > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > true. > > I found these two commits below in ovs-2.14.2 which strips out the "- > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > DPDK. The reason for this is rightly mentioned below as well. Hope this > helps. > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > Author: Ian Stokes > > Date: Fri Jan 15 15:46:02 2021 + > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. But OVS could be > built > > with a completely different or fixed toolchain with correct avx512 > > support. > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > march'. > > This will allow the OVS to decide if the AVX512 can be used. > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > an > > option since autotools might reorder them back later and it's very > > unpredictable. > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > Signed-off-by: Ilya Maximets > > Co-authored-by: Ilya Maximets > > Signed-off-by: Ian Stokes > > > > commit e9f9104d6a83ce7efd702120171835991779 > > Author: Ian Stokes > > Date: Fri Jan 15 14:54:04 2021 + > > > > acinclude: Strip out -march provided by DPDK. > > > > DPDK flags may include -march. Forcing -march could be > > considered too heavy a requirement when users compile OVS from > > source and could override user provided options. > > > > Resolve this by stripping -march from provided DPDK flags. > > > > Signed-off-by: Ian Stokes > > > > > > > > Thanks and Regards, > > Sunil > > > > > -Or
Re: [ovs-discuss] One question about AVX512 support
Hi Pai, I have been trying to disable avx512 in OVS for the platform which doesn't support avx512. Building fails and it seems that it is not disabled. Any idea about it? # ./configure --with-dpdk=yes --prefix=/usr --localstatedir=/var --sysconfdir=/etc CFLAGS="-mno-avx512f" # make libtool: compile: gcc -DHAVE_CONFIG_H -I. -I ./include -I ./include -I ./lib -I ./lib -Wstrict-prototypes -Wall -Wextra -Wno-sign-compare -Wpointer-arith -Wformat -Wformat-security -Wswitch-enum -Wunused-parameter -Wbad-function-cast -Wcast-align -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-field-initializers -fno-strict-aliasing -Wswitch-bool -Wlogical-not-parentheses -Wsizeof-array-argument -Wbool-compare -Wshift-negative-value -Wduplicated-cond -Wshadow -mssse3 -include rte_config.h -DOPENSSL_LOAD_CONF -I/usr/local/include -D_FILE_OFFSET_BITS=64 -mno-avx512f -DHAVE_AVX512F -DHAVE_LD_AVX512_GOOD -MT lib/netdev-dpdk.lo -MD -MP -MF lib/.deps/netdev-dpdk.Tpo -c lib/netdev-dpdk.c -o lib/netdev-dpdk.o In file included from /usr/lib64/gcc/x86_64-suse-linux/7/include/immintrin.h:41:0, from /usr/lib64/gcc/x86_64-suse-linux/7/include/x86intrin.h:48, from /usr/local/include/rte_vect.h:28, from /usr/local/include/rte_memcpy.h:17, from /usr/local/include/rte_ether.h:21, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h: In function ‘rte_mov32’: /usr/lib64/gcc/x86_64-suse-linux/7/include/avxintrin.h:926:1: error: inlining failed in call to always_inline ‘_mm256_storeu_si256’: target specific option mismatch _mm256_storeu_si256 (__m256i_u *__P, __m256i __A) ^~~ In file included from /usr/local/include/rte_ether.h:21:0, from /usr/local/include/rte_ethdev.h:159, from lib/netdev-dpdk.c:39: /usr/local/include/rte_memcpy.h:320:2: note: called from here _mm256_storeu_si256((__m256i *)dst, ymm0); On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > Hi Li, > > The assumption of ovs being dependent on dpdk for avx512 might not be true. > I found these two commits below in ovs-2.14.2 which strips out the "-march" > and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The > reason for this is rightly mentioned below as well. Hope this helps. > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > Author: Ian Stokes > Date: Fri Jan 15 15:46:02 2021 + > > acinclude: Strip out -mno-avx512f provided by DPDK. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. But OVS could be built > with a completely different or fixed toolchain with correct avx512 > support. > > Fix that by stripping out `-mno-avx512f` as we already do for '-march'. > This will allow the OVS to decide if the AVX512 can be used. > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an > option since autotools might reorder them back later and it's very > unpredictable. > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > Signed-off-by: Ilya Maximets > Co-authored-by: Ilya Maximets > Signed-off-by: Ian Stokes > > commit e9f9104d6a83ce7efd702120171835991779 > Author: Ian Stokes > Date: Fri Jan 15 14:54:04 2021 + > > acinclude: Strip out -march provided by DPDK. > > DPDK flags may include -march. Forcing -march could be > considered too heavy a requirement when users compile OVS from > source and could override user provided options. > > Resolve this by stripping -march from provided DPDK flags. > > Signed-off-by: Ian Stokes > > > > Thanks and Regards, > Sunil > > > -Original Message- > > From: discuss On Behalf Of Li Zhang > > Sent: Thursday, June 2, 2022 6:35 PM > > To: ovs-discuss@openvswitch.org > > Subject: [ovs-discuss] One question about AVX512 support > > > > Hi all, > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > enabled by default but it's disabled in DPDK. But I think ovs is dependent > > on the dpdk library, right? But why does ovs work with > > avx512 disabled in DPDK? > > > > I am not quite sure about the relationship between OVS and DPDK, any > > suggestions? > > > > -- > > > > Best Regards > > -Li > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 7:31 PM Pai G, Sunil wrote: > > Hi Li, > > My responses are inline. > > Thanks and Regards, > Sunil > > > -Original Message- > > From: Li Zhang > > Sent: Thursday, June 2, 2022 9:56 PM > > To: Pai G, Sunil > > Cc: ovs-discuss@openvswitch.org > > Subject: Re: [ovs-discuss] One question about AVX512 support > > > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Well,...not quite, they simply remove the dependency on DPDK i.e > the decision to use AVX512/not in OVS is left to OVS only [DPDK arch > options/flags are simply ignored]. > I see. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > To get better performance and make the most out of the features offered by > the platform, I would say so, yes. > > > It is disabled in DPDK because it is experimental. > > I would recommend enabling AVX-512 in DPDK as well. > Here is a link which describes how to: > https://doc.dpdk.org/guides/howto/avx512.html > > i.e for ovs, use the following command before launching vswitchd: > ovs-vsctl --no-wait set Open_vSwitch . > other_config:dpdk-extra="--force-max-simd-bitwidth=512" > > > > I wonder if I should disable it, it may cause performance worse. > > Here are few links I have handy that describe the performance improvements > seen with AVX-512 along with its impacts. > https://networkbuilders.intel.com/solutionslibrary/open-vswitch-optimized-deployment-benchmark-technology-guide > . > https://networkbuilders.intel.com/intel-technologies/3rd-gen-intel-xeon-scalable-processors-experience-kits > , Acceleration section. > > I would encourage you to go through these links. > Really really thanks. It's very useful. I need to look at it. > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > > > Hi Li, > > > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > > true. > > > I found these two commits below in ovs-2.14.2 which strips out the "- > > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > > DPDK. The reason for this is rightly mentioned below as well. Hope this > > helps. > > > > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > > Author: Ian Stokes > > > Date: Fri Jan 15 15:46:02 2021 + > > > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. > > > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > > used to build DPDK had broken AVX512 support. But OVS could be > > built > > > with a completely different or fixed toolchain with correct avx512 > > > support. > > > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > > march'. > > > This will allow the OVS to decide if the AVX512 can be used. > > > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > > an > > > option since autotools might reorder them back later and it's very > > > unpredictable. > > > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > > Signed-off-by: Ilya Maximets > > > Co-authored-by: Ilya Maximets > > > Signed-off-by: Ian Stokes > > > > > > commit e9f9104d6a83ce7efd702120171835991779 > > > Author: Ian Stokes > > > Date: Fri Jan 15 14:54:04 2021 + > > > > > > acinclude: Strip out -march provided by DPDK. > > > > > > DPDK flags may include -march. Forcing -march could be > > > considered too heavy a requirement when users compile OVS from > > > source and could override user provided options. > > > > > > Resolve this by stripping -march from provided DPDK flags. > > > > > > Signed-off-by: Ian Stokes > > > > > > > > > > > > Thanks and Regards, > > > Sunil > > > > > > > -Original Message- > > > > From: discuss On Behalf Of Li > > > > Zhang > > > > Sent: Thursday, June 2, 2022 6:35 PM > > > > To: ovs-discuss@openvswitch.org
Re: [ovs-discuss] One question about AVX512 support
Hi Li, My responses are inline. Thanks and Regards, Sunil > -Original Message- > From: Li Zhang > Sent: Thursday, June 2, 2022 9:56 PM > To: Pai G, Sunil > Cc: ovs-discuss@openvswitch.org > Subject: Re: [ovs-discuss] One question about AVX512 support > > Hi Pai, > > Thanks a lot. It helps. > These patches allow ovs to decide to disable or enable AVX512 if I > understand it correctly. Well,...not quite, they simply remove the dependency on DPDK i.e the decision to use AVX512/not in OVS is left to OVS only [DPDK arch options/flags are simply ignored]. > Is it recommended to enable AVX512? AVX512 is enabled by default. To get better performance and make the most out of the features offered by the platform, I would say so, yes. > It is disabled in DPDK because it is experimental. I would recommend enabling AVX-512 in DPDK as well. Here is a link which describes how to: https://doc.dpdk.org/guides/howto/avx512.html i.e for ovs, use the following command before launching vswitchd: ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra="--force-max-simd-bitwidth=512" > I wonder if I should disable it, it may cause performance worse. Here are few links I have handy that describe the performance improvements seen with AVX-512 along with its impacts. https://networkbuilders.intel.com/solutionslibrary/open-vswitch-optimized-deployment-benchmark-technology-guide . https://networkbuilders.intel.com/intel-technologies/3rd-gen-intel-xeon-scalable-processors-experience-kits , Acceleration section. I would encourage you to go through these links. > > Thanks > Li > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > > > Hi Li, > > > > The assumption of ovs being dependent on dpdk for avx512 might not be > true. > > I found these two commits below in ovs-2.14.2 which strips out the "- > march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on > DPDK. The reason for this is rightly mentioned below as well. Hope this > helps. > > > > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > > Author: Ian Stokes > > Date: Fri Jan 15 15:46:02 2021 + > > > > acinclude: Strip out -mno-avx512f provided by DPDK. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. > > > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > > used to build DPDK had broken AVX512 support. But OVS could be > built > > with a completely different or fixed toolchain with correct avx512 > > support. > > > > Fix that by stripping out `-mno-avx512f` as we already do for '- > march'. > > This will allow the OVS to decide if the AVX512 can be used. > > > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > an > > option since autotools might reorder them back later and it's very > > unpredictable. > > > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > > Signed-off-by: Ilya Maximets > > Co-authored-by: Ilya Maximets > > Signed-off-by: Ian Stokes > > > > commit e9f9104d6a83ce7efd702120171835991779 > > Author: Ian Stokes > > Date: Fri Jan 15 14:54:04 2021 + > > > > acinclude: Strip out -march provided by DPDK. > > > > DPDK flags may include -march. Forcing -march could be > > considered too heavy a requirement when users compile OVS from > > source and could override user provided options. > > > > Resolve this by stripping -march from provided DPDK flags. > > > > Signed-off-by: Ian Stokes > > > > > > > > Thanks and Regards, > > Sunil > > > > > -Original Message- > > > From: discuss On Behalf Of Li > > > Zhang > > > Sent: Thursday, June 2, 2022 6:35 PM > > > To: ovs-discuss@openvswitch.org > > > Subject: [ovs-discuss] One question about AVX512 support > > > > > > Hi all, > > > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > > enabled by default but it's disabled in DPDK. But I think ovs is > > > dependent on the dpdk library, right? But why does ovs work with > > > avx512 disabled in DPDK? > > > > > > I am not quite sure about the relationship between OVS and DPDK, any > > > suggestions? > > > > > > -- > > > > > > Best Regards > > > -Li > > > ___ > > > discuss mailing list > > > disc...@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > -- > > Best Regards > -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 6:56 PM Raymond Burkholder wrote: > > > > On 2022-06-02 10:26, Li Zhang wrote: > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > It is disabled in DPDK because it is experimental. I wonder if I > > should disable it, it may cause performance worse. > > Look at https://en.wikipedia.org/wiki/AVX-512 Got it, thanks. > > AVX512 is CPU model dependent. Code will break if the instructions are > run on a cpu without them. Which is why they are typically not-compiled > in. Each environment will need to make that decision explicit. If your > environment supports the instructions, then I'd probably compile in for > OVS & DPRK. If you need code to transport all over the place, then best > to disable them. > > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > >> Hi Li, > >> > >> The assumption of ovs being dependent on dpdk for avx512 might not be true. > >> I found these two commits below in ovs-2.14.2 which strips out the > >> "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency > >> on DPDK. The reason for this is rightly mentioned below as well. Hope this > >> helps. > >> > >> > >> commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > >> Author: Ian Stokes > >> Date: Fri Jan 15 15:46:02 2021 + > >> > >> acinclude: Strip out -mno-avx512f provided by DPDK. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. But OVS could be built > >> with a completely different or fixed toolchain with correct avx512 > >> support. > >> > >> Fix that by stripping out `-mno-avx512f` as we already do for > >> '-march'. > >> This will allow the OVS to decide if the AVX512 can be used. > >> > >> Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > >> an > >> option since autotools might reorder them back later and it's very > >> unpredictable. > >> > >> Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > >> Signed-off-by: Ilya Maximets > >> Co-authored-by: Ilya Maximets > >> Signed-off-by: Ian Stokes > >> > >> commit e9f9104d6a83ce7efd702120171835991779 > >> Author: Ian Stokes > >> Date: Fri Jan 15 14:54:04 2021 + > >> > >> acinclude: Strip out -march provided by DPDK. > >> > >> DPDK flags may include -march. Forcing -march could be > >> considered too heavy a requirement when users compile OVS from > >> source and could override user provided options. > >> > >> Resolve this by stripping -march from provided DPDK flags. > >> > >> Signed-off-by: Ian Stokes > >> > >> > >> > >> Thanks and Regards, > >> Sunil > >> > >>> -Original Message- > >>> From: discuss On Behalf Of Li Zhang > >>> Sent: Thursday, June 2, 2022 6:35 PM > >>> To: ovs-discuss@openvswitch.org > >>> Subject: [ovs-discuss] One question about AVX512 support > >>> > >>> Hi all, > >>> > >>> We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > >>> enabled by default but it's disabled in DPDK. But I think ovs is dependent > >>> on the dpdk library, right? But why does ovs work with > >>> avx512 disabled in DPDK? > >>> > >>> I am not quite sure about the relationship between OVS and DPDK, any > >>> suggestions? > >>> > >>> -- > >>> > >>> Best Regards > >>> -Li > >>> ___ > >>> discuss mailing list > >>> disc...@openvswitch.org > >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On 2022-06-02 10:26, Li Zhang wrote: Hi Pai, Thanks a lot. It helps. These patches allow ovs to decide to disable or enable AVX512 if I understand it correctly. Is it recommended to enable AVX512? AVX512 is enabled by default. It is disabled in DPDK because it is experimental. I wonder if I should disable it, it may cause performance worse. Look at https://en.wikipedia.org/wiki/AVX-512 AVX512 is CPU model dependent. Code will break if the instructions are run on a cpu without them. Which is why they are typically not-compiled in. Each environment will need to make that decision explicit. If your environment supports the instructions, then I'd probably compile in for OVS & DPRK. If you need code to transport all over the place, then best to disable them. Thanks Li On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: Hi Li, The assumption of ovs being dependent on dpdk for avx512 might not be true. I found these two commits below in ovs-2.14.2 which strips out the "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The reason for this is rightly mentioned below as well. Hope this helps. commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 Author: Ian Stokes Date: Fri Jan 15 15:46:02 2021 + acinclude: Strip out -mno-avx512f provided by DPDK. DPDK forces '-mno-avx512f' flag for the application if the toolchain used to build DPDK had broken AVX512 support. DPDK forces '-mno-avx512f' flag for the application if the toolchain used to build DPDK had broken AVX512 support. But OVS could be built with a completely different or fixed toolchain with correct avx512 support. Fix that by stripping out `-mno-avx512f` as we already do for '-march'. This will allow the OVS to decide if the AVX512 can be used. Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an option since autotools might reorder them back later and it's very unpredictable. Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 Signed-off-by: Ilya Maximets Co-authored-by: Ilya Maximets Signed-off-by: Ian Stokes commit e9f9104d6a83ce7efd702120171835991779 Author: Ian Stokes Date: Fri Jan 15 14:54:04 2021 + acinclude: Strip out -march provided by DPDK. DPDK flags may include -march. Forcing -march could be considered too heavy a requirement when users compile OVS from source and could override user provided options. Resolve this by stripping -march from provided DPDK flags. Signed-off-by: Ian Stokes Thanks and Regards, Sunil -Original Message- From: discuss On Behalf Of Li Zhang Sent: Thursday, June 2, 2022 6:35 PM To: ovs-discuss@openvswitch.org Subject: [ovs-discuss] One question about AVX512 support Hi all, We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is enabled by default but it's disabled in DPDK. But I think ovs is dependent on the dpdk library, right? But why does ovs work with avx512 disabled in DPDK? I am not quite sure about the relationship between OVS and DPDK, any suggestions? -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
On Thu, Jun 2, 2022 at 6:30 PM Raymond Burkholder wrote: > > > > On 2022-06-02 10:26, Li Zhang wrote: > > Hi Pai, > > > > Thanks a lot. It helps. > > These patches allow ovs to decide to disable or enable AVX512 if I > > understand it correctly. > > Is it recommended to enable AVX512? AVX512 is enabled by default. > > It is disabled in DPDK because it is experimental. I wonder if I > > should disable it, it may cause performance worse. > > AVX512 is CPU model dependent. Code will break if the instructions are > run on a cpu without them. Which is why they are typically not-compiled > in. Each environment will need to make that decision explicit. If your > environment supports the instructions, then I'd probably compile in for > OVS & DPRK. If you need code to transport all over the place, then best > to disable them. Got it, thanks! > > > > > Thanks > > Li > > > > On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > >> Hi Li, > >> > >> The assumption of ovs being dependent on dpdk for avx512 might not be true. > >> I found these two commits below in ovs-2.14.2 which strips out the > >> "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency > >> on DPDK. The reason for this is rightly mentioned below as well. Hope this > >> helps. > >> > >> > >> commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > >> Author: Ian Stokes > >> Date: Fri Jan 15 15:46:02 2021 + > >> > >> acinclude: Strip out -mno-avx512f provided by DPDK. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. > >> > >> DPDK forces '-mno-avx512f' flag for the application if the toolchain > >> used to build DPDK had broken AVX512 support. But OVS could be built > >> with a completely different or fixed toolchain with correct avx512 > >> support. > >> > >> Fix that by stripping out `-mno-avx512f` as we already do for > >> '-march'. > >> This will allow the OVS to decide if the AVX512 can be used. > >> > >> Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not > >> an > >> option since autotools might reorder them back later and it's very > >> unpredictable. > >> > >> Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > >> Signed-off-by: Ilya Maximets > >> Co-authored-by: Ilya Maximets > >> Signed-off-by: Ian Stokes > >> > >> commit e9f9104d6a83ce7efd702120171835991779 > >> Author: Ian Stokes > >> Date: Fri Jan 15 14:54:04 2021 + > >> > >> acinclude: Strip out -march provided by DPDK. > >> > >> DPDK flags may include -march. Forcing -march could be > >> considered too heavy a requirement when users compile OVS from > >> source and could override user provided options. > >> > >> Resolve this by stripping -march from provided DPDK flags. > >> > >> Signed-off-by: Ian Stokes > >> > >> > >> > >> Thanks and Regards, > >> Sunil > >> > >>> -Original Message- > >>> From: discuss On Behalf Of Li Zhang > >>> Sent: Thursday, June 2, 2022 6:35 PM > >>> To: ovs-discuss@openvswitch.org > >>> Subject: [ovs-discuss] One question about AVX512 support > >>> > >>> Hi all, > >>> > >>> We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > >>> enabled by default but it's disabled in DPDK. But I think ovs is dependent > >>> on the dpdk library, right? But why does ovs work with > >>> avx512 disabled in DPDK? > >>> > >>> I am not quite sure about the relationship between OVS and DPDK, any > >>> suggestions? > >>> > >>> -- > >>> > >>> Best Regards > >>> -Li > >>> ___ > >>> discuss mailing list > >>> disc...@openvswitch.org > >>> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > > > > -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
Hi Pai, Thanks a lot. It helps. These patches allow ovs to decide to disable or enable AVX512 if I understand it correctly. Is it recommended to enable AVX512? AVX512 is enabled by default. It is disabled in DPDK because it is experimental. I wonder if I should disable it, it may cause performance worse. Thanks Li On Thu, Jun 2, 2022 at 4:00 PM Pai G, Sunil wrote: > > Hi Li, > > The assumption of ovs being dependent on dpdk for avx512 might not be true. > I found these two commits below in ovs-2.14.2 which strips out the "-march" > and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The > reason for this is rightly mentioned below as well. Hope this helps. > > > commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 > Author: Ian Stokes > Date: Fri Jan 15 15:46:02 2021 + > > acinclude: Strip out -mno-avx512f provided by DPDK. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. > > DPDK forces '-mno-avx512f' flag for the application if the toolchain > used to build DPDK had broken AVX512 support. But OVS could be built > with a completely different or fixed toolchain with correct avx512 > support. > > Fix that by stripping out `-mno-avx512f` as we already do for '-march'. > This will allow the OVS to decide if the AVX512 can be used. > > Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an > option since autotools might reorder them back later and it's very > unpredictable. > > Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 > Signed-off-by: Ilya Maximets > Co-authored-by: Ilya Maximets > Signed-off-by: Ian Stokes > > commit e9f9104d6a83ce7efd702120171835991779 > Author: Ian Stokes > Date: Fri Jan 15 14:54:04 2021 + > > acinclude: Strip out -march provided by DPDK. > > DPDK flags may include -march. Forcing -march could be > considered too heavy a requirement when users compile OVS from > source and could override user provided options. > > Resolve this by stripping -march from provided DPDK flags. > > Signed-off-by: Ian Stokes > > > > Thanks and Regards, > Sunil > > > -----Original Message- > > From: discuss On Behalf Of Li Zhang > > Sent: Thursday, June 2, 2022 6:35 PM > > To: ovs-discuss@openvswitch.org > > Subject: [ovs-discuss] One question about AVX512 support > > > > Hi all, > > > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > > enabled by default but it's disabled in DPDK. But I think ovs is dependent > > on the dpdk library, right? But why does ovs work with > > avx512 disabled in DPDK? > > > > I am not quite sure about the relationship between OVS and DPDK, any > > suggestions? > > > > -- > > > > Best Regards > > -Li > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] One question about AVX512 support
Hi Li, The assumption of ovs being dependent on dpdk for avx512 might not be true. I found these two commits below in ovs-2.14.2 which strips out the "-march" and "-mno-avx512f" flags exported by dpdk i.e removes dependency on DPDK. The reason for this is rightly mentioned below as well. Hope this helps. commit bb8f0e2a810889241f1d886d160ccee9b96c4d63 Author: Ian Stokes Date: Fri Jan 15 15:46:02 2021 + acinclude: Strip out -mno-avx512f provided by DPDK. DPDK forces '-mno-avx512f' flag for the application if the toolchain used to build DPDK had broken AVX512 support. DPDK forces '-mno-avx512f' flag for the application if the toolchain used to build DPDK had broken AVX512 support. But OVS could be built with a completely different or fixed toolchain with correct avx512 support. Fix that by stripping out `-mno-avx512f` as we already do for '-march'. This will allow the OVS to decide if the AVX512 can be used. Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an option since autotools might reorder them back later and it's very unpredictable. Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 Signed-off-by: Ilya Maximets Co-authored-by: Ilya Maximets Signed-off-by: Ian Stokes commit e9f9104d6a83ce7efd702120171835991779 Author: Ian Stokes Date: Fri Jan 15 14:54:04 2021 + acinclude: Strip out -march provided by DPDK. DPDK flags may include -march. Forcing -march could be considered too heavy a requirement when users compile OVS from source and could override user provided options. Resolve this by stripping -march from provided DPDK flags. Signed-off-by: Ian Stokes Thanks and Regards, Sunil > -Original Message- > From: discuss On Behalf Of Li Zhang > Sent: Thursday, June 2, 2022 6:35 PM > To: ovs-discuss@openvswitch.org > Subject: [ovs-discuss] One question about AVX512 support > > Hi all, > > We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is > enabled by default but it's disabled in DPDK. But I think ovs is dependent > on the dpdk library, right? But why does ovs work with > avx512 disabled in DPDK? > > I am not quite sure about the relationship between OVS and DPDK, any > suggestions? > > -- > > Best Regards > -Li > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] One question about AVX512 support
Hi all, We are using openvswitch 2.14.2, and dpdk-19.11.4. I found avx512 is enabled by default but it's disabled in DPDK. But I think ovs is dependent on the dpdk library, right? But why does ovs work with avx512 disabled in DPDK? I am not quite sure about the relationship between OVS and DPDK, any suggestions? -- Best Regards -Li ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
