Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Matthew Smith via lists.fd.io 
On Fri, Oct 2, 2020 at 12:52 PM Christian Hopps  wrote:

> I just noticed this error I'm getting
>
> net_mlx4: cannot load glue library:
> /home/chopps/w/vpp/build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue/librte_pmd_mlx4_glue.so.18.02.0:
> cannot open shared object file: No such file or directory
> net_mlx4: cannot initialize PMD due to missing run-time dependency on
> rdma-core libraries (libibverbs, libmlx4)
> common_mlx5: Cannot load glue library:
> /home/chopps/w/vpp/build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue/librte_pmd_mlx5_glue.so.20.02.0:
> cannot open shared object file: No such file or directory
> common_mlx5: Cannot initialize MLX5 common due to missing run-time
> dependency on rdma-core libraries (libibverbs, libmlx5)
>
> And:
>
> (default) [17:50:17 ...]$ find ../build-root/*-vpp-native/external -name
> '*glue*'
>
> ../build-root/build-vpp-native/external/src-dpdk/drivers/common/mlx5/linux/mlx5_glue.c
>
> ../build-root/build-vpp-native/external/src-dpdk/drivers/common/mlx5/linux/mlx5_glue.h
>
> ../build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/mlx4_glue.c
>
> ../build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/mlx4_glue.h
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/librte_pmd_mlx5_glue.so
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/b7c1ada@
> @rte_pmd_mlx5_glue@sha
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/b7c1ada@
> @rte_pmd_mlx5_glue@sha/mlx5_glue.c.o
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/librte_pmd_mlx5_glue.so.20.02.0
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/librte_pmd_mlx4_glue.so
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/8672f8e@
> @rte_pmd_mlx4_glue@sha
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/8672f8e@
> @rte_pmd_mlx4_glue@sha/mlx4_glue.c.o
>
> ../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/librte_pmd_mlx4_glue.so.18.02.0
> ../build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue
> (default) [17:50:24 ...]
>
> So it looks like the glue is being built but not installed?
>
> Thanks,
> Chris.
>
>
Yeah, it seems so. Either build configurations need to be updated to
install the glue libraries or you might be able to change
ibverbs_link=dlopen to ibverbs_link=static.

-Matt




>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17636): https://lists.fd.io/g/vpp-dev/message/17636
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Damjan Marion via lists.fd.io 


> On 02.10.2020., at 19:14, Christian Hopps  wrote:
> 
> 
> 
>> On Oct 2, 2020, at 1:07 PM, Damjan Marion via lists.fd.io 
>>  > > wrote:
>> 
>> 
>> May be a good time to consider moving to rdma/native implementation.
>> What are the gaps for you?
> 
> We have some conditional chained decap code based on the underlying dpdk 
> mbufs to do buffer indirection -- it's not overly pretty but it works :) 
> Also, although we haven't tested with it yet, we want to use async crypto 
> offload, and once we pay the mbuf penalty for that ... *shrug*

You are paying additional mbuf penalty if you also use PMD.
Also we have native async crypto working now in VPP with QAT support.

> 
> When using copy (rather than chain) and inline crypto, native drivers are an 
> option.
> 
> Thanks,
> Chris.
> 
>> 
>> 
>> — 
>> Damjan
>> 
>>> On 02.10.2020., at 19:01, Christian Hopps >> > wrote:
>>> 
>>> FWIW, I got this building and running; however, since the bump to 20.08 
>>> VPP+DPDK no longer finds the mlx5 interfaces.
>>> 
>>> 2020/10/02 16:58:53:176 warn   dpdk   EAL init args: -c 3f -n 4 
>>> --in-memory --file-prefix vpp -w :21:00.1 -w :21:00.0 
>>> --master-lcore 0
>>> 2020/10/02 16:58:53:508 notice dpdk   DPDK drivers found no 
>>> Ethernet devices...
>>> 
>>> Those devices are found when dpdk 20.05 is used instead.
>>> 
 On Oct 2, 2020, at 12:21 PM, Matthew Smith via lists.fd.io 
  >>> > wrote:
 
 Hi Mohammed,
 
 I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I 
 cannot say for sure. The scripts/configurations I build with always set 
 that flag, so I have not tried to generate a build without it set recently.
 
 -Matt
 
 
 On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI >>> > wrote:
 Hello Chris, Matthew,
 
 Thanks for raising that issue. Just to be clear and better understand, 
 does the problem occur with the default config, i.e., without trying to 
 compile any MLX driver in DPDK? Or does it only appear when setting 
 DPDK_MLX5_PMD=y ?
 Thanks
 Best regards
 Mohammed
> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io 
>   > wrote:
> 
> Hi Chris,
> 
> I did this in my local build:
> 
> diff --git a/build/external/packages/dpdk.mk  
> b/build/external/packages/dpdk.mk 
> index 49761cd56..a30ffd2ac 100644
> --- a/build/external/packages/dpdk.mk 
> +++ b/build/external/packages/dpdk.mk 
> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
> -Dtests=false \
> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
> +   -Dibverbs_link=dlopen \
> -Db_pie=true \
> -Dmachine=$(DPDK_MACHINE) \
> --buildtype=$(DPDK_BUILD_TYPE) 
> 
> If I try to submit it upstream, I would probably do something nicer using 
> a DPDK_ environment variable, but for the moment this got me 
> past that error. I have not actually tested with an mlx5 device yet, so I 
> don't know if something additional will be required in order to forward 
> packets via an mlx5 NIC, but it did fix the error you pasted and allow 
> dpdk_plugin.so to be loaded.
> 
> -Matt
> 
> 
> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps  > wrote:
> I've rebased my local branch on the latest master and dpdk is failing to 
> load now b/c
> 
> 2020/10/01 18:31:54:514 errplugin/load
> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>  undefined symbol: ibv_fork_init
> 
> I noticed that the dpdk build system has been changed is there something 
> I need to do to get it to link properly with the required libraries now?
> 
> I did change "n" in
> 
>   DPDK_MLX5_PMD?= n
>   DPDK_MLX5_COMMON_PMD ?= n
> 
> to "y" to try and get MLX5 PMD to build.
> 
> I also added
> 
>   vpp_uses_dpdk_mlx5_pmd = yes
> 
> to build-data/platforms/vpp.mk 
> 
> I also tried adding:
> 
>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
> 
> which didn't fix the problem.
> 
> Any suggestions on how to fix this?
> 
> Thanks,
> Chris.
> 
> 
> 
> 
 
 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17635): https://lists.fd.io/g/vpp-dev/message/17635
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner:

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Christian Hopps 
I just noticed this error I'm getting

net_mlx4: cannot load glue library: 
/home/chopps/w/vpp/build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue/librte_pmd_mlx4_glue.so.18.02.0:
 cannot open shared object file: No such file or directory
net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core 
libraries (libibverbs, libmlx4)
common_mlx5: Cannot load glue library: 
/home/chopps/w/vpp/build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue/librte_pmd_mlx5_glue.so.20.02.0:
 cannot open shared object file: No such file or directory
common_mlx5: Cannot initialize MLX5 common due to missing run-time dependency 
on rdma-core libraries (libibverbs, libmlx5)

And:

(default) [17:50:17 ...]$ find ../build-root/*-vpp-native/external -name 
'*glue*'
../build-root/build-vpp-native/external/src-dpdk/drivers/common/mlx5/linux/mlx5_glue.c
../build-root/build-vpp-native/external/src-dpdk/drivers/common/mlx5/linux/mlx5_glue.h
../build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/mlx4_glue.c
../build-root/build-vpp-native/external/src-dpdk/drivers/net/mlx4/mlx4_glue.h
../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/librte_pmd_mlx5_glue.so
../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/b7c1ada@@rte_pmd_mlx5_glue@sha
../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/b7c1ada@@rte_pmd_mlx5_glue@sha/mlx5_glue.c.o
../build-root/build-vpp-native/external/build-dpdk/drivers/common/mlx5/linux/librte_pmd_mlx5_glue.so.20.02.0
../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/librte_pmd_mlx4_glue.so
../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/8672f8e@@rte_pmd_mlx4_glue@sha
../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/8672f8e@@rte_pmd_mlx4_glue@sha/mlx4_glue.c.o
../build-root/build-vpp-native/external/build-dpdk/drivers/net/mlx4/librte_pmd_mlx4_glue.so.18.02.0
../build-root/install-vpp-native/external/lib/dpdk/pmds-20.0.3-glue
(default) [17:50:24 ...]

So it looks like the glue is being built but not installed?

Thanks,
Chris.

> On Oct 2, 2020, at 12:21 PM, Matthew Smith  wrote:
> 
> Hi Mohammed,
> 
> I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I cannot 
> say for sure. The scripts/configurations I build with always set that flag, 
> so I have not tried to generate a build without it set recently.
> 
> -Matt
> 
> 
> On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI  > wrote:
> Hello Chris, Matthew,
> 
> Thanks for raising that issue. Just to be clear and better understand, does 
> the problem occur with the default config, i.e., without trying to compile 
> any MLX driver in DPDK? Or does it only appear when setting DPDK_MLX5_PMD=y ?
> Thanks
> Best regards
> Mohammed
>> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io  
>> mailto:mgsmith=netgate@lists.fd.io>> 
>> wrote:
>> 
>> Hi Chris,
>> 
>> I did this in my local build:
>> 
>> diff --git a/build/external/packages/dpdk.mk  
>> b/build/external/packages/dpdk.mk 
>> index 49761cd56..a30ffd2ac 100644
>> --- a/build/external/packages/dpdk.mk 
>> +++ b/build/external/packages/dpdk.mk 
>> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
>> -Dtests=false \
>> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
>> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
>> +   -Dibverbs_link=dlopen \
>> -Db_pie=true \
>> -Dmachine=$(DPDK_MACHINE) \
>> --buildtype=$(DPDK_BUILD_TYPE)
>> 
>> If I try to submit it upstream, I would probably do something nicer using a 
>> DPDK_ environment variable, but for the moment this got me past 
>> that error. I have not actually tested with an mlx5 device yet, so I don't 
>> know if something additional will be required in order to forward packets 
>> via an mlx5 NIC, but it did fix the error you pasted and allow 
>> dpdk_plugin.so to be loaded.
>> 
>> -Matt
>> 
>> 
>> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps > > wrote:
>> I've rebased my local branch on the latest master and dpdk is failing to 
>> load now b/c
>> 
>> 2020/10/01 18:31:54:514 errplugin/load
>> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>>  undefined symbol: ibv_fork_init
>> 
>> I noticed that the dpdk build system has been changed is there something I 
>> need to do to get it to link properly with the required libraries now?
>> 
>> I did change "n" in
>> 
>>   DPDK_MLX5_PMD?= n
>>   DPDK_MLX5_COMMON_PMD ?= n
>> 
>> to "y" to try and get MLX5 PMD to build.
>> 
>> I also added
>> 
>>   vpp_uses_dpdk_mlx5_pmd = yes
>> 
>> to build-data/platforms/vpp.mk 
>> 
>> I also tried adding:
>> 
>>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
>> 
>> which didn't fix the problem.
>> 
>> Any suggestions on

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Christian Hopps 


> On Oct 2, 2020, at 1:07 PM, Damjan Marion via lists.fd.io 
>  wrote:
> 
> 
> May be a good time to consider moving to rdma/native implementation.
> What are the gaps for you?

We have some conditional chained decap code based on the underlying dpdk mbufs 
to do buffer indirection -- it's not overly pretty but it works :) Also, 
although we haven't tested with it yet, we want to use async crypto offload, 
and once we pay the mbuf penalty for that ... *shrug*

When using copy (rather than chain) and inline crypto, native drivers are an 
option.

Thanks,
Chris.

> 
> 
> —
> Damjan
> 
>> On 02.10.2020., at 19:01, Christian Hopps > > wrote:
>> 
>> FWIW, I got this building and running; however, since the bump to 20.08 
>> VPP+DPDK no longer finds the mlx5 interfaces.
>> 
>> 2020/10/02 16:58:53:176 warn   dpdk   EAL init args: -c 3f -n 4 
>> --in-memory --file-prefix vpp -w :21:00.1 -w :21:00.0 --master-lcore >> 0
>> 2020/10/02 16:58:53:508 notice dpdk   DPDK drivers found no 
>> Ethernet devices...
>> 
>> Those devices are found when dpdk 20.05 is used instead.
>> 
>>> On Oct 2, 2020, at 12:21 PM, Matthew Smith via lists.fd.io 
>>>  >> > wrote:
>>> 
>>> Hi Mohammed,
>>> 
>>> I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I 
>>> cannot say for sure. The scripts/configurations I build with always set 
>>> that flag, so I have not tried to generate a build without it set recently.
>>> 
>>> -Matt
>>> 
>>> 
>>> On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI >> > wrote:
>>> Hello Chris, Matthew,
>>> 
>>> Thanks for raising that issue. Just to be clear and better understand, does 
>>> the problem occur with the default config, i.e., without trying to compile 
>>> any MLX driver in DPDK? Or does it only appear when setting DPDK_MLX5_PMD=y 
>>> ?
>>> Thanks
>>> Best regards
>>> Mohammed
 On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io 
  >>> > wrote:
 
 Hi Chris,
 
 I did this in my local build:
 
 diff --git a/build/external/packages/dpdk.mk  
 b/build/external/packages/dpdk.mk 
 index 49761cd56..a30ffd2ac 100644
 --- a/build/external/packages/dpdk.mk 
 +++ b/build/external/packages/dpdk.mk 
 @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
 -Dtests=false \
 "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
 "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
 +   -Dibverbs_link=dlopen \
 -Db_pie=true \
 -Dmachine=$(DPDK_MACHINE) \
 --buildtype=$(DPDK_BUILD_TYPE)
 
 If I try to submit it upstream, I would probably do something nicer using 
 a DPDK_ environment variable, but for the moment this got me 
 past that error. I have not actually tested with an mlx5 device yet, so I 
 don't know if something additional will be required in order to forward 
 packets via an mlx5 NIC, but it did fix the error you pasted and allow 
 dpdk_plugin.so to be loaded.
 
 -Matt
 
 
 On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps >>> > wrote:
 I've rebased my local branch on the latest master and dpdk is failing to 
 load now b/c
 
 2020/10/01 18:31:54:514 errplugin/load
 /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
  undefined symbol: ibv_fork_init
 
 I noticed that the dpdk build system has been changed is there something I 
 need to do to get it to link properly with the required libraries now?
 
 I did change "n" in
 
   DPDK_MLX5_PMD?= n
   DPDK_MLX5_COMMON_PMD ?= n
 
 to "y" to try and get MLX5 PMD to build.
 
 I also added
 
   vpp_uses_dpdk_mlx5_pmd = yes
 
 to build-data/platforms/vpp.mk 
 
 I also tried adding:
 
   vpp_uses_dpdk_ibverbs_link_dlopen = yes
 
 which didn't fix the problem.
 
 Any suggestions on how to fix this?
 
 Thanks,
 Chris.
 
 
 
 
>>> 
>>> 
>> 
>> 
>> 
>> 
> 
> 
> 



signature.asc
Description: Message signed with OpenPGP

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17633): https://lists.fd.io/g/vpp-dev/message/17633
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Damjan Marion via lists.fd.io 

May be a good time to consider moving to rdma/native implementation.
What are the gaps for you?


— 
Damjan

> On 02.10.2020., at 19:01, Christian Hopps  wrote:
> 
> FWIW, I got this building and running; however, since the bump to 20.08 
> VPP+DPDK no longer finds the mlx5 interfaces.
> 
> 2020/10/02 16:58:53:176 warn   dpdk   EAL init args: -c 3f -n 4 
> --in-memory --file-prefix vpp -w :21:00.1 -w :21:00.0 --master-lcore 0
> 2020/10/02 16:58:53:508 notice dpdk   DPDK drivers found no 
> Ethernet devices...
> 
> Those devices are found when dpdk 20.05 is used instead.
> 
>> On Oct 2, 2020, at 12:21 PM, Matthew Smith via lists.fd.io 
>>  > > wrote:
>> 
>> Hi Mohammed,
>> 
>> I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I 
>> cannot say for sure. The scripts/configurations I build with always set that 
>> flag, so I have not tried to generate a build without it set recently.
>> 
>> -Matt
>> 
>> 
>> On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI > > wrote:
>> Hello Chris, Matthew,
>> 
>> Thanks for raising that issue. Just to be clear and better understand, does 
>> the problem occur with the default config, i.e., without trying to compile 
>> any MLX driver in DPDK? Or does it only appear when setting DPDK_MLX5_PMD=y ?
>> Thanks
>> Best regards
>> Mohammed
>>> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io 
>>>  >> > wrote:
>>> 
>>> Hi Chris,
>>> 
>>> I did this in my local build:
>>> 
>>> diff --git a/build/external/packages/dpdk.mk  
>>> b/build/external/packages/dpdk.mk 
>>> index 49761cd56..a30ffd2ac 100644
>>> --- a/build/external/packages/dpdk.mk 
>>> +++ b/build/external/packages/dpdk.mk 
>>> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
>>> -Dtests=false \
>>> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
>>> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
>>> +   -Dibverbs_link=dlopen \
>>> -Db_pie=true \
>>> -Dmachine=$(DPDK_MACHINE) \
>>> --buildtype=$(DPDK_BUILD_TYPE) 
>>> 
>>> If I try to submit it upstream, I would probably do something nicer using a 
>>> DPDK_ environment variable, but for the moment this got me past 
>>> that error. I have not actually tested with an mlx5 device yet, so I don't 
>>> know if something additional will be required in order to forward packets 
>>> via an mlx5 NIC, but it did fix the error you pasted and allow 
>>> dpdk_plugin.so to be loaded.
>>> 
>>> -Matt
>>> 
>>> 
>>> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps >> > wrote:
>>> I've rebased my local branch on the latest master and dpdk is failing to 
>>> load now b/c
>>> 
>>> 2020/10/01 18:31:54:514 errplugin/load
>>> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>>>  undefined symbol: ibv_fork_init
>>> 
>>> I noticed that the dpdk build system has been changed is there something I 
>>> need to do to get it to link properly with the required libraries now?
>>> 
>>> I did change "n" in
>>> 
>>>   DPDK_MLX5_PMD?= n
>>>   DPDK_MLX5_COMMON_PMD ?= n
>>> 
>>> to "y" to try and get MLX5 PMD to build.
>>> 
>>> I also added
>>> 
>>>   vpp_uses_dpdk_mlx5_pmd = yes
>>> 
>>> to build-data/platforms/vpp.mk 
>>> 
>>> I also tried adding:
>>> 
>>>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
>>> 
>>> which didn't fix the problem.
>>> 
>>> Any suggestions on how to fix this?
>>> 
>>> Thanks,
>>> Chris.
>>> 
>>> 
>>> 
>>> 
>> 
>> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17632): https://lists.fd.io/g/vpp-dev/message/17632
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Christian Hopps 
FWIW, I got this building and running; however, since the bump to 20.08 
VPP+DPDK no longer finds the mlx5 interfaces.

2020/10/02 16:58:53:176 warn   dpdk   EAL init args: -c 3f -n 4 
--in-memory --file-prefix vpp -w :21:00.1 -w :21:00.0 --master-lcore 0
2020/10/02 16:58:53:508 notice dpdk   DPDK drivers found no 
Ethernet devices...

Those devices are found when dpdk 20.05 is used instead.

> On Oct 2, 2020, at 12:21 PM, Matthew Smith via lists.fd.io 
>  wrote:
> 
> Hi Mohammed,
> 
> I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I cannot 
> say for sure. The scripts/configurations I build with always set that flag, 
> so I have not tried to generate a build without it set recently.
> 
> -Matt
> 
> 
> On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI  > wrote:
> Hello Chris, Matthew,
> 
> Thanks for raising that issue. Just to be clear and better understand, does 
> the problem occur with the default config, i.e., without trying to compile 
> any MLX driver in DPDK? Or does it only appear when setting DPDK_MLX5_PMD=y ?
> Thanks
> Best regards
> Mohammed
>> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io  
>> mailto:mgsmith=netgate@lists.fd.io>> 
>> wrote:
>> 
>> Hi Chris,
>> 
>> I did this in my local build:
>> 
>> diff --git a/build/external/packages/dpdk.mk  
>> b/build/external/packages/dpdk.mk 
>> index 49761cd56..a30ffd2ac 100644
>> --- a/build/external/packages/dpdk.mk 
>> +++ b/build/external/packages/dpdk.mk 
>> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
>> -Dtests=false \
>> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
>> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
>> +   -Dibverbs_link=dlopen \
>> -Db_pie=true \
>> -Dmachine=$(DPDK_MACHINE) \
>> --buildtype=$(DPDK_BUILD_TYPE)
>> 
>> If I try to submit it upstream, I would probably do something nicer using a 
>> DPDK_ environment variable, but for the moment this got me past 
>> that error. I have not actually tested with an mlx5 device yet, so I don't 
>> know if something additional will be required in order to forward packets 
>> via an mlx5 NIC, but it did fix the error you pasted and allow 
>> dpdk_plugin.so to be loaded.
>> 
>> -Matt
>> 
>> 
>> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps > > wrote:
>> I've rebased my local branch on the latest master and dpdk is failing to 
>> load now b/c
>> 
>> 2020/10/01 18:31:54:514 errplugin/load
>> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>>  undefined symbol: ibv_fork_init
>> 
>> I noticed that the dpdk build system has been changed is there something I 
>> need to do to get it to link properly with the required libraries now?
>> 
>> I did change "n" in
>> 
>>   DPDK_MLX5_PMD?= n
>>   DPDK_MLX5_COMMON_PMD ?= n
>> 
>> to "y" to try and get MLX5 PMD to build.
>> 
>> I also added
>> 
>>   vpp_uses_dpdk_mlx5_pmd = yes
>> 
>> to build-data/platforms/vpp.mk 
>> 
>> I also tried adding:
>> 
>>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
>> 
>> which didn't fix the problem.
>> 
>> Any suggestions on how to fix this?
>> 
>> Thanks,
>> Chris.
>> 
>> 
>> 
>> 
> 
> 
> 



signature.asc
Description: Message signed with OpenPGP

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17631): https://lists.fd.io/g/vpp-dev/message/17631
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Matthew Smith via lists.fd.io 
Hi Mohammed,

I think it will only affect builds where DPDK_MLX5_PMD=y is set, but I
cannot say for sure. The scripts/configurations I build with always set
that flag, so I have not tried to generate a build without it set recently.

-Matt


On Fri, Oct 2, 2020 at 12:57 AM Mohammed HAWARI 
wrote:

> Hello Chris, Matthew,
>
> Thanks for raising that issue. Just to be clear and better understand,
> does the problem occur with the default config, i.e., without trying to
> compile any MLX driver in DPDK? Or does it only appear when setting 
> DPDK_MLX5_PMD=y
> ?
> Thanks
> Best regards
> Mohammed
>
> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io <
> mgsmith=netgate@lists.fd.io> wrote:
>
> Hi Chris,
>
> I did this in my local build:
>
> diff --git a/build/external/packages/dpdk.mk b/build/external/packages/
> dpdk.mk
> index 49761cd56..a30ffd2ac 100644
> --- a/build/external/packages/dpdk.mk
> +++ b/build/external/packages/dpdk.mk
> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
> -Dtests=false \
> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
> +   -Dibverbs_link=dlopen \
> -Db_pie=true \
> -Dmachine=$(DPDK_MACHINE) \
> --buildtype=$(DPDK_BUILD_TYPE)
>
> If I try to submit it upstream, I would probably do something nicer using
> a DPDK_ environment variable, but for the moment this got me
> past that error. I have not actually tested with an mlx5 device yet, so I
> don't know if something additional will be required in order to forward
> packets via an mlx5 NIC, but it did fix the error you pasted and
> allow dpdk_plugin.so to be loaded.
>
> -Matt
>
>
> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps  wrote:
>
>> I've rebased my local branch on the latest master and dpdk is failing to
>> load now b/c
>>
>> 2020/10/01 18:31:54:514 errplugin/load
>> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>> undefined symbol: ibv_fork_init
>>
>> I noticed that the dpdk build system has been changed is there something
>> I need to do to get it to link properly with the required libraries now?
>>
>> I did change "n" in
>>
>>   DPDK_MLX5_PMD?= n
>>   DPDK_MLX5_COMMON_PMD ?= n
>>
>> to "y" to try and get MLX5 PMD to build.
>>
>> I also added
>>
>>   vpp_uses_dpdk_mlx5_pmd = yes
>>
>> to build-data/platforms/vpp.mk
>>
>> I also tried adding:
>>
>>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
>>
>> which didn't fix the problem.
>>
>> Any suggestions on how to fix this?
>>
>> Thanks,
>> Chris.
>>
>>
>>
>>
> 
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17629): https://lists.fd.io/g/vpp-dev/message/17629
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Vpp Load balancing plugin strange behaviour

2020-10-02 Thread Dimitrios Markou 
Hello all,

I would like to ask you about the strange behavior that I am seeing when I am 
using the vpp load-balancing plugin.

Below you can see the configuration that I am using inside the vpp:

vpp# set int ip addr HundredGigabitEthernet84/0/4 172.16.1.2/30
vpp# set int ip addr HundredGigabitEthernet84/9/1 192.168.1.215/24
vpp# set int state HundredGigabitEthernet84/9/1 up
vpp# set int state HundredGigabitEthernet84/0/4 up

lb vip 20.20.20.20/32 protocol udp port 1000 encap nat4 type nodeport 
target_port 1000
lb as 20.20.20.20/32 protocol udp port 1000 172.16.1.1
lb set interface nat4 in HundredGigabitEthernet84/0/4
lb conf ip4-src-address 172.16.1.2

vpp# show lb vip verbose
 ip4-nat4 [1] 20.20.20.20/32
  new_size:1024
  protocol:17 port:1000
  type:nodeport port:59395 target_port:1000  counters:
packet from existing sessions: 0
first session packet: 0
untracked packet: 0
no server configured: 0
  #as:1
172.16.1.1 1024 buckets   0 flows  dpo:18 used


I am starting executing udp traffic towards the VIP address of the Load 
balancer with the below netcat command:

nc -u 20.20.20.20 1000

The traffic reaches the receiver as you can see from the TCPdump output below 
but it doesn’t get SNATed as it should be and as I configured it in vpp.

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on nsm0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:49:35.620264 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 8
14:49:36.975004 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 9
14:49:37.715221 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 7
14:49:39.010709 IP 192.168.1.106.22445 > 172.16.1.1.1000: UDP, length 7

The Source Ip in the above tcpdump output should be 172.16.1.2 and not 
192.168.1.106. Do you have any idea why this is happening ? Am I configuring 
the vpp lb plugin incorrectly ? Attached you can find the vpp trace that is 
captured when the traffic was flowing.

Thank you in advance,
Dimitris



{\rtf1\ansi\ansicpg1252\cocoartf2511
\cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\paperw11900\paperh16840\margl1440\margr1440\vieww35640\viewh22400\viewkind0
\deftab720
\pard\pardeftab720\partightenfactor0

\f0\fs26 \cf0 \expnd0\expndtw0\kerning0
--- Start of thread 0 vpp_main ---\
No packets in trace buffer\
--- Start of thread 1 vpp_wk_0 ---\
Packet 1\
\'a0\
02:11:08:763316: dpdk-input\
\'a0 HundredGigabitEthernet84/0/4 rx queue 0\
\'a0 buffer 0xfed5b8: current data 0, length 80, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x100\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed l4-cksum-correct\
\'a0\'a0PKT MBUF: port 0, nb_segs 1, pkt_len 80\
\'a0\'a0\'a0 buf_len 2176, data_len 80, ol_flags 0x80, data_off 128, phys_addr 0x3fb56e80\
\'a0\'a0\'a0 packet_type 0x691 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0\
\'a0\'a0\'a0 rss 0x0 fdir.hi 0x0 fdir.lo 0x0\
\'a0\'a0\'a0 Packet Offload Flags\
\'a0\'a0\'a0\'a0\'a0 PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid\
\'a0\'a0\'a0 Packet Types\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers\
\'a0\'a0\'a0\'a0\'a0 RTE_PTYPE_L4_NONFRAG (0x0600) Non-fragmented IP packet\
\'a0 IP4: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
\'a0 ICMP: 172.16.1.1 -> 192.168.1.106\
\'a0\'a0\'a0 tos 0xc0, ttl 64, length 66, checksum 0x4bd6 dscp CS6 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0xbf01\
\'a0 ICMP destination_unreachable port_unreachable checksum 0x6c44\
02:11:08:763329: ethernet-input\
\'a0frame: flags 0x3, hw-if-index 1, sw-if-index 1\
\'a0 IP4: fa:6e:3d:37:9b:3f -> 6a:91:5f:b0:da:4c\
02:11:08:763341: ip4-input-no-checksum\
\'a0 ICMP: 172.16.1.1 -> 192.168.1.106\
\'a0\'a0\'a0 tos 0xc0, ttl 64, length 66, checksum 0x4bd6 dscp CS6 ecn NON_ECN\
\'a0\'a0\'a0 fragment id 0xbf01\
\'a0 ICMP destination_unreachable port_unreachable checksum 0x6c44\
02:11:08:763348: lb-nat4-in2out\
\'a0 lb vip[0]: ip6-gre6 :::::::/0 new_size:1 #as:0\
lb as[0]: ::::::: removed\
lb nat: rx_sw_if_index = 1, next_index = 0\
02:11:08:763357: error-drop\
\'a0 rx:HundredGigabitEthernet84/0/4\
02:11:08:763359: drop\
\'a0 dpdk-input: no error\
\'a0\
Packet 2\
\'a0\
02:11:13:403106: dpdk-input\
\'a0 HundredGigabitEthernet84/0/4 rx queue 0\
\'a0 buffer 0xfed480: current data 0, length 60, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x101\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0 ext-hdr-valid\
\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0\'a0l4-cksum-computed

Re: [vpp-dev] Latest master, dpdk and mlx5 failing

2020-10-02 Thread Mohammed HAWARI 
Hello Chris, Matthew,

Thanks for raising that issue. Just to be clear and better understand, does the 
problem occur with the default config, i.e., without trying to compile any MLX 
driver in DPDK? Or does it only appear when setting DPDK_MLX5_PMD=y ?
Thanks
Best regards
Mohammed
> On 1 Oct 2020, at 22:33, Matthew Smith via lists.fd.io 
>  wrote:
> 
> Hi Chris,
> 
> I did this in my local build:
> 
> diff --git a/build/external/packages/dpdk.mk  
> b/build/external/packages/dpdk.mk 
> index 49761cd56..a30ffd2ac 100644
> --- a/build/external/packages/dpdk.mk 
> +++ b/build/external/packages/dpdk.mk 
> @@ -139,6 +139,7 @@ DPDK_MESON_ARGS = \
> -Dtests=false \
> "-Ddisable_drivers=$(DPDK_DRIVERS_DISABLED)" \
> "-Ddisable_libs=$(DPDK_LIBS_DISABLED)" \
> +   -Dibverbs_link=dlopen \
> -Db_pie=true \
> -Dmachine=$(DPDK_MACHINE) \
> --buildtype=$(DPDK_BUILD_TYPE) 
> 
> If I try to submit it upstream, I would probably do something nicer using a 
> DPDK_ environment variable, but for the moment this got me past 
> that error. I have not actually tested with an mlx5 device yet, so I don't 
> know if something additional will be required in order to forward packets via 
> an mlx5 NIC, but it did fix the error you pasted and allow dpdk_plugin.so to 
> be loaded.
> 
> -Matt
> 
> 
> On Thu, Oct 1, 2020 at 2:10 PM Christian Hopps  > wrote:
> I've rebased my local branch on the latest master and dpdk is failing to load 
> now b/c
> 
> 2020/10/01 18:31:54:514 errplugin/load
> /home/chopps/w/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins/dpdk_plugin.so:
>  undefined symbol: ibv_fork_init
> 
> I noticed that the dpdk build system has been changed is there something I 
> need to do to get it to link properly with the required libraries now?
> 
> I did change "n" in
> 
>   DPDK_MLX5_PMD?= n
>   DPDK_MLX5_COMMON_PMD ?= n
> 
> to "y" to try and get MLX5 PMD to build.
> 
> I also added
> 
>   vpp_uses_dpdk_mlx5_pmd = yes
> 
> to build-data/platforms/vpp.mk 
> 
> I also tried adding:
> 
>   vpp_uses_dpdk_ibverbs_link_dlopen = yes
> 
> which didn't fix the problem.
> 
> Any suggestions on how to fix this?
> 
> Thanks,
> Chris.
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17628): https://lists.fd.io/g/vpp-dev/message/17628
Mute This Topic: https://lists.fd.io/mt/77247865/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-