Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

> 
> That's a broad question... Which kind of info are you looking for?
> 

I think you kind answered one the gaps I have filled referencing linux 
macvlans, and how the packets are forwarded. However when I run `ip -br a` I 
dont see another interface created for the RDMA (maybe I shouldn't be seeing 
one?) Is there a specific interface for RDMA created in /dev somewhere that vpp 
takes control of? Looking mostly to how the DPDK + RDMA functionality works in 
the linux kernel, and how packets get from my PF interface all the way to VPP.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21676): https://lists.fd.io/g/vpp-dev/message/21676
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Benoit Ganne (bganne) via lists.fd.io]

> Thanks for the info. I am not too familiar with rdma interfaces, well at
> least the way vpp is utilizing them. It seems its polling the interfaces
> just like dpdk.

Polling is the default for both dpdk and rdma interfaces. You can set them to 
interrupt mode or adaptive mode (automatically switch between interrupt and 
polling based on the load) mode with 'set interface rx-mode  [queue 
] [polling | interrupt | adaptive]'

> I am also able to see traffic flowing through vpp, however
> its seems I can still interact with the ports via the kernel. This means
> that the interfaces are still being utilized by the kernel yes?

Yes, both dpdk and rdma drivers use a bifurcated model like Linux macvlan: each 
new interface gets a specific mac address and the NIC HW is programmed to steer 
packets based on the dest mac. So Linux netdev gets a mac address and VPP 
interfaces other mac, and you can use them all in parallel.

> However my test traffic is flowing much more efficiently through the
> system than using just the interfaces with a basic linux kernel set up. I
> can only assume dpdk or rdma or a combination of both are being used
> properly. Matt was a great help and shared  this link  docs.fd.io/vpp/22.10/developer/devicedrivers/rdma.html?highlight=rdma>
> to documentation from vpp on rdma, however I would to read more about how
> the RDMA + DPDK system works. Do you have any other literature or
> documentation I can read?

That's a broad question... Which kind of info are you looking for?

> Also, unrelated question, it appears with startup.conf I can set the
> number of rx/tx ques and size, what is the default values for this? Does
> it use whatever ethtool -l and ethtool -g uses? Or do I need to set to
> maximums for those values to apply?

ethtool parameters are not used for VPP.
For queue size, the defaults should be sane but can be tuned if needs be - 
basically you want it to be big enough to hide your processing latency but not 
bigger.
Number of tx queues defaults to the number of VPP threads by default which is 
the right thing to do.
Number of rx queues defaults to 1 which means only 1 VPP worker thread will 
process packets coming from this interface. Depending on your setup that might 
not be what you want. For example, if you have 4 VPP worker threads and only 1 
interface, you'll probably want the 4 worker threads to process packets. In 
that case you need to create 4 rx queues (1 for each worker).

ben

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21674): https://lists.fd.io/g/vpp-dev/message/21674
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

@Rupesh,

Thanks for the info. I am not too familiar with rdma interfaces, well at least 
the way vpp is utilizing them. It seems its polling the interfaces just like 
dpdk. I am also able to see traffic flowing through vpp, however its seems I 
can still interact with the ports via the kernel. This means that the 
interfaces are still being utilized by the kernel yes?

However my test traffic is flowing much more efficiently through the system 
than using just the interfaces with a basic linux kernel set up. I can only 
assume dpdk or rdma or a combination of both are being used properly. Matt was 
a great help and shared this link ( 
https://s3-docs.fd.io/vpp/22.10/developer/devicedrivers/rdma.html?highlight=rdma
 ) to documentation from vpp on rdma, however I would to read more about how 
the RDMA + DPDK system works. Do you have any other literature or documentation 
I can read?

Also, unrelated question, it appears with startup.conf I can set the number of 
rx/tx ques and size, what is the default values for this? Does it use whatever 
ethtool -l and ethtool -g uses? Or do I need to set to maximums for those 
values to apply?

I think its all working now, thanks again to all your help and quick responses 
:)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21638): https://lists.fd.io/g/vpp-dev/message/21638
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Rupesh Raghuvaran]

The device will still be available in linux, it is managed via  ib_uverbs
by dpdk.  Thats why it is required to have ib_uverbs, mlx5_core, mlx5_ib
modules required .

-Rupesh

On Fri, Jul 8, 2022 at 9:37 PM Dave Houser  wrote:

> OK I am not sure what combination got everything to work but the
> interfaces are showing up now in vppctl!
>
> also rdma commands work as well.
>
> I think it was a combination of the following:
>
>1. git clone the repo
>2. edit build/external/packages/dpdk.mk,
>   1. turn the following to "y"
>   (DPDK_MLX4_PMD, DPDK_MLX5_PMD, DPDK_MLX5_COMMON_PMD)
>   2. Set DPDK_MLX_IBV_LINK to static
>3. make make wipe-release;make build-release
>4. make pkg-deb
>5. sudo dpkg -i build-root/*.deb
>6. Create conf file without comments (optinal)
>   1. sudo cp /etc/vpp/startup.conf /etc/vpp/startup.conf.original;
>   cat /etc/vpp/startup.conf | grep -v "^#.*" | grep -v ".*#.*" | awk 'NF' 
> >
>   /etc/vpp/startup.conf
>   2. edit the file and add your dpdk interfaces make sure to add the
>   dpdk section
>7. restart vpp
>
> I suppose the only question I have is how is it possible the dpdk
> interfaces are showing up in vpp?
>
> I ask this as the interfaces I configured in startup.conf are still linked
> to the kernel and up. However they still show up in vpp as interfaces. If I
> deactivate the interfaces, unbind from the kernel, and bind to dpdk, then
> restart vpp, then they dont show up. Is this expected?
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21637): https://lists.fd.io/g/vpp-dev/message/21637
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

OK I am not sure what combination got everything to work but the interfaces are 
showing up now in vppctl!

also rdma commands work as well.

I think it was a combination of the following:

* git clone the repo
* edit build/external/packages/dpdk.mk, * turn the following to "y" 
(DPDK_MLX4_PMD, DPDK_MLX5_PMD, DPDK_MLX5_COMMON_PMD)
* Set DPDK_MLX_IBV_LINK to static

* make make wipe-release;make build-release
* make pkg-deb
* sudo dpkg -i build-root/*.deb
* Create conf file without comments (optinal) * sudo cp /etc/vpp/startup.conf 
/etc/vpp/startup.conf.original; cat /etc/vpp/startup.conf | grep -v "^#.*" | 
grep -v ".*#.*" | awk 'NF' > /etc/vpp/startup.conf
* edit the file and add your dpdk interfaces make sure to add the dpdk section

* restart vpp

I suppose the only question I have is how is it possible the dpdk interfaces 
are showing up in vpp?

I ask this as the interfaces I configured in startup.conf are still linked to 
the kernel and up. However they still show up in vpp as interfaces. If I 
deactivate the interfaces, unbind from the kernel, and bind to dpdk, then 
restart vpp, then they dont show up. Is this expected?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21636): https://lists.fd.io/g/vpp-dev/message/21636
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

I was able to `mke pkg-dep` and `sudo dpkg -i build-root/*.deb`, however after 
install when running vpp service and checking logs I see the following:

```

$ sudo vpp show logs

mlx5_common: Cannot load glue library: 
/home/ubuntu/vpp/vpp/build-root/install-vpp-native/external/lib/dpdk/pmds-22.1-glue/librte_common_mlx5_glue.so.22.1:
 cannot open shared object file: No such file or directory

mlx5_common: Cannot initialize MLX5 common due to missing run-time dependency 
on rdma-core libraries (libibverbs, libmlx5)

vlib_call_all_config_functions: unknown input `show logs'

```
Its not clear to me from these instructions ( 
https://wiki.fd.io/view/VPP/Build,_install,_and_test_images#Build_A_VPP_Package 
) if I need to do anything after `make build`. Here is the output after make 
build completes

```
-- Configuring done
-- Generating done
-- Build files have been written to: 
/home/ubuntu/vpp/vpp/build-root/build-vpp_debug-native/vpp
 Building vpp in /home/ubuntu/vpp/vpp/build-root/build-vpp_debug-native/vpp 

[2498/2498] Linking C shared library 
lib/x86_64-linux-gnu/vpp_plugins/unittest_plugin.so
 Installing vpp 
[0/1] Install the project...
-- Install configuration: "debug"
make[1]: Leaving directory '/home/ubuntu/vpp/vpp/build-root'
```
Did it install? Do I need to create a deb package instead? Will my dpdk.mk 
changes be used in the `make pkg-deb`

I read this post ( https://lists.fd.io/g/vpp-dev/topic/77247865#17634 ) seems 
other have had the same issue. @Matt you stated

> 
> "Yeah, it seems so. Either build configurations need to be updated to
> install the glue libraries or you might be able to change
> ibverbs_link=dlopen to ibverbs_link=static."
> 

I adjusted build/external/packages/dpdk.mk, DPDK_MLX_IBV_LINK ?=static, then 
ran make pkg-deb, and installed but I get the same error. It seems DPDK is not 
linking, is there any fix for this? I am fine with using RDMA, however my 
previous post shows the commands just dont work.

Any recommendations on next steps?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21635): https://lists.fd.io/g/vpp-dev/message/21635
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Matthew Smith via lists.fd.io]

The command you typed looks like it should be valid, at least with the
current code. I'm not sure what the correct syntax was in 21.10.

If you type 'create int rdma ?' and press enter it should print the help
text which describes what the valid options are.

Is the rdma plugin loaded? If you run 'show plugins' in vppctl, does it
list rdma_plugin.so?

-Matt


On Thu, Jul 7, 2022 at 3:19 PM Dave Houser  wrote:

> On Thu, Jul 7, 2022 at 01:53 PM, Matthew Smith wrote:
>
>
> https://s3-docs.fd.io/vpp/22.10/developer/devicedrivers/rdma.html?highlight=rdma
>
> Thanks Matt,
>
> I unbound the interfaces from dpdk and gave back to the kernel, then
> restart vpp.
> Working through the instructions, however they don't seem to work, or
> maybe not clear to me.
>
> create int rdma host-if enp94s0f0 name rdma-0
>
>
> This returns
> `create interface: unknown input `rdma host-if enp65s0f6 name rd...'`
>
> I wish there was a way to check commands I can use in vppctl (Like
> pressing `?` or tab-tab to see commands I can use based on the semantics I
> am using, but I don't see way to do this)
>
> Dont think I ever shared the version I am running so I am now:
> ```
> vpp# show vers
> vpp v21.10.1-release built by root on 4f6ead0c141f at 2021-11-17T14:25:30
> ```
>
> Am I using the command wrong?
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21634): https://lists.fd.io/g/vpp-dev/message/21634
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

Also, I tried performing `make pkg-deb` but get the following error while its 
building

[41/708] Linking target drivers/librte_common_mlx5.so.22.1

*FAILED: drivers/librte_common_mlx5.so.22.1*

cc  -o drivers/librte_common_mlx5.so.22.1 
drivers/librte_common_mlx5.so.22.1.p/meson-generated_.._rte_common_mlx5.pmd.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_devx_cmds.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common_mp.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common_mr.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_malloc.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common_pci.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common_devx.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_mlx5_common_utils.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_linux_mlx5_nl.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_linux_mlx5_common_auxiliary.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_linux_mlx5_common_os.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_linux_mlx5_common_verbs.c.o 
drivers/libtmp_rte_common_mlx5.a.p/common_mlx5_linux_mlx5_glue.c.o 
-Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group 
-Wl,-soname,librte_common_mlx5.so.22 -Wl,--no-as-needed -pthread -lm -ldl 
lib/librte_hash.so.22.1 lib/librte_eal.so.22.1 lib/librte_kvargs.so.22.1 
lib/librte_telemetry.so.22.1 lib/librte_net.so.22.1 lib/librte_mbuf.so.22.1 
lib/librte_mempool.so.22.1 lib/librte_ring.so.22.1 lib/librte_rcu.so.22.1 
lib/librte_pci.so.22.1 drivers/librte_bus_pci.so.22.1 
drivers/librte_bus_auxiliary.so.22.1 
-Wl,--version-script=/home/ubuntu/vpp/vpp/build-root/build-vpp-native/external/src-dpdk/drivers/common/mlx5/version.map
 -lpthread -l:libmlx5.a -l:libibverbs.a -lnl-route-3 -lnl-3 -Wl,--end-group 
'-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' 
-Wl,-rpath-link,/home/ubuntu/vpp/vpp/build-root/build-vpp-native/external/build-dpdk/lib
 
-Wl,-rpath-link,/home/ubuntu/vpp/vpp/build-root/build-vpp-native/external/build-dpdk/drivers

/usr/bin/ld: 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libmlx5.a(mlx5.c.o): 
relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used 
when making a shared object; recompile with -fPIC

/usr/bin/ld: final link failed: Bad value

collect2: error: ld returned 1 exit status

[138/708] Compiling C object 
drivers/libtmp_rte_net_mlx5.a.p/net_mlx5_mlx5_tx_empw.c.o

ninja: build stopped: subcommand failed.

Could not rebuild .

packages/dpdk.mk:208: recipe for target 
'/home/ubuntu/vpp/vpp/build-root/build-vpp-native/external/.dpdk.install.ok' 
failed

make[3]: *** 
[/home/ubuntu/vpp/vpp/build-root/build-vpp-native/external/.dpdk.install.ok] 
Error 255

make[3]: Leaving directory '/home/ubuntu/vpp/vpp/build/external'

Makefile:171: recipe for target 'ebuild-install' failed

make[2]: *** [ebuild-install] Error 2

make[2]: Leaving directory '/home/ubuntu/vpp/vpp/build/external'

Makefile:732: recipe for target 'external-install' failed

make[1]: *** [external-install] Error 2

make[1]: Leaving directory '/home/ubuntu/vpp/vpp/build-root'

Makefile:604: recipe for target 'pkg-deb' failed

make: *** [pkg-deb] Error 2

```

I see the linked library is on my host

```

Doing a search in the repo I found the following directories but no file
```
$ find . | grep librte_common_mlx5.so.22
./build-root/build-vpp-native/external/build-dpdk/drivers/librte_common_mlx5.so.22.1.p
./build-root/build-vpp-native/external/build-dpdk/drivers/librte_common_mlx5.so.22.1.p/meson-generated_.._rte_common_mlx5.pmd.c.o
./build-root/build-vpp-native/external/build-dpdk/drivers/librte_common_mlx5.so.22
```

I am not sure how else to fix this, any recommendations?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21627): https://lists.fd.io/g/vpp-dev/message/21627
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

On Thu, Jul 7, 2022 at 01:53 PM, Matthew Smith wrote:

> 
> https://s3-docs.fd.io/vpp/22.10/developer/devicedrivers/rdma.html?highlight=rdma
> 

Thanks Matt,

I unbound the interfaces from dpdk and gave back to the kernel, then restart 
vpp.
Working through the instructions, however they don't seem to work, or maybe not 
clear to me.

create int rdma host-if enp94s0f0 name rdma-0
This returns
`create interface: unknown input `rdma host-if enp65s0f6 name rd...'`

I wish there was a way to check commands I can use in vppctl (Like pressing `?` 
or tab-tab to see commands I can use based on the semantics I am using, but I 
don't see way to do this)

Dont think I ever shared the version I am running so I am now:
```
vpp# show vers
vpp v21.10.1-release built by root on 4f6ead0c141f at 2021-11-17T14:25:30
```

Am I using the command wrong?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21626): https://lists.fd.io/g/vpp-dev/message/21626
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Matthew Smith via lists.fd.io]

Hi Dave,

The mlx5 driver code gets statically linked by the shared library in
vpp-plugin-dpdk, so that's the package that needs to be updated. I'm not
aware of a way to build just that one package, running 'make pkg-deb' will
rebuild all of the VPP deb packages. Once they're built, you might be able
to get by with only updating vpp-plugin-dpdk, but I recommend you update
vpp, libvppinfra, vpp-plugin-core and vpp-plugin-dpdk just to avoid
headaches.

I personally have not tried using the rdma plugin. I have heard other VPP
devs recommend using it though. It's supposed to eliminate some processing
overhead caused by differences between VPP's internal structure of a buffer
and DPDK's, so it's possible that using an rdma interface would be more
efficient than using a DPDK interface. And using the rdma plugin should not
require rebuilding VPP for you to try it. This is the only doc I know of on
using rdma interfaces -
https://s3-docs.fd.io/vpp/22.10/developer/devicedrivers/rdma.html?highlight=rdma
.

-Matt


On Thu, Jul 7, 2022 at 12:31 PM Dave Houser  wrote:

> Hi Matt,
>
> These are Mellanox ConnectX6 Nics.
>
> Looking at the logs more I found these two lines:
>
> ```
>
> $vppctl show logs
>
> 2022/07/07 17:14:19:050 notice dpdk   EAL init args: -c 2 -n 4
> --in-memory --no-telemetry --file-prefix vpp -a :41:00.6 -a
> :41:01.6 --main-lcore 1
>
> 2022/07/07 17:14:19:161 notice dpdk   DPDK drivers found no
> Ethernet devices...
> ```
>
> I just found this post
>  in the mailing list.
> I assume this is my issue. I did not compile from source / configure
> appropriately. (I followed these instructions here
>  for
> how to deploy)
>
> Which specific package would I need to compile from source? (vpp
> vpp-plugin-core vpp-plugin-dpdk)?
> Is there any documentation on how to create a RMDA interface? Would an
> RDMA interface be just as efficient as a DPDK interface?
>
> - Dave
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21625): https://lists.fd.io/g/vpp-dev/message/21625
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

Hi Matt,

These are Mellanox ConnectX6 Nics.

Looking at the logs more I found these two lines:

```

$vppctl show logs

2022/07/07 17:14:19:050 notice     dpdk           EAL init args: -c 2 -n 4 
--in-memory --no-telemetry --file-prefix vpp -a :41:00.6 -a :41:01.6 
--main-lcore 1

2022/07/07 17:14:19:161 notice     dpdk           DPDK drivers found no 
Ethernet devices...

```

I just found this post ( https://lists.fd.io/g/vpp-dev/topic/87800334#20646 ) 
in the mailing list. I assume this is my issue. I did not compile from source / 
configure appropriately. (I followed these instructions here ( 
https://s3-docs.fd.io/vpp/22.10/gettingstarted/installing/index.html ) for how 
to deploy)

Which specific package would I need to compile from source? ( vpp 
vpp-plugin-core vpp-plugin-dpdk)?
Is there any documentation on how to create a RMDA interface? Would an RDMA 
interface be just as efficient as a DPDK interface?

- Dave

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21624): https://lists.fd.io/g/vpp-dev/message/21624
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Matthew Smith via lists.fd.io]

Hi Dave,

What is the manufacturer & model of the NICs you're trying to use? It's
possible that either VPP's DPDK plugin does not consider them valid or that
the DPDK PMD which is capable of managing them is not built by default.

-Matt


On Thu, Jul 7, 2022 at 11:12 AM Dave Houser  wrote:

> Hello,
>
> I am not able to see my white listed dpdk interfaces in vppctl with `show
> int` or `show hardware`. I did the following to integrate the interfaces.
>
> - I shutdown the interfaces with `ifconfig`
> `ifconfig enp65s0f6 down`
> `ifconfig enp65s1f6 down`
>
> - I unbound them from the kernel with dpdk_devbind.py
> `sudo dpdk-devbind.py -b uio_pci_generic :41:01.6`
> `sudo dpdk-devbind.py -b uio_pci_generic :41:01.6`
>
> - I confirmed the correct interfaces were removed and bound by dpdk
> `ip -br a` (interfaces removed)
> `dpdk-devbind.py --status` (interfaces show up as dpdk bound interfaces)
>
> - I made sure the interfaces were white listed in my startup.conf
> ```
> unix {
>   nodaemon
>   log /var/log/vpp/vpp.log
>   full-coredump
>   cli-listen /run/vpp/cli.sock
> }
>
> api-trace {
>   on
> }
>
> dpdk {
>   dev :41:00.6
>   dev :41:01.6
> }
> ```
>
> - I restarted vpp
> `systemctl restart vpp`
>
> - Watched journalctl -fu output
> ```
> Jul 07 15:56:17 host1 systemd[1]: Stopping vector packet processing
> engine...
> Jul 07 15:56:17 host1 vnet[66266]: unix_signal_handler:190: received
> signal SIGCONT, PC 0x7f5f7e426da0
> Jul 07 15:56:17 host1 vnet[66266]: received SIGTERM, exiting...
> Jul 07 15:56:17 host1 vnet[66266]: unix_signal_handler:190: received
> signal SIGCONT, PC 0x7f5f7e426da0
> Jul 07 15:56:17 host1 systemd[1]: Stopped vector packet processing engine.
> Jul 07 15:56:40 host1 systemd[1]: Starting vector packet processing
> engine...
> Jul 07 15:56:40 host1 systemd[1]: Started vector packet processing engine.
> Jul 07 15:56:40 host1 vnet[71758]: dpdk/cryptodev: dpdk_cryptodev_init:
> Failed to configure cryptodev
> ```
>
> - I accessed vpp
> `vppctl`
>
> Interfaces never show up. I followed everything in the following guides
> and posts:
> - https://lists.fd.io/g/vpp-dev/topic/10642649
> - https://s3-docs.fd.io/vpp/22.02/configuration/reference.html
>
> Here is my apt list
> ```
> $sudo apt list | grep vpp
> libvppinfra/bionic,now 21.10.1-release amd64 [installed,automatic]
> libvppinfra-dev/bionic,now 21.10.1-release amd64 [installed,automatic]
> python3-vpp-api/bionic 21.10.1-release amd64
> vpp/bionic,now 21.10.1-release amd64 [installed]
> vpp-api-java/bionic 19.04-release amd64
> vpp-api-lua/bionic 19.01.3-release amd64
> vpp-api-python/bionic 21.01.1-release amd64
> vpp-dbg/bionic,now 21.10.1-release amd64 [installed]
> vpp-dev/bionic,now 21.10.1-release amd64 [installed]
> vpp-ext-deps/bionic 19.04-16 amd64
> vpp-lib/bionic 19.01.3-release amd64
> vpp-plugin-core/bionic,now 21.10.1-release amd64 [installed]
> vpp-plugin-dpdk/bionic,now 21.10.1-release amd64 [installed]
> vpp-plugins/bionic 19.01.3-release amd64
> ```
>
> What am I missing? Why does vpp refuses to recognize my dpdk interfaces?
> Note: these are SRIOV vf interfaces, but I dont think that matter does it?
>
> - Dave
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21623): https://lists.fd.io/g/vpp-dev/message/21623
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] vpp refuses to use of see/use dpdk interfaces

[Dave Houser]

Hello,

I am not able to see my white listed dpdk interfaces in vppctl with `show
int` or `show hardware`. I did the following to integrate the interfaces.

- I shutdown the interfaces with `ifconfig`
`ifconfig enp65s0f6 down`
`ifconfig enp65s1f6 down`

- I unbound them from the kernel with dpdk_devbind.py
`sudo dpdk-devbind.py -b uio_pci_generic :41:01.6`
`sudo dpdk-devbind.py -b uio_pci_generic :41:01.6`

- I confirmed the correct interfaces were removed and bound by dpdk
`ip -br a` (interfaces removed)
`dpdk-devbind.py --status` (interfaces show up as dpdk bound interfaces)

- I made sure the interfaces were white listed in my startup.conf
```
unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen /run/vpp/cli.sock
}

api-trace {
  on
}

dpdk {
  dev :41:00.6
  dev :41:01.6
}
```

- I restarted vpp
`systemctl restart vpp`

- Watched journalctl -fu output
```
Jul 07 15:56:17 host1 systemd[1]: Stopping vector packet processing
engine...
Jul 07 15:56:17 host1 vnet[66266]: unix_signal_handler:190: received signal
SIGCONT, PC 0x7f5f7e426da0
Jul 07 15:56:17 host1 vnet[66266]: received SIGTERM, exiting...
Jul 07 15:56:17 host1 vnet[66266]: unix_signal_handler:190: received signal
SIGCONT, PC 0x7f5f7e426da0
Jul 07 15:56:17 host1 systemd[1]: Stopped vector packet processing engine.
Jul 07 15:56:40 host1 systemd[1]: Starting vector packet processing
engine...
Jul 07 15:56:40 host1 systemd[1]: Started vector packet processing engine.
Jul 07 15:56:40 host1 vnet[71758]: dpdk/cryptodev: dpdk_cryptodev_init:
Failed to configure cryptodev
```

- I accessed vpp
`vppctl`

Interfaces never show up. I followed everything in the following guides and
posts:
- https://lists.fd.io/g/vpp-dev/topic/10642649
- https://s3-docs.fd.io/vpp/22.02/configuration/reference.html

Here is my apt list
```
$sudo apt list | grep vpp
libvppinfra/bionic,now 21.10.1-release amd64 [installed,automatic]
libvppinfra-dev/bionic,now 21.10.1-release amd64 [installed,automatic]
python3-vpp-api/bionic 21.10.1-release amd64
vpp/bionic,now 21.10.1-release amd64 [installed]
vpp-api-java/bionic 19.04-release amd64
vpp-api-lua/bionic 19.01.3-release amd64
vpp-api-python/bionic 21.01.1-release amd64
vpp-dbg/bionic,now 21.10.1-release amd64 [installed]
vpp-dev/bionic,now 21.10.1-release amd64 [installed]
vpp-ext-deps/bionic 19.04-16 amd64
vpp-lib/bionic 19.01.3-release amd64
vpp-plugin-core/bionic,now 21.10.1-release amd64 [installed]
vpp-plugin-dpdk/bionic,now 21.10.1-release amd64 [installed]
vpp-plugins/bionic 19.01.3-release amd64
```

What am I missing? Why does vpp refuses to recognize my dpdk interfaces?
Note: these are SRIOV vf interfaces, but I dont think that matter does it?

- Dave

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21622): https://lists.fd.io/g/vpp-dev/message/21622
Mute This Topic: https://lists.fd.io/mt/92231790/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-