Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Bill Fischofer]

Thanks. I've opened Bug https://bugs.linaro.org/show_bug.cgi?id=3657
to track this.

On Tue, Mar 6, 2018 at 1:36 PM, gyanesh patra  wrote:
> Hi,
> I am coming back with the same issue here. As the odp-dpdk is updated to
> the recent DPDK version inline with the ODP code base, now both ODP repo
> and odp-dpdk repos stops working with mellanox interfaces. Please let me
> know if this is a known issue. I tried to run with master branch,
> caterpillar branch with and without 'abi-compat'. The ODP and odp-dpdk code
> compilation process is kept equal. The dpdk is compiled with mlx flags and
> working properly.
>
> Here is the configuratin details and the errors i get while trying to run
> ./test/performance/odp_l2fwd use case with odp github code base (master
> branch and has the v1.18.0.0 tag).
>
> <>
> ubuntu@ubuntu:~# uname -a
> Linux ubuntu 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018
> x86_64 x86_64 x86_64 GNU/Linux
>
> ubuntu@ubuntu:/home/gyanesh/dpdk-17.11# ./usertools/dpdk-devbind.py -s
>
> Network devices using DPDK-compatible driver
> 
> 
>
> Network devices using kernel driver
> ===
> :81:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp129s0f0 drv=mlx5_core
> unused=
> :81:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp129s0f1 drv=mlx5_core
> unused=
>
> <>
> ubuntu@ubuntu:/home/gyanesh/odp# git branch -a
>   caterpillar
> * master
>   remotes/origin/HEAD -> origin/master
>   remotes/origin/api-next
>   remotes/origin/caterpillar
>
> ubuntu@ubuntu:/home/gyanesh/odp# ./bootstrap
> ubuntu@ubuntu:/home/gyanesh/odp# ./configure
> --prefix=/home/gyanesh/odp/build
> --with-dpdk-path=/home/gyanesh/dpdk-17.11/x86_64-native-linuxapp-gcc/
> --enable-dpdk-zero-copy --disable-abi-compat --enable-debug-print
> --enable-debug --enable-helper-linux --disable-doxygen-doc
> --enable-helper-debug-print --enable-test-example --enable-test-helper
> --enable-test-perf --enable-test-vald LDFLAGS="-libverbs -lmlx5"
>
> ubuntu@ubuntu:/home/gyanesh/odp# make
> ubuntu@ubuntu:/home/gyanesh/odp# cd test/performance
>
> << performance testcase >>
> ubuntu@ubuntu:/home/gyanesh/odp/test/performance# export
> ODP_PLATFORM_PARAMS="-n 4 -m 15240"
> ubuntu@ubuntu:/home/gyanesh/odp/test/performance# ./odp_l2fwd -i 0,1 -c 2
> HW time counter freq: 240786 hz
>
> odp_system_info.c:100:default_huge_page_size():defaut hp size is 1048576 kB
> odp_system_info.c:100:default_huge_page_size():defaut hp size is 1048576 kB
> _ishm.c:1468:_odp_ishm_init_global():ishm: using dir /dev/shm
> _ishm.c:1484:_odp_ishm_init_global():Huge pages mount point is:
> /dev/hugepages
> _ishmphy.c:65:_odp_ishmphy_book_va():VA Reserved: 0x7f07dd70b000,
> len=0x6000
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=0, fd=3
> _fdserver.c:468:handle_request():storing {ctx=1, key=0}->fd=5
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=1, fd=4
> _fdserver.c:468:handle_request():storing {ctx=1, key=1}->fd=6
> odp_pool.c:108:odp_pool_init_global():
> Pool init global
> odp_pool.c:109:odp_pool_init_global():  odp_buffer_hdr_t size 256
> odp_pool.c:110:odp_pool_init_global():  odp_packet_hdr_t size 576
> odp_pool.c:111:odp_pool_init_global():
> odp_queue_basic.c:78:queue_init_global():Queue init ...
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=2, fd=5
> _fdserver.c:468:handle_request():storing {ctx=1, key=2}->fd=7
> odp_queue_lf.c:272:queue_lf_init_global():
> Lock-free queue init
> odp_queue_lf.c:273:queue_lf_init_global():  u128 lock-free: 1
>
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=3, fd=6
> _fdserver.c:468:handle_request():storing {ctx=1, key=3}->fd=8
> odp_queue_basic.c:103:queue_init_global():done
> odp_queue_basic.c:104:queue_init_global():Queue init global
> odp_queue_basic.c:106:queue_init_global():  struct queue_entry_s size 256
> odp_queue_basic.c:108:queue_init_global():  queue_entry_t size256
> odp_queue_basic.c:109:queue_init_global():
> Using scheduler 'basic'
> odp_schedule_basic.c:305:schedule_init_global():Schedule init ...
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=4, fd=7
> _fdserver.c:468:handle_request():storing {ctx=1, key=4}->fd=9
> odp_schedule_basic.c:365:schedule_init_global():done
> _fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
> key=5, fd=8
> _fdserver.c:468:handle_request():storing {ctx=1, key=5}->fd=10
> PKTIO: initialized loop interface.
> PKTIO: initialized dpdk pktio, use export ODP_PKTIO_DISABLE_DPDK=1 to
> disable.
> PKTIO: initialized pcap interface.
> PKTIO: initialized ipc interface.
> PKTIO: initialized null interface.
> PKTIO: initialized socket mmap, use export ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> to disable.
> PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> to 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

Hi,
I am coming back with the same issue here. As the odp-dpdk is updated to
the recent DPDK version inline with the ODP code base, now both ODP repo
and odp-dpdk repos stops working with mellanox interfaces. Please let me
know if this is a known issue. I tried to run with master branch,
caterpillar branch with and without 'abi-compat'. The ODP and odp-dpdk code
compilation process is kept equal. The dpdk is compiled with mlx flags and
working properly.

Here is the configuratin details and the errors i get while trying to run
./test/performance/odp_l2fwd use case with odp github code base (master
branch and has the v1.18.0.0 tag).

<>
ubuntu@ubuntu:~# uname -a
Linux ubuntu 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018
x86_64 x86_64 x86_64 GNU/Linux

ubuntu@ubuntu:/home/gyanesh/dpdk-17.11# ./usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver



Network devices using kernel driver
===
:81:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp129s0f0 drv=mlx5_core
unused=
:81:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp129s0f1 drv=mlx5_core
unused=

<>
ubuntu@ubuntu:/home/gyanesh/odp# git branch -a
  caterpillar
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/api-next
  remotes/origin/caterpillar

ubuntu@ubuntu:/home/gyanesh/odp# ./bootstrap
ubuntu@ubuntu:/home/gyanesh/odp# ./configure
--prefix=/home/gyanesh/odp/build
--with-dpdk-path=/home/gyanesh/dpdk-17.11/x86_64-native-linuxapp-gcc/
--enable-dpdk-zero-copy --disable-abi-compat --enable-debug-print
--enable-debug --enable-helper-linux --disable-doxygen-doc
--enable-helper-debug-print --enable-test-example --enable-test-helper
--enable-test-perf --enable-test-vald LDFLAGS="-libverbs -lmlx5"

ubuntu@ubuntu:/home/gyanesh/odp# make
ubuntu@ubuntu:/home/gyanesh/odp# cd test/performance

<< performance testcase >>
ubuntu@ubuntu:/home/gyanesh/odp/test/performance# export
ODP_PLATFORM_PARAMS="-n 4 -m 15240"
ubuntu@ubuntu:/home/gyanesh/odp/test/performance# ./odp_l2fwd -i 0,1 -c 2
HW time counter freq: 240786 hz

odp_system_info.c:100:default_huge_page_size():defaut hp size is 1048576 kB
odp_system_info.c:100:default_huge_page_size():defaut hp size is 1048576 kB
_ishm.c:1468:_odp_ishm_init_global():ishm: using dir /dev/shm
_ishm.c:1484:_odp_ishm_init_global():Huge pages mount point is:
/dev/hugepages
_ishmphy.c:65:_odp_ishmphy_book_va():VA Reserved: 0x7f07dd70b000,
len=0x6000
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=0, fd=3
_fdserver.c:468:handle_request():storing {ctx=1, key=0}->fd=5
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=1, fd=4
_fdserver.c:468:handle_request():storing {ctx=1, key=1}->fd=6
odp_pool.c:108:odp_pool_init_global():
Pool init global
odp_pool.c:109:odp_pool_init_global():  odp_buffer_hdr_t size 256
odp_pool.c:110:odp_pool_init_global():  odp_packet_hdr_t size 576
odp_pool.c:111:odp_pool_init_global():
odp_queue_basic.c:78:queue_init_global():Queue init ...
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=2, fd=5
_fdserver.c:468:handle_request():storing {ctx=1, key=2}->fd=7
odp_queue_lf.c:272:queue_lf_init_global():
Lock-free queue init
odp_queue_lf.c:273:queue_lf_init_global():  u128 lock-free: 1

_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=3, fd=6
_fdserver.c:468:handle_request():storing {ctx=1, key=3}->fd=8
odp_queue_basic.c:103:queue_init_global():done
odp_queue_basic.c:104:queue_init_global():Queue init global
odp_queue_basic.c:106:queue_init_global():  struct queue_entry_s size 256
odp_queue_basic.c:108:queue_init_global():  queue_entry_t size256
odp_queue_basic.c:109:queue_init_global():
Using scheduler 'basic'
odp_schedule_basic.c:305:schedule_init_global():Schedule init ...
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=4, fd=7
_fdserver.c:468:handle_request():storing {ctx=1, key=4}->fd=9
odp_schedule_basic.c:365:schedule_init_global():done
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=5, fd=8
_fdserver.c:468:handle_request():storing {ctx=1, key=5}->fd=10
PKTIO: initialized loop interface.
PKTIO: initialized dpdk pktio, use export ODP_PKTIO_DISABLE_DPDK=1 to
disable.
PKTIO: initialized pcap interface.
PKTIO: initialized ipc interface.
PKTIO: initialized null interface.
PKTIO: initialized socket mmap, use export ODP_PKTIO_DISABLE_SOCKET_MMAP=1
to disable.
PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=1
to disable.
odp_timer.c:1253:odp_timer_init_global():Using lock-less timer
implementation
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=6, fd=9
_fdserver.c:468:handle_request():storing {ctx=1, key=6}->fd=11
_fdserver.c:284:_odp_fdserver_register_fd():FD client register: pid=43825
key=7, fd=10
_fdserver.c:468:handle_request():storing {ctx=1, 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

With my initial testing, ODP-DPDK is working perfectly with mellanox
drivers.
Thank you

P Gyanesh Kumar Patra

On Thu, Nov 9, 2017 at 2:18 PM, Maxim Uvarov 
wrote:

> Nice to see it working. I think we did not yet tested it with Mellanox
> drivers.
>
> For linux-generic refer to .travis.yaml or ./scripts/build-pktio-dpdk
> scripts. Also all required steps are in README.
>
> Maxim.
>
> On 11/09/17 14:47, Elo, Matias (Nokia - FI/Espoo) wrote:
> > Hi Gyanesh,
> >
> > Pretty much the same steps should also work with odp linux-generic. The
> main difference is configure script. With linux-generic you use
> '--with-dpdk-path=' option and optionally
> --enable-dpdk-zero-copy flag. The supported dpdk  version is v17.08.
> >
> > -Matias
> >
> >> On 9 Nov 2017, at 10:34, gyanesh patra 
> wrote:
> >>
> >> Hi Maxim,
> >> Thanks for the help. I managed to figure out the configuration error
> and it
> >> works fine for "ODP-DPDK". The MLX5 pmd was not included properly.
> >>
> >> But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps
> to
> >> be able to use MLX ???
> >>
> >>
> >> P Gyanesh Kumar Patra
> >>
> >> On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
> >> wrote:
> >>
> >>> On 11/08/17 19:32, gyanesh patra wrote:
>  I am not sure what you mean. Can you please elaborate?
> 
>  As i mentioned before I am able to run dpdk examples. Hence the
> drivers
>  are available and working fine.
>  I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
>  work with mellanox. I followed the same while compiling dpdk too.
> 
>  Is there anything i am missing?
> 
>  P Gyanesh Kumar Patra
> >>>
> >>>
> >>> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
> >>> work. I think we did test only with ixgbe. But in general it's common
> code.
> >>>
> >>> "Unable to init any I/O type." means it it called all open for all
> pktio
> >>> in list here:
> >>> ./platform/linux-generic/pktio/io_ops.c
> >>>
> >>> and setup_pkt_dpdk() failed for some reason.
> >>>
> >>> I do not like allocations errors in your log.
> >>>
> >>> Try to compile ODP with --enable-debug-print --enable-debug it will
> make
> >>> ODP_DBG() macro work and it will be visible why it does not opens
> pktio.
> >>>
> >>> Maxim
> >>>
> >>>
> 
>  On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov   > wrote:
> 
> is Mellanox pmd compiled in?
> 
> Maxim.
> 
> On 11/08/17 17:58, gyanesh patra wrote:
> > Hi,
> > I am trying to run ODP & ODP-DPDK examples on our server with
> mellanox 100G
> > NICs. I am using the odp_l2fwd example. While running the example,
> I am
> > facing some issues.
> > -> When I run "ODP" example using the if names given by kernel as
> > arguments, I am not getting enough throughput.(the value is very
> >>> low)
> > -> And when I try "ODP-DPDK" example using port ID as "0,1", it
> >>> can't
> > create pktio. Whereas I am able to run the examples from "DPDK"
> > repo with portID "0,1" for the same mellanox NICs. I tried running
> with
> > "81:00.0,81:00.1" and also with if-names too without any success.
> Adding
> > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> >
> > Am I missing any steps to use mellanox NICs? OR is there a
> different method
> > to specify the device details to create pktio?
> > I am providing the output of "odp_l2fwd" examples for ODP and
> >>> ODP-DPDK
> > repository here.
> >
> > The NICs being used:
> >
> > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
> >>> drv=mlx5_core
> > unused=
> > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
> >>> drv=mlx5_core
> > unused=
> >
> > ODP l2fwd example run details:
> > --
> > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > enp129s0f0,enp129s0f1
> > HW time counter freq: 239886 
> <(239)%20999-9886> hz
> >
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> pages.
> > check: /proc/sys/vm/nr_hugepages.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > PKTIO: initialized loop interface.
> > PKTIO: initialized pcap interface.
> > PKTIO: initialized ipc interface.
> > PKTIO: initialized socket mmap, use export
> ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> > to disable.
> > PKTIO: initialized socket mmsg,use export
> 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

This is good news. I was wondering if it was in the pipeline. Thank you

P Gyanesh Kumar Patra

On Thu, Nov 9, 2017 at 7:25 PM, Francois Ozog 
wrote:

> ODP2.0 should allow ODP to leverage directly libiverbs from a native ODP
> pktio without DPDK layer.
> Mellanox has created a userland framework based on libiverbs while we try
> to promote an extension of Mediated Device (vfio-mdev).
>
> FF
>
> On 9 November 2017 at 14:18, Maxim Uvarov  wrote:
>
>> Nice to see it working. I think we did not yet tested it with Mellanox
>> drivers.
>>
>> For linux-generic refer to .travis.yaml or ./scripts/build-pktio-dpdk
>> scripts. Also all required steps are in README.
>>
>> Maxim.
>>
>> On 11/09/17 14:47, Elo, Matias (Nokia - FI/Espoo) wrote:
>> > Hi Gyanesh,
>> >
>> > Pretty much the same steps should also work with odp linux-generic. The
>> main difference is configure script. With linux-generic you use
>> '--with-dpdk-path=' option and optionally
>> --enable-dpdk-zero-copy flag. The supported dpdk  version is v17.08.
>> >
>> > -Matias
>> >
>> >> On 9 Nov 2017, at 10:34, gyanesh patra 
>> wrote:
>> >>
>> >> Hi Maxim,
>> >> Thanks for the help. I managed to figure out the configuration error
>> and it
>> >> works fine for "ODP-DPDK". The MLX5 pmd was not included properly.
>> >>
>> >> But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps
>> to
>> >> be able to use MLX ???
>> >>
>> >>
>> >> P Gyanesh Kumar Patra
>> >>
>> >> On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
>> >> wrote:
>> >>
>> >>> On 11/08/17 19:32, gyanesh patra wrote:
>>  I am not sure what you mean. Can you please elaborate?
>> 
>>  As i mentioned before I am able to run dpdk examples. Hence the
>> drivers
>>  are available and working fine.
>>  I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
>>  work with mellanox. I followed the same while compiling dpdk too.
>> 
>>  Is there anything i am missing?
>> 
>>  P Gyanesh Kumar Patra
>> >>>
>> >>>
>> >>> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has
>> to
>> >>> work. I think we did test only with ixgbe. But in general it's common
>> code.
>> >>>
>> >>> "Unable to init any I/O type." means it it called all open for all
>> pktio
>> >>> in list here:
>> >>> ./platform/linux-generic/pktio/io_ops.c
>> >>>
>> >>> and setup_pkt_dpdk() failed for some reason.
>> >>>
>> >>> I do not like allocations errors in your log.
>> >>>
>> >>> Try to compile ODP with --enable-debug-print --enable-debug it will
>> make
>> >>> ODP_DBG() macro work and it will be visible why it does not opens
>> pktio.
>> >>>
>> >>> Maxim
>> >>>
>> >>>
>> 
>>  On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov <
>> maxim.uva...@linaro.org
>>  > wrote:
>> 
>> is Mellanox pmd compiled in?
>> 
>> Maxim.
>> 
>> On 11/08/17 17:58, gyanesh patra wrote:
>> > Hi,
>> > I am trying to run ODP & ODP-DPDK examples on our server with
>> mellanox 100G
>> > NICs. I am using the odp_l2fwd example. While running the example,
>> I am
>> > facing some issues.
>> > -> When I run "ODP" example using the if names given by kernel as
>> > arguments, I am not getting enough throughput.(the value is very
>> >>> low)
>> > -> And when I try "ODP-DPDK" example using port ID as "0,1", it
>> >>> can't
>> > create pktio. Whereas I am able to run the examples from "DPDK"
>> > repo with portID "0,1" for the same mellanox NICs. I tried running
>> with
>> > "81:00.0,81:00.1" and also with if-names too without any success.
>> Adding
>> > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
>> >
>> > Am I missing any steps to use mellanox NICs? OR is there a
>> different method
>> > to specify the device details to create pktio?
>> > I am providing the output of "odp_l2fwd" examples for ODP and
>> >>> ODP-DPDK
>> > repository here.
>> >
>> > The NICs being used:
>> >
>> > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
>> >>> drv=mlx5_core
>> > unused=
>> > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
>> >>> drv=mlx5_core
>> > unused=
>> >
>> > ODP l2fwd example run details:
>> > --
>> > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
>> > enp129s0f0,enp129s0f1
>> > HW time counter freq: 239886 <(239)%20999-9886> > <(239)%20999-9886>>
>> <(239)%20999-9886> hz
>> >
>> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> >>> memory
>> > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
>> pages.
>> > check: /proc/sys/vm/nr_hugepages.
>> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> >>> memory
>> 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Francois Ozog]

ODP2.0 should allow ODP to leverage directly libiverbs from a native ODP
pktio without DPDK layer.
Mellanox has created a userland framework based on libiverbs while we try
to promote an extension of Mediated Device (vfio-mdev).

FF

On 9 November 2017 at 14:18, Maxim Uvarov  wrote:

> Nice to see it working. I think we did not yet tested it with Mellanox
> drivers.
>
> For linux-generic refer to .travis.yaml or ./scripts/build-pktio-dpdk
> scripts. Also all required steps are in README.
>
> Maxim.
>
> On 11/09/17 14:47, Elo, Matias (Nokia - FI/Espoo) wrote:
> > Hi Gyanesh,
> >
> > Pretty much the same steps should also work with odp linux-generic. The
> main difference is configure script. With linux-generic you use
> '--with-dpdk-path=' option and optionally
> --enable-dpdk-zero-copy flag. The supported dpdk  version is v17.08.
> >
> > -Matias
> >
> >> On 9 Nov 2017, at 10:34, gyanesh patra 
> wrote:
> >>
> >> Hi Maxim,
> >> Thanks for the help. I managed to figure out the configuration error
> and it
> >> works fine for "ODP-DPDK". The MLX5 pmd was not included properly.
> >>
> >> But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps
> to
> >> be able to use MLX ???
> >>
> >>
> >> P Gyanesh Kumar Patra
> >>
> >> On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
> >> wrote:
> >>
> >>> On 11/08/17 19:32, gyanesh patra wrote:
>  I am not sure what you mean. Can you please elaborate?
> 
>  As i mentioned before I am able to run dpdk examples. Hence the
> drivers
>  are available and working fine.
>  I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
>  work with mellanox. I followed the same while compiling dpdk too.
> 
>  Is there anything i am missing?
> 
>  P Gyanesh Kumar Patra
> >>>
> >>>
> >>> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
> >>> work. I think we did test only with ixgbe. But in general it's common
> code.
> >>>
> >>> "Unable to init any I/O type." means it it called all open for all
> pktio
> >>> in list here:
> >>> ./platform/linux-generic/pktio/io_ops.c
> >>>
> >>> and setup_pkt_dpdk() failed for some reason.
> >>>
> >>> I do not like allocations errors in your log.
> >>>
> >>> Try to compile ODP with --enable-debug-print --enable-debug it will
> make
> >>> ODP_DBG() macro work and it will be visible why it does not opens
> pktio.
> >>>
> >>> Maxim
> >>>
> >>>
> 
>  On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov   > wrote:
> 
> is Mellanox pmd compiled in?
> 
> Maxim.
> 
> On 11/08/17 17:58, gyanesh patra wrote:
> > Hi,
> > I am trying to run ODP & ODP-DPDK examples on our server with
> mellanox 100G
> > NICs. I am using the odp_l2fwd example. While running the example,
> I am
> > facing some issues.
> > -> When I run "ODP" example using the if names given by kernel as
> > arguments, I am not getting enough throughput.(the value is very
> >>> low)
> > -> And when I try "ODP-DPDK" example using port ID as "0,1", it
> >>> can't
> > create pktio. Whereas I am able to run the examples from "DPDK"
> > repo with portID "0,1" for the same mellanox NICs. I tried running
> with
> > "81:00.0,81:00.1" and also with if-names too without any success.
> Adding
> > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> >
> > Am I missing any steps to use mellanox NICs? OR is there a
> different method
> > to specify the device details to create pktio?
> > I am providing the output of "odp_l2fwd" examples for ODP and
> >>> ODP-DPDK
> > repository here.
> >
> > The NICs being used:
> >
> > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
> >>> drv=mlx5_core
> > unused=
> > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
> >>> drv=mlx5_core
> > unused=
> >
> > ODP l2fwd example run details:
> > --
> > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > enp129s0f0,enp129s0f1
> > HW time counter freq: 239886 
> <(239)%20999-9886> hz
> >
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> pages.
> > check: /proc/sys/vm/nr_hugepages.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> >>> memory
> > PKTIO: initialized loop interface.
> > PKTIO: initialized pcap interface.
> > PKTIO: initialized ipc interface.
> > PKTIO: initialized socket mmap, use export
> 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Maxim Uvarov]

Nice to see it working. I think we did not yet tested it with Mellanox
drivers.

For linux-generic refer to .travis.yaml or ./scripts/build-pktio-dpdk
scripts. Also all required steps are in README.

Maxim.

On 11/09/17 14:47, Elo, Matias (Nokia - FI/Espoo) wrote:
> Hi Gyanesh,
> 
> Pretty much the same steps should also work with odp linux-generic. The main 
> difference is configure script. With linux-generic you use 
> '--with-dpdk-path=' option and optionally --enable-dpdk-zero-copy 
> flag. The supported dpdk  version is v17.08.
> 
> -Matias
> 
>> On 9 Nov 2017, at 10:34, gyanesh patra  wrote:
>>
>> Hi Maxim,
>> Thanks for the help. I managed to figure out the configuration error and it
>> works fine for "ODP-DPDK". The MLX5 pmd was not included properly.
>>
>> But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps to
>> be able to use MLX ???
>>
>>
>> P Gyanesh Kumar Patra
>>
>> On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
>> wrote:
>>
>>> On 11/08/17 19:32, gyanesh patra wrote:
 I am not sure what you mean. Can you please elaborate?

 As i mentioned before I am able to run dpdk examples. Hence the drivers
 are available and working fine.
 I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
 work with mellanox. I followed the same while compiling dpdk too.

 Is there anything i am missing?

 P Gyanesh Kumar Patra
>>>
>>>
>>> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
>>> work. I think we did test only with ixgbe. But in general it's common code.
>>>
>>> "Unable to init any I/O type." means it it called all open for all pktio
>>> in list here:
>>> ./platform/linux-generic/pktio/io_ops.c
>>>
>>> and setup_pkt_dpdk() failed for some reason.
>>>
>>> I do not like allocations errors in your log.
>>>
>>> Try to compile ODP with --enable-debug-print --enable-debug it will make
>>> ODP_DBG() macro work and it will be visible why it does not opens pktio.
>>>
>>> Maxim
>>>
>>>

 On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov > wrote:

is Mellanox pmd compiled in?

Maxim.

On 11/08/17 17:58, gyanesh patra wrote:
> Hi,
> I am trying to run ODP & ODP-DPDK examples on our server with
mellanox 100G
> NICs. I am using the odp_l2fwd example. While running the example,
I am
> facing some issues.
> -> When I run "ODP" example using the if names given by kernel as
> arguments, I am not getting enough throughput.(the value is very
>>> low)
> -> And when I try "ODP-DPDK" example using port ID as "0,1", it
>>> can't
> create pktio. Whereas I am able to run the examples from "DPDK"
> repo with portID "0,1" for the same mellanox NICs. I tried running
with
> "81:00.0,81:00.1" and also with if-names too without any success.
Adding
> the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
>
> Am I missing any steps to use mellanox NICs? OR is there a
different method
> to specify the device details to create pktio?
> I am providing the output of "odp_l2fwd" examples for ODP and
>>> ODP-DPDK
> repository here.
>
> The NICs being used:
>
> :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
>>> drv=mlx5_core
> unused=
> :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
>>> drv=mlx5_core
> unused=
>
> ODP l2fwd example run details:
> --
> root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> enp129s0f0,enp129s0f1
> HW time counter freq: 239886 
<(239)%20999-9886> hz
>
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
pages.
> check: /proc/sys/vm/nr_hugepages.
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> PKTIO: initialized loop interface.
> PKTIO: initialized pcap interface.
> PKTIO: initialized ipc interface.
> PKTIO: initialized socket mmap, use export
ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> to disable.
> PKTIO: initialized socket mmsg,use export
ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> to disable.
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>>> memory
>
> ODP system info
> ---
> ODP API version: 1.15.0
> ODP impl name:   "odp-linux"

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Elo, Matias (Nokia - FI/Espoo)]

Hi Gyanesh,

Pretty much the same steps should also work with odp linux-generic. The main 
difference is configure script. With linux-generic you use 
'--with-dpdk-path=' option and optionally --enable-dpdk-zero-copy 
flag. The supported dpdk  version is v17.08.

-Matias

> On 9 Nov 2017, at 10:34, gyanesh patra  wrote:
> 
> Hi Maxim,
> Thanks for the help. I managed to figure out the configuration error and it
> works fine for "ODP-DPDK". The MLX5 pmd was not included properly.
> 
> But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps to
> be able to use MLX ???
> 
> 
> P Gyanesh Kumar Patra
> 
> On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
> wrote:
> 
>> On 11/08/17 19:32, gyanesh patra wrote:
>>> I am not sure what you mean. Can you please elaborate?
>>> 
>>> As i mentioned before I am able to run dpdk examples. Hence the drivers
>>> are available and working fine.
>>> I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
>>> work with mellanox. I followed the same while compiling dpdk too.
>>> 
>>> Is there anything i am missing?
>>> 
>>> P Gyanesh Kumar Patra
>> 
>> 
>> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
>> work. I think we did test only with ixgbe. But in general it's common code.
>> 
>> "Unable to init any I/O type." means it it called all open for all pktio
>> in list here:
>> ./platform/linux-generic/pktio/io_ops.c
>> 
>> and setup_pkt_dpdk() failed for some reason.
>> 
>> I do not like allocations errors in your log.
>> 
>> Try to compile ODP with --enable-debug-print --enable-debug it will make
>> ODP_DBG() macro work and it will be visible why it does not opens pktio.
>> 
>> Maxim
>> 
>> 
>>> 
>>> On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov >> > wrote:
>>> 
>>>is Mellanox pmd compiled in?
>>> 
>>>Maxim.
>>> 
>>>On 11/08/17 17:58, gyanesh patra wrote:
 Hi,
 I am trying to run ODP & ODP-DPDK examples on our server with
>>>mellanox 100G
 NICs. I am using the odp_l2fwd example. While running the example,
>>>I am
 facing some issues.
 -> When I run "ODP" example using the if names given by kernel as
 arguments, I am not getting enough throughput.(the value is very
>> low)
 -> And when I try "ODP-DPDK" example using port ID as "0,1", it
>> can't
 create pktio. Whereas I am able to run the examples from "DPDK"
 repo with portID "0,1" for the same mellanox NICs. I tried running
>>>with
 "81:00.0,81:00.1" and also with if-names too without any success.
>>>Adding
 the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
 
 Am I missing any steps to use mellanox NICs? OR is there a
>>>different method
 to specify the device details to create pktio?
 I am providing the output of "odp_l2fwd" examples for ODP and
>> ODP-DPDK
 repository here.
 
 The NICs being used:
 
 :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
>> drv=mlx5_core
 unused=
 :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
>> drv=mlx5_core
 unused=
 
 ODP l2fwd example run details:
 --
 root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
 enp129s0f0,enp129s0f1
 HW time counter freq: 239886 
>>><(239)%20999-9886> hz
 
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
>>>pages.
 check: /proc/sys/vm/nr_hugepages.
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 PKTIO: initialized loop interface.
 PKTIO: initialized pcap interface.
 PKTIO: initialized ipc interface.
 PKTIO: initialized socket mmap, use export
>>>ODP_PKTIO_DISABLE_SOCKET_MMAP=1
 to disable.
 PKTIO: initialized socket mmsg,use export
>>>ODP_PKTIO_DISABLE_SOCKET_MMSG=1
 to disable.
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
>> memory
 
 ODP system info
 ---
 ODP API version: 1.15.0
 ODP impl name:   "odp-linux"
 CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
 CPU freq (hz):   33
 Cache line size: 64
 CPU count:   56
 
 
 CPU features supported:
 SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA
>>>CMPXCHG16B
 XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE
>>>AES XSAVE
 OSXSAVE AVX F16C RDRAND 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

Hi Maxim,
Thanks for the help. I managed to figure out the configuration error and it
works fine for "ODP-DPDK". The MLX5 pmd was not included properly.

But regarding "ODP" repo (not odp-dpdk), do i need to follow any steps to
be able to use MLX ???


P Gyanesh Kumar Patra

On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
wrote:

> On 11/08/17 19:32, gyanesh patra wrote:
> > I am not sure what you mean. Can you please elaborate?
> >
> > As i mentioned before I am able to run dpdk examples. Hence the drivers
> > are available and working fine.
> > I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
> > work with mellanox. I followed the same while compiling dpdk too.
> >
> > Is there anything i am missing?
> >
> > P Gyanesh Kumar Patra
>
>
> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
> work. I think we did test only with ixgbe. But in general it's common code.
>
> "Unable to init any I/O type." means it it called all open for all pktio
> in list here:
> ./platform/linux-generic/pktio/io_ops.c
>
> and setup_pkt_dpdk() failed for some reason.
>
> I do not like allocations errors in your log.
>
> Try to compile ODP with --enable-debug-print --enable-debug it will make
> ODP_DBG() macro work and it will be visible why it does not opens pktio.
>
> Maxim
>
>
> >
> > On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov  > > wrote:
> >
> > is Mellanox pmd compiled in?
> >
> > Maxim.
> >
> > On 11/08/17 17:58, gyanesh patra wrote:
> > > Hi,
> > > I am trying to run ODP & ODP-DPDK examples on our server with
> > mellanox 100G
> > > NICs. I am using the odp_l2fwd example. While running the example,
> > I am
> > > facing some issues.
> > > -> When I run "ODP" example using the if names given by kernel as
> > > arguments, I am not getting enough throughput.(the value is very
> low)
> > > -> And when I try "ODP-DPDK" example using port ID as "0,1", it
> can't
> > > create pktio. Whereas I am able to run the examples from "DPDK"
> > > repo with portID "0,1" for the same mellanox NICs. I tried running
> > with
> > > "81:00.0,81:00.1" and also with if-names too without any success.
> > Adding
> > > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> > >
> > > Am I missing any steps to use mellanox NICs? OR is there a
> > different method
> > > to specify the device details to create pktio?
> > > I am providing the output of "odp_l2fwd" examples for ODP and
> ODP-DPDK
> > > repository here.
> > >
> > > The NICs being used:
> > >
> > > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
> drv=mlx5_core
> > > unused=
> > > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
> drv=mlx5_core
> > > unused=
> > >
> > > ODP l2fwd example run details:
> > > --
> > > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > > enp129s0f0,enp129s0f1
> > > HW time counter freq: 239886 
> > <(239)%20999-9886> hz
> > >
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> > pages.
> > > check: /proc/sys/vm/nr_hugepages.
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > >  PKTIO: initialized loop interface.
> > >  PKTIO: initialized pcap interface.
> > >  PKTIO: initialized ipc interface.
> > >  PKTIO: initialized socket mmap, use export
> > ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> > > to disable.
> > >  PKTIO: initialized socket mmsg,use export
> > ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> > > to disable.
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > >
> > > ODP system info
> > > ---
> > > ODP API version: 1.15.0
> > > ODP impl name:   "odp-linux"
> > > CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
> > > CPU freq (hz):   33
> > > Cache line size: 64
> > > CPU count:   56
> > >
> > >
> > > CPU features supported:
> > > SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA
> > CMPXCHG16B
> > > XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE
> > AES XSAVE
> > > OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC
> > SEP MTRR
> > > PGE MCA CMOV PAT PSE36 CLFSH DS ACPI 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

- where to set CONFIG_RTE_LIBRTE_MLX5_PMD ? I believe in DPDK we set it
whilie compiling. How to set it for ODP??

- with mellanox NICs, it is not required to explicitly bind the interface
to dpdk. I can use portID "0,1"  with dpdk example. Is it the same way to
specify with ODP-DPDK l2fwd example too??


P Gyanesh Kumar Patra

On Wed, Nov 8, 2017 at 7:56 PM, Maxim Uvarov 
wrote:

> On 11/08/17 19:32, gyanesh patra wrote:
> > I am not sure what you mean. Can you please elaborate?
> >
> > As i mentioned before I am able to run dpdk examples. Hence the drivers
> > are available and working fine.
> > I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
> > work with mellanox. I followed the same while compiling dpdk too.
> >
> > Is there anything i am missing?
> >
> > P Gyanesh Kumar Patra
>
>
> in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
> work. I think we did test only with ixgbe. But in general it's common code.
>
> "Unable to init any I/O type." means it it called all open for all pktio
> in list here:
> ./platform/linux-generic/pktio/io_ops.c
>
> and setup_pkt_dpdk() failed for some reason.
>
> I do not like allocations errors in your log.
>
> Try to compile ODP with --enable-debug-print --enable-debug it will make
> ODP_DBG() macro work and it will be visible why it does not opens pktio.
>
> Maxim
>
>
> >
> > On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov  > > wrote:
> >
> > is Mellanox pmd compiled in?
> >
> > Maxim.
> >
> > On 11/08/17 17:58, gyanesh patra wrote:
> > > Hi,
> > > I am trying to run ODP & ODP-DPDK examples on our server with
> > mellanox 100G
> > > NICs. I am using the odp_l2fwd example. While running the example,
> > I am
> > > facing some issues.
> > > -> When I run "ODP" example using the if names given by kernel as
> > > arguments, I am not getting enough throughput.(the value is very
> low)
> > > -> And when I try "ODP-DPDK" example using port ID as "0,1", it
> can't
> > > create pktio. Whereas I am able to run the examples from "DPDK"
> > > repo with portID "0,1" for the same mellanox NICs. I tried running
> > with
> > > "81:00.0,81:00.1" and also with if-names too without any success.
> > Adding
> > > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> > >
> > > Am I missing any steps to use mellanox NICs? OR is there a
> > different method
> > > to specify the device details to create pktio?
> > > I am providing the output of "odp_l2fwd" examples for ODP and
> ODP-DPDK
> > > repository here.
> > >
> > > The NICs being used:
> > >
> > > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0
> drv=mlx5_core
> > > unused=
> > > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1
> drv=mlx5_core
> > > unused=
> > >
> > > ODP l2fwd example run details:
> > > --
> > > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > > enp129s0f0,enp129s0f1
> > > HW time counter freq: 239886 
> > <(239)%20999-9886> hz
> > >
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> > pages.
> > > check: /proc/sys/vm/nr_hugepages.
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > >  PKTIO: initialized loop interface.
> > >  PKTIO: initialized pcap interface.
> > >  PKTIO: initialized ipc interface.
> > >  PKTIO: initialized socket mmap, use export
> > ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> > > to disable.
> > >  PKTIO: initialized socket mmsg,use export
> > ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> > > to disable.
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate
> memory
> > >
> > > ODP system info
> > > ---
> > > ODP API version: 1.15.0
> > > ODP impl name:   "odp-linux"
> > > CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
> > > CPU freq (hz):   33
> > > Cache line size: 64
> > > CPU count:   56
> > >
> > >
> > > CPU features supported:
> > > SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA
> > CMPXCHG16B
> > > XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE
> > AES XSAVE
> > > OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC
> > SEP 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Maxim Uvarov]

On 11/08/17 19:32, gyanesh patra wrote:
> I am not sure what you mean. Can you please elaborate?
> 
> As i mentioned before I am able to run dpdk examples. Hence the drivers
> are available and working fine.
> I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to
> work with mellanox. I followed the same while compiling dpdk too.
> 
> Is there anything i am missing?
> 
> P Gyanesh Kumar Patra


in general if CONFIG_RTE_LIBRTE_MLX5_PMD=y was specified then it has to
work. I think we did test only with ixgbe. But in general it's common code.

"Unable to init any I/O type." means it it called all open for all pktio
in list here:
./platform/linux-generic/pktio/io_ops.c

and setup_pkt_dpdk() failed for some reason.

I do not like allocations errors in your log.

Try to compile ODP with --enable-debug-print --enable-debug it will make
ODP_DBG() macro work and it will be visible why it does not opens pktio.

Maxim


> 
> On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov  > wrote:
> 
> is Mellanox pmd compiled in?
> 
> Maxim.
> 
> On 11/08/17 17:58, gyanesh patra wrote:
> > Hi,
> > I am trying to run ODP & ODP-DPDK examples on our server with
> mellanox 100G
> > NICs. I am using the odp_l2fwd example. While running the example,
> I am
> > facing some issues.
> > -> When I run "ODP" example using the if names given by kernel as
> > arguments, I am not getting enough throughput.(the value is very low)
> > -> And when I try "ODP-DPDK" example using port ID as "0,1", it can't
> > create pktio. Whereas I am able to run the examples from "DPDK"
> > repo with portID "0,1" for the same mellanox NICs. I tried running
> with
> > "81:00.0,81:00.1" and also with if-names too without any success.
> Adding
> > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> >
> > Am I missing any steps to use mellanox NICs? OR is there a
> different method
> > to specify the device details to create pktio?
> > I am providing the output of "odp_l2fwd" examples for ODP and ODP-DPDK
> > repository here.
> >
> > The NICs being used:
> >
> > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0 drv=mlx5_core
> > unused=
> > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1 drv=mlx5_core
> > unused=
> >
> > ODP l2fwd example run details:
> > --
> > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > enp129s0f0,enp129s0f1
> > HW time counter freq: 239886 
> <(239)%20999-9886> hz
> >
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> pages.
> > check: /proc/sys/vm/nr_hugepages.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> >  PKTIO: initialized loop interface.
> >  PKTIO: initialized pcap interface.
> >  PKTIO: initialized ipc interface.
> >  PKTIO: initialized socket mmap, use export
> ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> > to disable.
> >  PKTIO: initialized socket mmsg,use export
> ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> > to disable.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> >
> > ODP system info
> > ---
> > ODP API version: 1.15.0
> > ODP impl name:   "odp-linux"
> > CPU model:       Intel(R) Xeon(R) CPU E5-2680 v4
> > CPU freq (hz):   33
> > Cache line size: 64
> > CPU count:       56
> >
> >
> > CPU features supported:
> > SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA
> CMPXCHG16B
> > XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE
> AES XSAVE
> > OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC
> SEP MTRR
> > PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE
> > DIGTEMP TRBOBST ARAT PLN ECMD PTM MPERF_APERF_MSR ENERGY_EFF
> FSGSBASE HLE
> > AVX2 BMI2 ERMS INVPCID RTM LAHF_SAHF SYSCALL XD 1GB_PG RDTSCP
> EM64T INVTSC
> >
> > CPU features NOT supported:
> > CNXT_ID PSN ACNT2 BMI1 SMEP AVX512F LZCNT
> >
> > Running ODP appl: "odp_l2fwd"
> > -
> > IF-count:        2
> > Using IFs:       enp129s0f0 enp129s0f1
> > Mode:            PKTIN_DIRECT, PKTOUT_DIRECT
> >
> > num worker threads: 32
> > first CPU:          24
> > cpu mask:      

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

I am not sure what you mean. Can you please elaborate?

As i mentioned before I am able to run dpdk examples. Hence the drivers are
available and working fine.
I configured ODP & ODP-DPDK with "LDFLAGS=-libverbs" and compiled to work
with mellanox. I followed the same while compiling dpdk too.

Is there anything i am missing?

P Gyanesh Kumar Patra

On Wed, Nov 8, 2017 at 5:22 PM, Maxim Uvarov 
wrote:

> is Mellanox pmd compiled in?
>
> Maxim.
>
> On 11/08/17 17:58, gyanesh patra wrote:
> > Hi,
> > I am trying to run ODP & ODP-DPDK examples on our server with mellanox
> 100G
> > NICs. I am using the odp_l2fwd example. While running the example, I am
> > facing some issues.
> > -> When I run "ODP" example using the if names given by kernel as
> > arguments, I am not getting enough throughput.(the value is very low)
> > -> And when I try "ODP-DPDK" example using port ID as "0,1", it can't
> > create pktio. Whereas I am able to run the examples from "DPDK"
> > repo with portID "0,1" for the same mellanox NICs. I tried running with
> > "81:00.0,81:00.1" and also with if-names too without any success. Adding
> > the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> >
> > Am I missing any steps to use mellanox NICs? OR is there a different
> method
> > to specify the device details to create pktio?
> > I am providing the output of "odp_l2fwd" examples for ODP and ODP-DPDK
> > repository here.
> >
> > The NICs being used:
> >
> > :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0 drv=mlx5_core
> > unused=
> > :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1 drv=mlx5_core
> > unused=
> >
> > ODP l2fwd example run details:
> > --
> > root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> > enp129s0f0,enp129s0f1
> > HW time counter freq: 239886 <(239)%20999-9886> hz
> >
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal
> pages.
> > check: /proc/sys/vm/nr_hugepages.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> >  PKTIO: initialized loop interface.
> >  PKTIO: initialized pcap interface.
> >  PKTIO: initialized ipc interface.
> >  PKTIO: initialized socket mmap, use export
> ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> > to disable.
> >  PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=
> 1
> > to disable.
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> >
> > ODP system info
> > ---
> > ODP API version: 1.15.0
> > ODP impl name:   "odp-linux"
> > CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
> > CPU freq (hz):   33
> > Cache line size: 64
> > CPU count:   56
> >
> >
> > CPU features supported:
> > SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA
> CMPXCHG16B
> > XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE AES
> XSAVE
> > OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR
> > PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE
> > DIGTEMP TRBOBST ARAT PLN ECMD PTM MPERF_APERF_MSR ENERGY_EFF FSGSBASE HLE
> > AVX2 BMI2 ERMS INVPCID RTM LAHF_SAHF SYSCALL XD 1GB_PG RDTSCP EM64T
> INVTSC
> >
> > CPU features NOT supported:
> > CNXT_ID PSN ACNT2 BMI1 SMEP AVX512F LZCNT
> >
> > Running ODP appl: "odp_l2fwd"
> > -
> > IF-count:2
> > Using IFs:   enp129s0f0 enp129s0f1
> > Mode:PKTIN_DIRECT, PKTOUT_DIRECT
> >
> > num worker threads: 32
> > first CPU:  24
> > cpu mask:   0x00
> >
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> > _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> >
> > Pool info
> > -
> >   pool0
> >   namepacket pool
> >   pool type   packet
> >   pool shm11
> >   user area shm   0
> >   num 8192
> >   align   64
> >   headroom128
> >   seg len 8064
> >   max data len65536
> >   tailroom0
> >   block size  8768
> >   uarea size  0
> >   shm size72143104
> >   base addr   0x7f5fc1234000
> >   uarea shm size  0
> >   uarea base addr (nil)
> >
> > pktio/socket_mmap.c:401:mmap_setup_ring():setsockopt(pkt mmap): Invalid
> > argument
> > pktio/socket_mmap.c:496:sock_mmap_close():mmap_unmap_sock() Invalid
> argument
> > created pktio 1, dev: enp129s0f0, drv: socket
> > Sharing 1 input queues between 16 workers
> > Sharing 1 output queues between 16 workers
> > created 1 input and 1 output 

Re: [lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[Maxim Uvarov]

is Mellanox pmd compiled in?

Maxim.

On 11/08/17 17:58, gyanesh patra wrote:
> Hi,
> I am trying to run ODP & ODP-DPDK examples on our server with mellanox 100G
> NICs. I am using the odp_l2fwd example. While running the example, I am
> facing some issues.
> -> When I run "ODP" example using the if names given by kernel as
> arguments, I am not getting enough throughput.(the value is very low)
> -> And when I try "ODP-DPDK" example using port ID as "0,1", it can't
> create pktio. Whereas I am able to run the examples from "DPDK"
> repo with portID "0,1" for the same mellanox NICs. I tried running with
> "81:00.0,81:00.1" and also with if-names too without any success. Adding
> the whitelist using ODP_PLATFORM_PARAMS doesn't help either.
> 
> Am I missing any steps to use mellanox NICs? OR is there a different method
> to specify the device details to create pktio?
> I am providing the output of "odp_l2fwd" examples for ODP and ODP-DPDK
> repository here.
> 
> The NICs being used:
> 
> :81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0 drv=mlx5_core
> unused=
> :81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1 drv=mlx5_core
> unused=
> 
> ODP l2fwd example run details:
> --
> root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
> enp129s0f0,enp129s0f1
> HW time counter freq: 239886 <(239)%20999-9886> hz
> 
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal pages.
> check: /proc/sys/vm/nr_hugepages.
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
>  PKTIO: initialized loop interface.
>  PKTIO: initialized pcap interface.
>  PKTIO: initialized ipc interface.
>  PKTIO: initialized socket mmap, use export ODP_PKTIO_DISABLE_SOCKET_MMAP=1
> to disable.
>  PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=1
> to disable.
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> 
> ODP system info
> ---
> ODP API version: 1.15.0
> ODP impl name:   "odp-linux"
> CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
> CPU freq (hz):   33
> Cache line size: 64
> CPU count:   56
> 
> 
> CPU features supported:
> SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA CMPXCHG16B
> XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE AES XSAVE
> OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR
> PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE
> DIGTEMP TRBOBST ARAT PLN ECMD PTM MPERF_APERF_MSR ENERGY_EFF FSGSBASE HLE
> AVX2 BMI2 ERMS INVPCID RTM LAHF_SAHF SYSCALL XD 1GB_PG RDTSCP EM64T INVTSC
> 
> CPU features NOT supported:
> CNXT_ID PSN ACNT2 BMI1 SMEP AVX512F LZCNT
> 
> Running ODP appl: "odp_l2fwd"
> -
> IF-count:2
> Using IFs:   enp129s0f0 enp129s0f1
> Mode:PKTIN_DIRECT, PKTOUT_DIRECT
> 
> num worker threads: 32
> first CPU:  24
> cpu mask:   0x00
> 
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> 
> Pool info
> -
>   pool0
>   namepacket pool
>   pool type   packet
>   pool shm11
>   user area shm   0
>   num 8192
>   align   64
>   headroom128
>   seg len 8064
>   max data len65536
>   tailroom0
>   block size  8768
>   uarea size  0
>   shm size72143104
>   base addr   0x7f5fc1234000
>   uarea shm size  0
>   uarea base addr (nil)
> 
> pktio/socket_mmap.c:401:mmap_setup_ring():setsockopt(pkt mmap): Invalid
> argument
> pktio/socket_mmap.c:496:sock_mmap_close():mmap_unmap_sock() Invalid argument
> created pktio 1, dev: enp129s0f0, drv: socket
> Sharing 1 input queues between 16 workers
> Sharing 1 output queues between 16 workers
> created 1 input and 1 output queues on (enp129s0f0)
> pktio/socket_mmap.c:401:mmap_setup_ring():setsockopt(pkt mmap): Invalid
> argument
> pktio/socket_mmap.c:496:sock_mmap_close():mmap_unmap_sock() Invalid argument
> created pktio 2, dev: enp129s0f1, drv: socket
> Sharing 1 input queues between 16 workers
> Sharing 1 output queues between 16 workers
> created 1 input and 1 output queues on (enp129s0f1)
> 
> Queue binding (indexes)
> ---
> worker 0
>   rx: pktio 0, queue 0
>   tx: pktio 1, queue 0
> worker 1
>   rx: pktio 1, queue 0
>   tx: pktio 0, queue 0
> worker 2
>   rx: pktio 0, queue 0
>   tx: pktio 1, queue 0
> worker 3
>   rx: pktio 1, queue 0
>   tx: 

[lng-odp] issues with usage of mellanox 100G NICs with ODP & ODP-DPDK

[gyanesh patra]

Hi,
I am trying to run ODP & ODP-DPDK examples on our server with mellanox 100G
NICs. I am using the odp_l2fwd example. While running the example, I am
facing some issues.
-> When I run "ODP" example using the if names given by kernel as
arguments, I am not getting enough throughput.(the value is very low)
-> And when I try "ODP-DPDK" example using port ID as "0,1", it can't
create pktio. Whereas I am able to run the examples from "DPDK"
repo with portID "0,1" for the same mellanox NICs. I tried running with
"81:00.0,81:00.1" and also with if-names too without any success. Adding
the whitelist using ODP_PLATFORM_PARAMS doesn't help either.

Am I missing any steps to use mellanox NICs? OR is there a different method
to specify the device details to create pktio?
I am providing the output of "odp_l2fwd" examples for ODP and ODP-DPDK
repository here.

The NICs being used:

:81:00.0 'MT27700 Family [ConnectX-4]' if=enp129s0f0 drv=mlx5_core
unused=
:81:00.1 'MT27700 Family [ConnectX-4]' if=enp129s0f1 drv=mlx5_core
unused=

ODP l2fwd example run details:
--
root@ubuntu:/home/ubuntu/odp/test/performance# ./odp_l2fwd -i
enp129s0f0,enp129s0f1
HW time counter freq: 239886 <(239)%20999-9886> hz

_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishm.c:880:_odp_ishm_reserve():No huge pages, fall back to normal pages.
check: /proc/sys/vm/nr_hugepages.
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
 PKTIO: initialized loop interface.
 PKTIO: initialized pcap interface.
 PKTIO: initialized ipc interface.
 PKTIO: initialized socket mmap, use export ODP_PKTIO_DISABLE_SOCKET_MMAP=1
to disable.
 PKTIO: initialized socket mmsg,use export ODP_PKTIO_DISABLE_SOCKET_MMSG=1
to disable.
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory

ODP system info
---
ODP API version: 1.15.0
ODP impl name:   "odp-linux"
CPU model:   Intel(R) Xeon(R) CPU E5-2680 v4
CPU freq (hz):   33
Cache line size: 64
CPU count:   56


CPU features supported:
SSE3 PCLMULQDQ DTES64 MONITOR DS_CPL VMX SMX EIST TM2 SSSE3 FMA CMPXCHG16B
XTPR PDCM PCID DCA SSE4_1 SSE4_2 X2APIC MOVBE POPCNT TSC_DEADLINE AES XSAVE
OSXSAVE AVX F16C RDRAND FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR
PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE
DIGTEMP TRBOBST ARAT PLN ECMD PTM MPERF_APERF_MSR ENERGY_EFF FSGSBASE HLE
AVX2 BMI2 ERMS INVPCID RTM LAHF_SAHF SYSCALL XD 1GB_PG RDTSCP EM64T INVTSC

CPU features NOT supported:
CNXT_ID PSN ACNT2 BMI1 SMEP AVX512F LZCNT

Running ODP appl: "odp_l2fwd"
-
IF-count:2
Using IFs:   enp129s0f0 enp129s0f1
Mode:PKTIN_DIRECT, PKTOUT_DIRECT

num worker threads: 32
first CPU:  24
cpu mask:   0x00

_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory
_ishmphy.c:152:_odp_ishmphy_map():mmap failed:Cannot allocate memory

Pool info
-
  pool0
  namepacket pool
  pool type   packet
  pool shm11
  user area shm   0
  num 8192
  align   64
  headroom128
  seg len 8064
  max data len65536
  tailroom0
  block size  8768
  uarea size  0
  shm size72143104
  base addr   0x7f5fc1234000
  uarea shm size  0
  uarea base addr (nil)

pktio/socket_mmap.c:401:mmap_setup_ring():setsockopt(pkt mmap): Invalid
argument
pktio/socket_mmap.c:496:sock_mmap_close():mmap_unmap_sock() Invalid argument
created pktio 1, dev: enp129s0f0, drv: socket
Sharing 1 input queues between 16 workers
Sharing 1 output queues between 16 workers
created 1 input and 1 output queues on (enp129s0f0)
pktio/socket_mmap.c:401:mmap_setup_ring():setsockopt(pkt mmap): Invalid
argument
pktio/socket_mmap.c:496:sock_mmap_close():mmap_unmap_sock() Invalid argument
created pktio 2, dev: enp129s0f1, drv: socket
Sharing 1 input queues between 16 workers
Sharing 1 output queues between 16 workers
created 1 input and 1 output queues on (enp129s0f1)

Queue binding (indexes)
---
worker 0
  rx: pktio 0, queue 0
  tx: pktio 1, queue 0
worker 1
  rx: pktio 1, queue 0
  tx: pktio 0, queue 0
worker 2
  rx: pktio 0, queue 0
  tx: pktio 1, queue 0
worker 3
  rx: pktio 1, queue 0
  tx: pktio 0, queue 0
worker 4
  rx: pktio 0, queue 0
  tx: pktio 1, queue 0
worker 5
  rx: pktio 1, queue 0
  tx: pktio 0, queue 0
worker 6
  rx: pktio 0, queue 0
  tx: pktio 1, queue 0
worker 7
  rx: pktio 1, queue 0
  tx: pktio 0, queue 0
worker 8
  rx: pktio 0, queue 0
  tx: pktio 1, queue 0
worker 9
  rx: pktio 1, queue 0
  tx: pktio 0,