> On Aug 1, 2017, at 3:15 AM, Paul Tsvika <[email protected]> wrote: > > Hi zheng, > > Thanks for the feedback. > > You are right. To get pktgen work with mlx4, it needs to recompile DPDK and > pktgen as well. > > I am able to run pktgen now. > > By following the document from DPDK website for this NIC, > I did and change the following variables > > I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc. > set l*og_num_mgm_entry_size=-7* > > Disable pause frames: > *ethtool -A ens2d1 rx off tx 0ff* > > I did not change the NIC register ( *setpci* ) at this time. > > > and then I run the commands below in order: > > > *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib* > *test$: /etc/init.d/openibd restart* > *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 > --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b > 05:00.1 -- -P -m "[1-5.0]"* > > | Ports 0-1 of 2 <Main Page> Copyright (c) <2010-2017>, Intel Corporation > Flags:Port : P--------------:0 > Link State : <--Down--> ----TotalRate---- > Pkts/s Max/Rx : 0/0 0/0 > Max/Tx : 844160/732319 844160/732319 > MBits/s Rx/Tx : 0/*492 * 0/*492*
Note the link state is down and it needs to be up to determine the link speed for Pktgen. > Broadcast : 0 > Multicast : 0 > 64 Bytes : 0 > 65-127 : 0 > 128-255 : 0 > 256-511 : 0 > 512-1023 : 0 > 1024-1518 : 0 > Runts/Jumbos : 0/0 > Errors Rx/Tx : 0/0 > Total Rx Pkts : 0 > Tx Pkts : 59289215 > Rx MBs : 0 > Tx MBs : 39842 > ARP/ICMP Pkts : 0/0 > Tx Overrun : 0 > Cycles per Tx : 14880952/3200 > Missed Rx : 0 > mcasts Rx : None > No Mbuf Rx : 0 > : > Pattern Type : abcd... > Tx Count/% Rate : Forever /100% > PktSize/Tx Burst : 64 / 32 > Src/Dest Port : 1234 / 5678 > Pkt Type:VLAN ID : IPv4 / TCP:0001 > Dst IP Address : 192.168.1.1 > Src IP Address : 192.168.0.1/24 > Src MAC Address : e4:1d:2d:02:44:81 > Powered by DPDK -------------------------- > > The transmitting rate should be around 40000 since it is 40G NIC. > However, the rate is much lower than it and it is probably related to > configurations. > > I actually have two mother boards and each one has 40G NICs installed. > My intention is to do the tx / rx experiment. > My first step is to check tx rate can reach as close to 40000 as possible. > > Anyone has related experience please advise. > > > > P.T > > > 2017-08-01 12:15 GMT+08:00 zhilong zheng <[email protected]>: > >> >> Hi, >> >> After enabling the MLX4 flag and recompiling the DPDK source code, Do you >> recompile the code of pktgen? I think you can recompile it by running make >> clean and make. >> >> Regards, >> Zhilong >> >>> 在 2017年8月1日，11:38，Paul Tsvika <[email protected]> 写道： >>> >>> Hi Olga, >>> >>> 2017-08-01 0:11 GMT+08:00 Olga Shern <[email protected]>: >>> >>>> Hi Paul, >>>> >>>> Looks like your ConnectX-3 configured to be IB and not ETH >>>> >>>>> pci@0000:04:00.0 ib0 network MT27500 Family [ConnectX-3] >>>> >>>> Use connectx_port_config script. >>>> Set all the ports to Ethernet, and follow the instructions on the >> screen. >>>> >>> >>> Thanks. I followed the script and configured both ports to Ethernet. >>> >>> ibstat info below: >>> >>> CA 'mlx4_0' >>> CA type: MT4099 >>> Number of ports: 2 >>> Firmware version: 2.40.7000 >>> Hardware version: 1 >>> Node GUID: 0xe41d2d0300024480 >>> System image GUID: 0xe41d2d0300024483 >>> Port 1: >>> State: Down >>> Physical state: Disabled >>> Rate: 10 >>> Base lid: 0 >>> LMC: 0 >>> SM lid: 0 >>> Capability mask: 0x04010000 >>> Port GUID: 0xe61d2dfffe024481 >>> Link layer: Ethernet >>> Port 2: >>> State: Active >>> Physical state: LinkUp >>> Rate: 40 >>> Base lid: 0 >>> LMC: 0 >>> SM lid: 0 >>> Capability mask: 0x04010000 >>> Port GUID: 0xe61d2dfffe024482 >>> Link layer: Ethernet >>> >>> I also change the default value from n to CONFIG_RTE_LIBRTE_MLX4_PMD=y. >>> Recompiled DPDK and it all worked. >>> >>> And Below is my ifconfig info: >>> >>> ens2 Link encap:Ethernet HWaddr e4:1d:2d:02:44:81 >>> UP BROADCAST MULTICAST MTU:1500 Metric:1 >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> collisions:0 txqueuelen:1000 >>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >>> >>> ens2d1 Link encap:Ethernet HWaddr e4:1d:2d:02:44:82 >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> RX packets:126 errors:0 dropped:0 overruns:0 frame:0 >>> TX packets:220 errors:0 dropped:0 overruns:0 carrier:0 >>> collisions:0 txqueuelen:1000 >>> RX bytes:29494 (29.4 KB) TX bytes:38063 (38.0 KB) >>> >>> >>> Since I assume that I don't have to bind the device manually, I executed >>> ./dpdk-devbind.py --status with the following info popped out: >>> >>> Network devices using kernel driver >>> =================================== >>> 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3 >>> drv=ixgbe unused= >>> 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4 >>> drv=ixgbe unused= >>> 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1 >>> drv=mlx4_core unused= >>> 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb >> unused= >>> 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb >> unused= >>> *Active* >>> >>> >>> With these configurations, I run commands in order but still had the same >>> result: >>> >>> $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib >>> $test: /etc/init.d/openibd restart >>> $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1 >> --proc-type >>> auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P >> -m >>> "[1-5.0]" >>> >>> >>>>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf >> cache >>> 1024 >>> !PANIC!: *** Did not find any ports to use *** >>> PANIC in pktgen_config_ports(): >>> *** Did not find any ports to use ***6: >>> [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29) >>> [0x447239]] >>> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) >>> [0x7f0ec0931830]] >>> 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]] >>> 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614) >>> [0x46e324]] >>> 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) >> [0x43ead4]] >>> 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) >> [0x4b7ecb]] >>> Aborted (core dumped) >>> >>> >>> >>> Is there anything I've done wrong here? >>> >>> >>> Thanks >>> >>> >>> P.T >>> >>> >>>> You can use -w to specify ConnectX-3 PCI address, you don't need to >> use -b >>>> >>>> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default >>>> >>>> Best Regards, >>>> Olga >>>> >>>> >>>> >>> >>> -- >>> P.T >> > > > > -- > P.T Regards, Keith
