Thank you for your reply. I am trying to use the latest version of pktgen-dpdk, ovs, and dpdk. But I meet some problem when setting up the platform.
The version of my platform: 1. OVS: 2.11.1 2. DPDK: 18.11 3. pktgen-dpdk: 3.6.6 Using the previous version (3.4.9), pktgen-dpdk works well. This time it shows up some error: ##############################error log################################## Pktgen:/> PANIC in pktgen_main_rxtx_loop(): *** port 0 socket ID 4294967295 has different socket ID for lcore 5 socket ID 0 7: [/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc5d6a0288f]] 6: [/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc5d6cd96db]] 5: [./app/x86_64-native-linuxapp-gcc/pktgen(eal_thread_loop+0x1e1) [0x57c641]] 4: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_launch_one_lcore+0xb7) [0x49d247]] 3: [./app/x86_64-native-linuxapp-gcc/pktgen() [0x49bef5]] 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3) [0x46352c]] 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b) [0x5829fb]] Aborted (core dumped) ##############################error log END################################## When I look up the log of OVS, I find: ##############################OVS error log################################## 2019-06-24T09:11:13.780Z|00113|dpdk|ERR|VHOST_CONFIG: recvmsg failed 2019-06-24T09:11:13.780Z|00114|dpdk|INFO|VHOST_CONFIG: vhost peer closed 2019-06-24T09:11:13.781Z|00115|netdev_dpdk|INFO|vHost Device '/usr/local/var/run/openvswitch/vhost-user1' has been removed 2019-06-24T09:11:13.781Z|00116|dpdk|ERR|VHOST_CONFIG: recvmsg failed 2019-06-24T09:11:13.781Z|00117|dpdk|INFO|VHOST_CONFIG: vhost peer closed 2019-06-24T09:11:13.781Z|00118|netdev_dpdk|INFO|vHost Device '/usr/local/var/run/openvswitch/vhost-user2' has been removed ##############################OVS error log############################################# Could you share some hints to me, thank you for your time. Best wishes Xia Rui At 2019-06-23 21:26:15, "Wiles, Keith" <[email protected]> wrote: > > >> On Jun 23, 2019, at 3:00 AM, Xia Rui <[email protected]> wrote: >> >> Hello, everyone. >> I am using pktgen-dpdk and testpmd to test the functionality of ovs-dpdk. >> The network topology is : >> +-------------+----------------------+ host(OVS-DPDK) >> +-----------------------+-----------------+ >> | | vhost-user port 1 >> |<----------------------------------->| vhost-user port 3 | >> | >> | +----------------------+ >> +-----------------------+ | >> | container | pktgen | >> | testpmd | container | >> | +----------------------+ >> +-------------------+ | >> | | vhost-user port 2 >> |<------------------------------------>| vhost-user port 4 | >> | >> +--------------+---------------------+ >> +----------------------+----------------+ >> >> The version of my platform: >> 1. host OS: ubuntu 16.04.5 LTS >> 2. host linux kernel: 4.15.0-15 >> 3. host OVS: 2.8.0 >> 4. host DPDK : 17.05.2 >> 5. container pktgen-dpdk: 3.4.9 + DPDK 17.05.2 >> 6. container DPDK (testpmd): 17.05.2 > >At one point virtio drivers were changing the length of the tx packet in the >mbuf for some type of meta data being shared between the two virtio points. >Pktgen expected the length to remain the same when the mbuf was returned to >pktgen mempool. This explains the non-64 byte frames. The fix was added to >virtio in later releases to restore the length in the mbuf, plus I may’ve >added code to pktgen in later release to fix the length back to what I >expected when sending the packets. I looked in the latests version of pktgen >and found the code in pktgen_setup_cb() which reset the length in the packet. >The code should call rte_pktmbuf_reset() routine to restore the mbuf to >expected state. > >If OVS can work with a later version of DPDK then I would upgrade to the >latest release, if not then I would look at and compare the two versions of >virtio PMD and see if you can find that fix. If not then you can look in >pktgen and repair the length in the mbuf before it is sent plus you may have >to fix the location of the read/write offsets in the mbuf as well. This change >will effect pktgen performance, but for virtio that should not be a problem. >You can also connect the Maintainers of virtio and see if they remember the >fix. > >For the traffic stopping, I do not remember if this fix solved that problem. >> >> There are two docker containers. One is running pktgen-dpdk with: >> ############################pktgen-dpdk start >> script############################ >> ./app/x86_64-native-linuxapp-gcc/pktgen -c 0x70 --master-lcore 4 -n 1 >> --file-prefix pktgen --no-pci \ >> --vdev >> 'net_virtio_user1,mac=00:00:00:00:00:01,path=/var/run/openvswitch/vhost-user1' >> \ >> --vdev >> 'net_virtio_user2,mac=00:00:00:00:00:02,path=/var/run/openvswitch/vhost-user2' >> \ >> -- -T -P -m "5.0,6.1" >> ############################pktgen-dpdk start script >> END############################ >> >> The other is running testpmd with: >> ############################testpmd start script############################ >> testpmd -c 0xE0 -n 1 --socket-mem=1024,0 --file-prefix testpmd --no-pci \ >> --vdev >> 'net_virtio_user3,mac=00:00:00:00:00:03,path=/var/run/openvswitch/vhost-user3' >> \ >> --vdev >> 'net_virtio_user4,mac=00:00:00:00:00:04,path=/var/run/openvswitch/vhost-user4' >> \ >> -- -i --burst=64 --disable-hw-vlan --txd=2048 --rxd=2048 --auto-start >> --coremask=0xc0 >> ############################testpmd start script >> END############################ >> >> The two containers are connected using ovs-docker in the host. I create four >> vhost-user ports, two of which are for pktgen, the other of which are for >> testpmd. I connect the vhost-user ports by adding the >> routes between them. The start script of ovs is: >> >> ############################ovs-dpdk start script############################ >> sudo ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ >> --remote=db:Open_vSwitch,Open_vSwitch,manager_options \ >> --private-key=db:Open_vSwitch,SSL,private_key \ >> --certificate=db:Open_vSwitch,SSL,certificate \ >> --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \ >> --pidfile --detach >> >> sudo ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true >> >> sudo ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0x02 >> >> sudo ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x04 >> >> sudo ovs-vswitchd --pidfile --detach >> --log-file=/var/log/openvswitch/vhost-ovs-vswitchd.log >> >> sudo /usr/local/share/openvswitch/scripts/ovs-ctl --no-ovsdb-server >> --db-sock="$DB_SOCK" start >> ############################ovs-dpdk start script >> END############################ >> >> When I start the pktgen-dpdk to send packets, there are something wrong. >> >> First, I set the packet size to 64Bytes, there are some big packets (more >> than 64Bytes). I set the rate to 10%, 300 packets, and get: >> >> ###########################CMD shot####################### >> Ports 0-1 of 2 <Main Page> Copyright (c) <2010-2017>, Intel Corporation >> Flags:Port : P--------------:0 P--------------:1 >> Link State : <UP-10000-FD> <UP-10000-FD> >> ----TotalRate---- >> Pkts/s Max/Rx : 256/0 300/0 >> 556/0 >> Max/Tx : 300/0 300/0 >> 600/0 >> MBits/s Rx/Tx : 0/0 0/0 >> 0/0 >> Broadcast : 0 0 >> Multicast : 0 0 >> 64 Bytes : 1104 1148 >> 65-127 : 64 152 >> 128-255 : 0 0 >> 256-511 : 0 0 >> 512-1023 : 0 0 >> 1024-1518 : 0 0 >> Runts/Jumbos : 0/0 0/0 >> Errors Rx/Tx : 0/0 0/0 >> Total Rx Pkts : 1168 1300 >> Tx Pkts : 1300 1300 >> Rx MBs : 0 0 >> Tx MBs : 1 1 >> ARP/ICMP Pkts : 0/0 0/0 >> : >> Pattern Type : abcd... abcd... >> Tx Count/% Rate : 300 /10% 300 /10% >> PktSize/Tx Burst : 64 / 64 64 / 64 >> Src/Dest Port : 1234 / 5678 1234 / 5678 >> Pkt Type:VLAN ID : IPv4 / TCP:0001 IPv4 / TCP:0001 >> 802.1p CoS : 0 0 >> ToS Value: : 0 0 >> - DSCP value : 0 0 >> - IPP value : 0 0 >> Dst IP Address : 192.168.1.1 192.168.0.1 >> Src IP Address : 192.168.0.1/24 192.168.1.1/24 >> Dst MAC Address : 00:00:00:00:00:02 00:00:00:00:00:01 >> Src MAC Address : 00:00:00:00:00:01 00:00:00:00:00:02 >> VendID/PCI Addr : 0000:0000/00:00.0 0000:0000/00:00.0 >> >> -- Pktgen Ver: 3.4.9 (DPDK 17.05.2) Powered by DPDK >> -------------------------- >> ###########################CMD shot END####################### >> >> There ought to be no packets greater than 64Bytes, but there exist. >> >> Second, I reset the configuration ("rst") and try to start send packets >> continuously. However, the pktgen works few seconds and stop sending >> packets, with output: >> >> >> ###########################CMD shot####################### >> Ports 0-1 of 2 <Main Page> Copyright (c) <2010-2017>, Intel Corporation >> Flags:Port : P--------------:0 P--------------:1 >> Link State : <UP-10000-FD> <UP-10000-FD> >> ----TotalRate---- >> Pkts/s Max/Rx : 176288/0 146016/0 >> 308224/0 >> Max/Tx : 1344832/0 767520/0 >> 1535040/0 >> MBits/s Rx/Tx : 0/0 0/0 >> 0/0 >> Broadcast : 0 0 >> Multicast : 0 0 >> 64 Bytes : 15872 15104 >> 65-127 : 50368 61248 >> 128-255 : 44096 62848 >> 256-511 : 51840 93216 >> 512-1023 : 63264 151456 >> 1024-1518 : 51936 126240 >> Runts/Jumbos : 0/0 0/0 >> Errors Rx/Tx : 0/0 0/0 >> Total Rx Pkts : 277376 510112 >> Tx Pkts : 4529248 1162368 >> Rx MBs : 1215 2701 >> Tx MBs : 665276 57380 >> ARP/ICMP Pkts : 0/0 0/0 >> : >> Pattern Type : abcd... abcd... >> Tx Count/% Rate : Forever /100% Forever /100% >> PktSize/Tx Burst : 64 / 64 64 / 64 >> Src/Dest Port : 1234 / 5678 1234 / 5678 >> Pkt Type:VLAN ID : IPv4 / TCP:0001 IPv4 / TCP:0001 >> 802.1p CoS : 0 0 >> ToS Value: : 0 0 >> - DSCP value : 0 0 >> - IPP value : 0 0 >> Dst IP Address : 192.168.1.1 192.168.0.1 >> Src IP Address : 192.168.0.1/24 192.168.1.1/24 >> Dst MAC Address : 00:00:00:00:00:02 00:00:00:00:00:01 >> Src MAC Address : 00:00:00:00:00:01 00:00:00:00:00:02 >> VendID/PCI Addr : 0000:0000/00:00.0 0000:0000/00:00.0 >> >> -- Pktgen Ver: 3.4.9 (DPDK 17.05.2) Powered by DPDK >> -------------------------- >> ###########################CMD shot END####################### >> >> Pktgen is stuck at this setting. There are a lot of large packets! >> >> I check the log of ovs-dpdk and get: >> >> #############################ovs-dpdk log############################# >> 2019-06-23T07:51:39.349Z|00022|netdev_dpdk(pmd8)|WARN|Dropped 5803564 log >> messages in last 102581 seconds (most recently, 102576 seconds ago) due to >> excessive rate >> 2019-06-23T07:51:39.349Z|00023|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00024|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00025|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00026|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00027|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00028|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00029|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00030|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00031|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00032|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00033|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00034|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00035|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00036|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00037|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00038|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00039|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00040|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00041|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> 2019-06-23T07:51:39.349Z|00042|netdev_dpdk(pmd8)|WARN|vhost-user3: Too big >> size 1524 max_packet_len 1518 >> #############################ovs-dpdk log END############################# >> Thank you for share your ideas. >> >> >> Best wishes, >> Xia Rui > >Regards, >Keith >
