Re: [vpp-dev] multiple worker thread failed with vfio

2018-03-11 Thread wuxp
i checked again and it's fixed after 18.04-rc0~374_g51c52c0~b3859
special thanks to Damjan Marion


[vpp-dev] multiple worker thread failed with vfio

2018-03-07 Thread wuxp
Hi,
when using config
unix { interactive cli-listen /run/vpp/cli.sock gid 0  } cpu { main-core 1 } 
dpdk {  dev :05:00.1 } *

* vpp# set int state TenGigabitEthernet5/0/1 up
vpp# set int ip addr TenGigabitEthernet5/0/1 192.168.1.1/24
vpp# ping 192.168.1.254 repeat 6 verbose
Source address: 192.168.1.1
64 bytes from 192.168.1.254: icmp_seq=3 ttl=64 time=.2041 ms *WORK AS EXPECTED

BUT:
* add corelist-workers 2,3 in config
unix { interactive cli-listen /run/vpp/cli.sock gid 0  } cpu { main-core 1 
*corelist-workers 2,3* } dpdk {  dev :05:00.1 }

ping *cant work,and kernel error:*
3月 08 11:52:09 h7 kernel: DMAR: DRHD: handling fault status reg 102
3月 08 11:52:09 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b738000
    DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:10 h7 kernel: DMAR: DRHD: handling fault status reg 202
3月 08 11:52:10 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b737000
    DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:11 h7 kernel: DMAR: DRHD: handling fault status reg 302
3月 08 11:52:11 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b736000
    DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:12 h7 kernel: DMAR: DRHD: handling fault status reg 402
3月 08 11:52:12 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b736000
    DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:13 h7 kernel: DMAR: DRHD: handling fault status reg 502
3月 08 11:52:13 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b735000
    DMAR:[fault reason 05] PTE Write access is not set
3月 08 11:52:14 h7 kernel: DMAR: DRHD: handling fault status reg 602
3月 08 11:52:14 h7 kernel: DMAR: DMAR:[DMA Write] Request device [05:00.1] fault 
addr 85b735000
    DMAR:[fault reason 05] PTE Write access is not set

enviroment:
DELL R430 with 2 cpus
centos 7.4 kernel 3.10.0-693.17.1.el7.x86_64
IOMMU kernel parameter: intel_iommu=on iommu=pt

*FULL LOG:*
vpp# set int state TenGigabitEthernet5/0/1 up
vpp# set int ip addr TenGigabitEthernet5/0/1 192.168.1.1/24
vpp# trace add dpdk-input 100
vpp# clear hardware
vpp# clear interfaces
vpp# clear error
vpp# clear run
vpp# ping 192.168.1.254 repeat 6 verbose 
Source address: 192.168.1.1  *( DMAR kernel error occur )*
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1
Source address: 192.168.1.1

Statistics: 6 sent, 0 received, 100% packet loss
vpp# show trace
--- Start of thread 0 vpp_main ---
No packets in trace buffer
--- Start of thread 1 vpp_wk_0 ---
Packet 1

01:45:58:568674: dpdk-input
  TenGigabitEthernet5/0/1 rx queue 0
  buffer 0xa4c2f: current data 0, length 60, free-list 0, clone-count 0, 
totlen-nifb 0, trace 0x0
  ext-hdr-valid
  l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
  PKT MBUF: port 0, nb_segs 1, pkt_len 60
    buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 
0x5b730c40
    packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    Packet Offload Flags
  PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
  PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
    Packet Types
  RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
  0x: 00:00:00:00:00:00 -> 00:00:00:00:00:00  
*(like ALL packet data is 0)*
01:45:58:568718: ethernet-input
  0x: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:58:568723: error-drop
  ethernet-input: l3 mac mismatch

Packet 2

01:45:59:568713: dpdk-input
  TenGigabitEthernet5/0/1 rx queue 0
  buffer 0xa4c08: current data 0, length 60, free-list 0, clone-count 0, 
totlen-nifb 0, trace 0x1
  ext-hdr-valid
  l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
  PKT MBUF: port 0, nb_segs 1, pkt_len 60
    buf_len 2176, data_len 60, ol_flags 0x180, data_off 128, phys_addr 
0x5b730280
    packet_type 0x1 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    Packet Offload Flags
  PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
  PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
    Packet Types
  RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
  0x: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:59:568714: ethernet-input
  0x: 00:00:00:00:00:00 -> 00:00:00:00:00:00
01:45:59:568716: error-drop
  ethernet-input: l3 mac mismatch

Packet 3

01:46:00:573520: dpdk-input
  TenGigabitEthernet5/0/1 rx queue 0
  buffer 0xa4be1: current data 0, length 60, free-list 0, clone-count 0, 
totlen-nifb 0, trace 0x2
  ext-hdr-valid
  l4-cksum-computed l4-cksum-correct l2-hdr-offset 0
  PKT MBUF: port 0,