Hello, I have installed ovs-dpdk in two ways，but both failed when I added a dpdk type port to the netdev type bridge.
Here is my environment and problem: * OS: ubuntu16.04 * NIC: Intel XL710 for 40GbE QSFP+ For the first time I installed ovs & dpdk using apt install. It installed ovs 2.5.4 and dpdk 2.2. I have bind the nic with vfio-pci $ dpdk_nic_bind --status Network devices using DPDK-compatible driver ============================================ 0000:81:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' drv=vfio-pci unused=i40e 0000:81:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=vfio-pci unused=i40e Network devices using kernel driver =================================== 0000:02:00.0 'I350 Gigabit Network Connection' if=enp2s0f0 drv=igb unused=vfio-pci 0000:02:00.1 'I350 Gigabit Network Connection' if=enp2s0f1 drv=igb unused=vfio-pci 0000:02:00.2 'I350 Gigabit Network Connection' if=enp2s0f2 drv=igb unused=vfio-pci 0000:02:00.3 'I350 Gigabit Network Connection' if=enp2s0f3 drv=igb unused=vfio-pci *Active* Other network devices ===================== The VT-d has enabled. $ dmesg | grep -e DMAR -e IOMMU [ 0.000000] ACPI: DMAR 0x000000007B7BD000 000118 (v01 INSYDE HSW-LPT 00000001 ACPI 00040000) [ 0.000000] DMAR: IOMMU enabled [ 0.052125] DMAR: Host address width 46 The vfio-pci module has alse been modprobed. $ lsmod | grep io vfio_pci 40960 2 vfio_virqfd 16384 1 vfio_pci vfio_iommu_type1 24576 1 vfio 28672 7 vfio_iommu_type1,vfio_pci irqbypass 16384 4 kvm,vfio_pci The dpdk-init option has been set to true. $ ovs-vsctl get Open_vSwitch . other_config:dpdk-init "true" Here is my hugepage config: $ grep HugePages_ /proc/meminfo HugePages_Total: 8 HugePages_Free: 6 HugePages_Rsvd: 0 HugePages_Surp: 0 I successfully created a userspace bridge, but when I add the dpdk type port $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev $ ovs-vsctl add-port br0 port3 -- set Interface port3 type=dpdk options:dpdk-devargs=0000:81:00.0 ovs-vsctl: Error detected while setting up 'port3'. See ovs-vswitchd log for details. Here is the ovs-vswitchd log: bridge|WARN|could not open network device port3 (No such device) I thought at first that it was caused by a version issue of ovs-dpdk, so for the second time I rebuilt ovs of 2.9.1 and dpdk of 17.11.2 from source. The other configurations are the same as before, but when I add the dpdk type port: $ ovs-vsctl add-port br0 port2 -- set Interface port2 type=dpdk options:dpdk-devargs=0000:81:00.0 ovs-vsctl: Error detected while setting up 'port2': Error attaching device '0000:81:00.1' to DPDK. See ovs-vswitchd log for det ails. Here is the ovs-vswitchd log: 2018-05-30T08:12:55Z|00001|ovs_numa|INFO|Discovered 12 CPU cores on NUMA node 0 2018-05-30T08:12:55Z|00002|ovs_numa|INFO|Discovered 12 CPU cores on NUMA node 1 2018-05-30T08:12:55Z|00003|ovs_numa|INFO|Discovered 2 NUMA nodes and 24 CPU cores 2018-05-30T08:12:55Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connecting... 2018-05-30T08:12:55Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: connected 2018-05-30T08:12:55Z|00006|dpdk|INFO|Using DPDK 17.11.2 2018-05-30T08:12:55Z|00007|dpdk|INFO|DPDK Enabled - initializing... 2018-05-30T08:12:55Z|00008|dpdk|INFO|No vhost-sock-dir provided - defaulting to /usr/local/var/run/openvswitch 2018-05-30T08:12:55Z|00009|dpdk|INFO|IOMMU support for vhost-user-client disabled. 2018-05-30T08:12:55Z|00010|dpdk|INFO|EAL ARGS: ovs-vswitchd -c 0x2 --socket-mem 1024,0 2018-05-30T08:12:55Z|00011|dpdk|INFO|EAL: Detected 24 lcore(s) 2018-05-30T08:12:55Z|00012|dpdk|INFO|EAL: Probing VFIO support... 2018-05-30T08:12:55Z|00013|dpdk|INFO|EAL: VFIO support initialized 2018-05-30T08:12:58Z|00014|dpdk|INFO|EAL: PCI device 0000:02:00.0 on NUMA socket 0 2018-05-30T08:12:58Z|00015|dpdk|INFO|EAL: probe driver: 8086:1521 net_e1000_igb 2018-05-30T08:12:58Z|00016|dpdk|INFO|EAL: PCI device 0000:02:00.1 on NUMA socket 0 2018-05-30T08:12:58Z|00017|dpdk|INFO|EAL: probe driver: 8086:1521 net_e1000_igb 2018-05-30T08:12:58Z|00018|dpdk|INFO|EAL: PCI device 0000:02:00.2 on NUMA socket 0 2018-05-30T08:12:58Z|00019|dpdk|INFO|EAL: probe driver: 8086:1521 net_e1000_igb 2018-05-30T08:12:58Z|00020|dpdk|INFO|EAL: PCI device 0000:02:00.3 on NUMA socket 0 2018-05-30T08:12:58Z|00021|dpdk|INFO|EAL: probe driver: 8086:1521 net_e1000_igb 2018-05-30T08:12:58Z|00022|dpdk|INFO|EAL: PCI device 0000:81:00.0 on NUMA socket 1 2018-05-30T08:12:58Z|00023|dpdk|INFO|EAL: probe driver: 8086:1583 net_i40e 2018-05-30T08:12:58Z|00024|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2018-05-30T08:12:58Z|00025|dpdk|INFO|EAL: Releasing pci mapped resource for 0000:81:00.0 2018-05-30T08:12:58Z|00026|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e80000000 2018-05-30T08:12:58Z|00027|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e80800000 2018-05-30T08:12:58Z|00028|dpdk|ERR|EAL: Requested device 0000:81:00.0 cannot be used 2018-05-30T08:12:58Z|00029|dpdk|INFO|EAL: PCI device 0000:81:00.1 on NUMA socket 1 2018-05-30T08:12:58Z|00030|dpdk|INFO|EAL: probe driver: 8086:1583 net_i40e 2018-05-30T08:12:58Z|00031|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2018-05-30T08:12:58Z|00032|dpdk|INFO|EAL: Releasing pci mapped resource for 0000:81:00.1 2018-05-30T08:12:58Z|00033|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.1 at 0x7f3e80808000 2018-05-30T08:12:58Z|00034|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.1 at 0x7f3e81008000 2018-05-30T08:12:58Z|00035|dpdk|ERR|EAL: Requested device 0000:81:00.1 cannot be used Zone 0: name:<rte_eth_dev_data>, IO:0x7f3e7ffb64c0, len:0x34900, virt:0x7f3e7ffb64c0, socket_id:0, flags:0 2018-05-30T08:12:58Z|00036|dpdk|INFO|DPDK Enabled - initialized 2018-05-30T08:12:58Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports recirculation 2018-05-30T08:12:58Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1 2018-05-30T08:12:58Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3 2018-05-30T08:12:58Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports truncate action 2018-05-30T08:12:58Z|00041|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports unique flow ids 2018-05-30T08:12:58Z|00042|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports clone action 2018-05-30T08:12:58Z|00043|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10 2018-05-30T08:12:58Z|00044|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports eventmask in conntrack action 2018-05-30T08:12:58Z|00045|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_clear action 2018-05-30T08:12:58Z|00046|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state 2018-05-30T08:12:58Z|00047|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_zone 2018-05-30T08:12:58Z|00048|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_mark 2018-05-30T08:12:58Z|00049|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_label 2018-05-30T08:12:58Z|00050|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_state_nat 2018-05-30T08:12:58Z|00051|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple 2018-05-30T08:12:58Z|00052|ofproto_dpif|INFO|netdev@ovs-netdev: Datapath supports ct_orig_tuple6 2018-05-30T08:12:58Z|00053|dpdk|INFO|EAL: PCI device 0000:81:00.0 on NUMA socket 1 2018-05-30T08:12:58Z|00054|dpdk|INFO|EAL: probe driver: 8086:1583 net_i40e 2018-05-30T08:12:58Z|00055|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2018-05-30T08:12:59Z|00056|dpdk|INFO|EAL: Releasing pci mapped resource for 0000:81:00.0 2018-05-30T08:12:59Z|00057|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e81010000 2018-05-30T08:12:59Z|00058|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e81810000 2018-05-30T08:12:59Z|00059|dpdk|ERR|EAL: Driver cannot attach the device (0000:81:00.0) 2018-05-30T08:12:59Z|00060|netdev_dpdk|WARN|Error attaching device '0000:81:00.0' to DPDK 2018-05-30T08:12:59Z|00061|netdev|WARN|port1: could not set configuration (Invalid argument) 2018-05-30T08:12:59Z|00062|bridge|INFO|bridge br0: added interface br0 on port 65534 2018-05-30T08:12:59Z|00063|bridge|INFO|bridge br0: using datapath ID 000016adc92d614d 2018-05-30T08:12:59Z|00064|connmgr|INFO|br0: added service controller "punix:/usr/local/var/run/openvswitch/br0.mgmt" 2018-05-30T08:12:59Z|00065|timeval|WARN|Unreasonably long 3600ms poll interval (12ms user, 2386ms system) 2018-05-30T08:12:59Z|00066|timeval|WARN|faults: 1588 minor, 0 major 2018-05-30T08:12:59Z|00067|timeval|WARN|context switches: 18 voluntary, 16 involuntary 2018-05-30T08:12:59Z|00068|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=539f6b57: 2018-05-30T08:12:59Z|00069|coverage|INFO|bridge_reconfigure 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00070|coverage|INFO|ofproto_flush 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00071|coverage|INFO|ofproto_update_port 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00072|coverage|INFO|rev_flow_table 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00073|coverage|INFO|cmap_expand 0.0/sec 0.000/sec 0.0000/sec total: 45 2018-05-30T08:12:59Z|00074|coverage|INFO|cmap_shrink 0.0/sec 0.000/sec 0.0000/sec total: 29 2018-05-30T08:12:59Z|00075|coverage|INFO|dpif_port_add 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00076|coverage|INFO|dpif_flow_flush 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00077|coverage|INFO|dpif_flow_get 0.0/sec 0.000/sec 0.0000/sec total: 21 2018-05-30T08:12:59Z|00078|coverage|INFO|dpif_flow_put 0.0/sec 0.000/sec 0.0000/sec total: 22 2018-05-30T08:12:59Z|00079|coverage|INFO|dpif_flow_del 0.0/sec 0.000/sec 0.0000/sec total: 21 2018-05-30T08:12:59Z|00080|coverage|INFO|dpif_execute 0.0/sec 0.000/sec 0.0000/sec total: 4 2018-05-30T08:12:59Z|00081|coverage|INFO|flow_extract 0.0/sec 0.000/sec 0.0000/sec total: 3 2018-05-30T08:12:59Z|00082|coverage|INFO|miniflow_malloc 0.0/sec 0.000/sec 0.0000/sec total: 25 2018-05-30T08:12:59Z|00083|coverage|INFO|hmap_expand 0.0/sec 0.000/sec 0.0000/sec total: 436 2018-05-30T08:12:59Z|00084|coverage|INFO|netdev_get_stats 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00085|coverage|INFO|txn_unchanged 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00086|coverage|INFO|txn_incomplete 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00087|coverage|INFO|poll_create_node 0.0/sec 0.000/sec 0.0000/sec total: 52 2018-05-30T08:12:59Z|00088|coverage|INFO|poll_zero_timeout 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00089|coverage|INFO|seq_change 0.0/sec 0.000/sec 0.0000/sec total: 632 2018-05-30T08:12:59Z|00090|coverage|INFO|pstream_open 0.0/sec 0.000/sec 0.0000/sec total: 3 2018-05-30T08:12:59Z|00091|coverage|INFO|stream_open 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00092|coverage|INFO|util_xalloc 0.0/sec 0.000/sec 0.0000/sec total: 9470 2018-05-30T08:12:59Z|00093|coverage|INFO|netdev_set_policing 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00094|coverage|INFO|netdev_get_ifindex 0.0/sec 0.000/sec 0.0000/sec total: 1 2018-05-30T08:12:59Z|00095|coverage|INFO|netdev_get_hwaddr 0.0/sec 0.000/sec 0.0000/sec total: 5 2018-05-30T08:12:59Z|00096|coverage|INFO|netdev_get_ethtool 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00097|coverage|INFO|netlink_received 0.0/sec 0.000/sec 0.0000/sec total: 19 2018-05-30T08:12:59Z|00098|coverage|INFO|netlink_recv_jumbo 0.0/sec 0.000/sec 0.0000/sec total: 2 2018-05-30T08:12:59Z|00099|coverage|INFO|netlink_sent 0.0/sec 0.000/sec 0.0000/sec total: 14 2018-05-30T08:12:59Z|00100|coverage|INFO|73 events never hit 2018-05-30T08:12:59Z|00101|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (<->/usr/local/var/run/openvswitch/db.sock) at lib/str eam-fd.c:157 (66% CPU usage)2018-05-30T08:12:59Z|00102|poll_loop|INFO|wakeup due to [POLLIN] on fd 9 (NETLINK_ROUTE<->NETLINK_ROUTE) at lib/netlink-socket.c :1331 (66% CPU usage)2018-05-30T08:12:59Z|00103|dpdk|INFO|EAL: PCI device 0000:81:00.0 on NUMA socket 1 2018-05-30T08:12:59Z|00104|dpdk|INFO|EAL: probe driver: 8086:1583 net_i40e 2018-05-30T08:12:59Z|00105|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2018-05-30T08:12:59Z|00106|dpdk|INFO|EAL: Releasing pci mapped resource for 0000:81:00.0 2018-05-30T08:12:59Z|00107|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e81818000 2018-05-30T08:12:59Z|00108|dpdk|INFO|EAL: Calling pci_unmap_resource for 0000:81:00.0 at 0x7f3e82018000 2018-05-30T08:12:59Z|00109|dpdk|ERR|EAL: Driver cannot attach the device (0000:81:00.0) 2018-05-30T08:12:59Z|00110|netdev_dpdk|WARN|Error attaching device '0000:81:00.0' to DPDK 2018-05-30T08:12:59Z|00111|netdev|WARN|port1: could not set configuration (Invalid argument) Is there something wrong with NIC XL710? What would cause such a situation?
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
