Hello, We are using OVS 2.12.0 with DPDK 18.11.2 release on Ubuntu 16.04 LTS. OVS process is taking ~25 second for bring up out of which EAL initialization itself is taking ~15 seconds.
Below is our hardware and memory configuration Hardware : Dell Server Sockets(NUMA) : 2 CPUs : Each NUMA with 10 CPUs (Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz) RAM : 128 GB Hugepages reserverd : 2048 huge pages of 2MB and 100 Huge pages of 1G (reserved during boot time). We are using only 2MB huge pages for OVS process. NIC : Ethernet 10G 2P X520 ( ixgbe driver ) Passing below EAL arguments with OVS process. 2020-07-23T09:46:58.878Z|00014|dpdk|INFO|EAL ARGS: ovs-vswitchd -w 0000:04:00.0,support-multi-driver=1 -w 0000:42:00.0,support-multi-driver=1 -c 0x1 --huge-dir /mnt/huge_ovs_2M --socket-mem 2048,2048 --socket-limit 2048,2048. We debugged more in DPDK and noticed that mmap to get virtual area is culprit and taking ~10 seconds for memory initialization. (rte_eal_init-> rte_eal_memory_init -> rte_eal_memseg_init -> memseg_primary_init -> alloc_va_space -> eal_get_virtual_area ->mmap ). We are observing increase in initialization time only when we are running with OVS + DPDK and do not see any issues with testpmd. Memory initialization is less than 1 second with DPDK 18.11.2 testpmd. Increase in initialization time is pretty high and noticeable difference on same set up with prior OVS release (OVS 2.8.2 with DPDK 17.11.3). Any pointers would be helpful. OVS vswitchd log 2020-07-23T09:14:23.228Z|00008|dpdk|INFO|Using DPDK 18.11.2 2020-07-23T09:14:23.228Z|00009|dpdk|INFO|DPDK Enabled - initializing... 2020-07-23T09:14:23.228Z|00010|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2020-07-23T09:14:23.228Z|00011|dpdk|INFO|IOMMU support for vhost-user-client disabled. 2020-07-23T09:14:23.228Z|00012|dpdk|INFO|POSTCOPY support for vhost-user-client disabled. 2020-07-23T09:14:23.228Z|00013|dpdk|INFO|Per port memory for DPDK devices disabled. 2020-07-23T09:14:23.228Z|00014|dpdk|INFO|EAL ARGS: ovs-vswitchd -w 0000:04:00.0,support-multi-driver=1 -w 0000:42:00.0,support-multi-driver=1 -c 0x1 --huge-dir /mnt/huge_ovs_2M --socket-mem 2048,2048 --socket-limit 2048,2048. 2020-07-23T09:14:23.233Z|00015|dpdk|INFO|EAL: Detected 40 lcore(s) 2020-07-23T09:14:23.233Z|00016|dpdk|INFO|EAL: Detected 2 NUMA nodes 2020-07-23T09:14:23.237Z|00017|dpdk|INFO|EAL: Multi-process socket /var/run/dpdk/rte/mp_socket 2020-07-23T09:14:23.291Z|00018|dpdk|INFO|EAL: 99 hugepages of size 1073741824 reserved, but no mounted hugetlbfs found for that size 2020-07-23T09:14:23.291Z|00019|dpdk|INFO|EAL: Probing VFIO support... 2020-07-23T09:14:23.291Z|00020|dpdk|INFO|EAL: VFIO support initialized 2020-07-23T09:14:33.898Z|00021|dpdk|INFO|EAL: PCI device 0000:04:00.0 on NUMA socket 0 2020-07-23T09:14:33.898Z|00022|dpdk|INFO|EAL: probe driver: 8086:154d net_ixgbe 2020-07-23T09:14:33.899Z|00023|dpdk|INFO|EAL: using IOMMU type 1 (Type 1) 2020-07-23T09:14:34.178Z|00024|dpdk|INFO|EAL: Ignore mapping IO port bar(2) 2020-07-23T09:14:34.525Z|00025|dpdk|INFO|EAL: PCI device 0000:42:00.0 on NUMA socket 1 2020-07-23T09:14:34.525Z|00026|dpdk|INFO|EAL: probe driver: 8086:154d net_ixgbe 2020-07-23T09:14:34.650Z|00027|dpdk|INFO|EAL: Ignore mapping IO port bar(2) 2020-07-23T09:14:35.001Z|00028|dpdk|INFO|DPDK Enabled - initialized Regards, Balaji
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
