> > I have ran into a ovs-vswitchd SEGV while trying to use the vhost-user > feature with the latest dpdk. I have attempted to provide as much detail as I > can. > > versions compiled from source on CentOS 7.1 x86_64: > - openvswitch 2.4.0 - http://openvswitch.org/releases/openvswitch- > 2.4.0.tar.gz > - dpdk 2.1.0 - http://dpdk.org/browse/dpdk/snapshot/dpdk-2.1.0.tar.gz > - qemu - 2.4.0.1 - http://wiki.qemu-project.org/download/qemu- > 2.4.0.1.tar.bz2 > > > Steps to reproduce: > > - Build and Install following these instructions > http://wiki.qemu.org/Features/vhost-user-ovs-dpdk > - setup details: > > hugepage config: > GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/root rd.lvm.lv=centos/swap > crashkernel=auto rhgb quiet iommu=pt intel_iommu=on > default_hugepagesz=1G hugepagesz=1G hugepages=8 > isolcpus=1,2,3,4,5,12,13,14,15,16,17,7,8,9,10,11,18,19,20,21,22,23 > transparent_hugepage=never" > > # export DPDK_DIR=/root/dpdk/dpdk-2.1.0 > # export DPDK_BUILD=$DPDK_DIR/x86_64-native-linuxapp-gcc/ > # export DB_SOCK=/usr/local/var/run/openvswitch/db.sock > > > # modprobe openvswitch > # modprobe uio > # insmod /root/dpdk/dpdk-2.1.0/x86_64-native-linuxapp- > gcc/kmod/igb_uio.ko > # insmod /root/dpdk/dpdk- > 2.1.0/lib/librte_vhost/eventfd_link/eventfd_link.ko > > # /root/dpdk/dpdk-2.1.0/tools/dpdk_nic_bind.py --bind=igb_uio ens1f0 > # /root/dpdk/dpdk-2.1.0/tools/dpdk_nic_bind.py --bind=igb_uio ens1f1 > > # /root/openvswitch/openvswitch-2.4.0/ovsdb/ovsdb-server -- > remote=punix:/usr/local/var/run/openvswitch/db.sock -- > remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile -- > detach > > # rm -f /usr/local/var/run/openvswitch/vhost-user* > > # /root/openvswitch/openvswitch-2.4.0/vswitchd/ovs-vswitchd --dpdk -c > 0xffff -n 4 --socket-mem 1024 -- unix:$DB_SOCK --pidfile --detach --log- > file=/var/log/openvswitch/ovs-vswitchd.log > > # lspci | grep Ethernet > 01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit > X540-AT2 (rev 01) > 01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit > X540-AT2 (rev 01) > > # /root/openvswitch/openvswitch-2.4.0/utilities/ovs-vsctl add-br br0 -- set > bridge br0 datapath_type=netdev > # /root/openvswitch/openvswitch-2.4.0/utilities/ovs-vsctl add-port br0 > dpdk0 -- set Interface dpdk0 type=dpdk > # /root/openvswitch/openvswitch-2.4.0/utilities/ovs-vsctl add-port br0 > dpdk1 -- set Interface dpdk1 type=dpdk > # /root/openvswitch/openvswitch-2.4.0/utilities/ovs-vsctl add-port br0 > vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser > # /root/openvswitch/openvswitch-2.4.0/utilities/ovs-vsctl add-port br0 > vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser > > - Start the first VM like this: > ./qemu-system-x86_64 -enable-kvm -m 1024 -smp 2 \
My first suggestion would be to try supplying your VM with more memory. ie. instead of "-m 1024", try "-m 4096" Thanks, Ciara > -display sdl \ > -sdl \ > -chardev socket,id=char0,path=/usr/local/var/run/openvswitch/vhost-user1 > \ > -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ > -device virtio-net-pci,netdev=mynet1,mac=52:54:00:02:d9:01 \ > -object memory-backend-file,id=mem,size=1024M,mem- > path=/dev/hugepages,share=on \ > -numa node,memdev=mem -mem-prealloc \ > -net user,hostfwd=tcp::10021-:22 \ > -net nic \ > /var/lib/libvirt/images/centos-vm1.qcow2 & > > - From another machine send iperf traffic to this VM. > iperf -s (in the VM) > iperf -c <ipaddr> -t 60 -P 12 (from another machine) > > - After about a few seconds, iperf stalls. network connectivity breaks. Can't > ping the VM anymore. > - Saw these in the ovs-vswitchd log files. > > Sep 29 17:51:08 localhost.localdomain ovs-vswitchd[6483]: VHOST_DATA: > Failed to allocate memory for mbuf. > Sep 29 17:51:08 localhost.localdomain ovs-vswitchd[6483]: VHOST_DATA: > Failed to allocate memory for mbuf. > Sep 29 17:51:08 localhost.localdomain ovs-vswitchd[6483]: VHOST_DATA: > Failed to allocate memory for mbuf. > Sep 29 17:51:08 localhost.localdomain ovs-vswitchd[6483]: VHOST_DATA: > Failed to allocate memory for mbuf. > Sep 29 17:51:08 localhost.localdomain ovs-vswitchd[6483]: VHOST_DATA: > Failed to allocate memory for mbuf. > > - shutdown iperf ... then shutdown the VM. > - at this point ovs-vswitchd crashes with the following SEGV. > > [root@localhost vswitchd]# gdb ./ovs-vswitchd /var/spool/abrt/ccpp-2015- > 09-29-18\:47\:37-22012/coredump > GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7 > Copyright (C) 2013 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show > copying" > and "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu". > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>... > Reading symbols from /root/openvswitch/openvswitch-2.4.0/vswitchd/ovs- > vswitchd...done. > [New LWP 22051] > [New LWP 22012] > [New LWP 22013] > [New LWP 22015] > [New LWP 22039] > [New LWP 22040] > [New LWP 22041] > [New LWP 22042] > [New LWP 22043] > [New LWP 22044] > [New LWP 22045] > [New LWP 22046] > [New LWP 22047] > [New LWP 22048] > [New LWP 22049] > [New LWP 22050] > [New LWP 22014] > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > Core was generated by `/root/openvswitch/openvswitch- > 2.4.0/vswitchd/ovs-vswitchd --dpdk -c 0xffff -n 4'. > Program terminated with signal 11, Segmentation fault. > #0 0x00007eff4d71df1f in rte_vhost_dequeue_burst () from > /lib64/libdpdk.so > Missing separate debuginfos, use: debuginfo-install glibc-2.17-78.el7.x86_64 > (gdb) where > #0 0x00007eff4d71df1f in rte_vhost_dequeue_burst () from > /lib64/libdpdk.so > #1 0x000000000056d103 in netdev_dpdk_vhost_rxq_recv > (rxq_=0x7efe5a537600, packets=0x7eff4c9a2930, c=0x7eff4c9a292c) > at lib/netdev-dpdk.c:952 > #2 0x00000000004a2dec in netdev_rxq_recv (rx=0x7efe5a537600, > buffers=0x7eff4c9a2930, cnt=0x7eff4c9a292c) > at lib/netdev.c:652 > #3 0x0000000000478f33 in dp_netdev_process_rxq_port (pmd=0x231c310, > port=0x2166990, rxq=0x7efe5a537600) > at lib/dpif-netdev.c:2520 > #4 0x0000000000479648 in pmd_thread_main (f_=0x231c310) at lib/dpif- > netdev.c:2680 > #5 0x00000000004f60f3 in ovsthread_wrapper (aux_=0x231c150) at lib/ovs- > thread.c:340 > #6 0x00007eff4d276df5 in start_thread () from /lib64/libpthread.so.0 > #7 0x00007eff4ca9a1ad in clone () from /lib64/libc.so.6 _______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
