Hi,
The ping started working once the mode is changed from active-standby to "round-robin". Now after changing it ping started working. Thanks for all your help [😊] Br, Deepak ________________________________ From: Dev <dev-boun...@lists.opencontrail.org> on behalf of Deepak Tiwari <deepak.tiw...@aricent.com> Sent: Friday, April 21, 2017 11:43 AM To: Rajagopalan Sivaramakrishnan; dev@lists.opencontrail.org Subject: [opencontrail-dev] Need help in setting up vRouter in DPDK-enabled mode - with Bond interface Hi Raja, Note: Initiating another thread of this mail for setting up vrouter with a bond interface I need to setup vrouter where two slave members are added to a bond interface. For this I followed below manual steps which were completed successfully. However I am not able to ping other machines over vhost0 interface. I have connected both slave ports to the switch. Please provide some clue for this issue so that we can proceed with testing the "vifdump-for-bond-slaves" feature changes 1. Bring up vrouter-dpdk binary and provide bond/slave information as --vdev option:- => sudo taskset -c 2 /usr/bin/contrail-vrouter-dpdk --no-daemon --vdev "eth_bond0,mode=4,xmit_policy=l34,socket_id=0,mac=00:15:1a:6e:2e:ca,slave=0000:01:00.0,slave=0000:01:00.1" --socket-mem 1024 Please note that mac for bond is kept same as that of first slave. I think that is how it should be? 2. Add bond and vhost0 interfaces to vrouter:- => sudo vif --create vhost0 --mac 00:15:1a:6e:2e:ca => sudo vif --add 0 --mac 00:15:1a:6e:2e:ca --vrf 0 --vhost-phys --type physical --pmd --transport pmd --id 0 => sudo vif --add vhost0 --mac 00:15:1a:6e:2e:ca --vrf 0 --xconnect 0 --type vhost --pmd --transport pmd --id 1 When adding bond interface to vrouter (2nd command), vrouter was giving as error. The reason was that bond interface API "bond_ethdev_info" returns max_rx_queues = 128 and max_tx_queues = 512 which is limited to max_rx_queues = 1 and max_tx_queues = 128 by vrouter (i.e. dpdk_ethdev_info_update procedure). Later on "slave_configure" procedure goes on to setup same number of queues (rx=1, tx=128) for the slaves. However slave have max_rx_queues and max_tx_queues configured as 1 so it fails. So to proceed further, I did a temporary change in DPDK bond pmd API "bond_ethdev_info" to return max_rx_queues = max_tx_queues = 1. After this change bond was created successfully and slaves got added to it. Then I added vhost interface as well 3. Bring up vhost0 interface with IP/gateway etc:- => sudo ifdown -i /tmp/interfaces vhost0 => sudo ifup -i /tmp/interfaces vhost0 Logs 1. Vrouter console output contrail@contrail-23:~/dev/contrail-installer$ sudo gdb --args taskset -c 2 /usr/bin/contrail-vrouter-dpdk --no-daemon --vdev "eth_bond0,mode=4,xmit_policy=l34,socket_id=0,mac=00:15:1a:6e:2e:ca,slave=0000:01:00.0,slave=0000:01:00.1" --socket-mem 1024 GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from taskset...(no debugging symbols found)...done. (gdb) r Starting program: /usr/bin/taskset -c 2 /usr/bin/contrail-vrouter-dpdk --no-daemon --vdev eth_bond0,mode=4,xmit_policy=l34,socket_id=0,mac=00:15:1a:6e:2e:ca,slave=0000:01:00.0,slave=0000:01:00.1 --socket-mem 1024 process 11961 is executing new program: /usr/bin/contrail-vrouter-dpdk [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 2017-04-21 10:35:35,066 VROUTER: vRouter/DPDK version: {"build-info": [{"build-time": "2017-04-21 04:37:16.403027", "build-hostname": "contrail-23", "build-user": "root", "build-version": "4.0.0.0"}]} 2017-04-21 10:35:35,122 VROUTER: Bridge Table limit: 262144 2017-04-21 10:35:35,122 VROUTER: Bridge Table overflow limit: 53248 2017-04-21 10:35:35,122 VROUTER: Flow Table limit: 524288 2017-04-21 10:35:35,122 VROUTER: Flow Table overflow limit: 105472 2017-04-21 10:35:35,122 VROUTER: MPLS labels limit: 5120 2017-04-21 10:35:35,122 VROUTER: Nexthops limit: 65536 2017-04-21 10:35:35,122 VROUTER: VRF tables limit: 4096 2017-04-21 10:35:35,122 VROUTER: Packet pool size: 16384 2017-04-21 10:35:35,122 VROUTER: Maximum packet size: 9216 2017-04-21 10:35:35,122 VROUTER: EAL arguments: 2017-04-21 10:35:35,122 VROUTER: -n "4" 2017-04-21 10:35:35,122 VROUTER: --socket-mem "1024" 2017-04-21 10:35:35,122 VROUTER: --vdev "eth_bond0,mode=4,xmit_policy=l34,socket_id=0,mac=00:15:1a:6e:2e:ca,slave=0000:01:00.0,slave=0000:01:00.1" 2017-04-21 10:35:35,122 VROUTER: --lcores "(0-2)@(0-3),(8-9)@(0-3),10@2" EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up physically contiguous memory... EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffea600000 (size = 0x200000) EAL: Ask a virtual area of 0x3cc00000 bytes EAL: Virtual area found at 0x7fffad800000 (size = 0x3cc00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fffad400000 (size = 0x200000) EAL: Ask a virtual area of 0x2000000 bytes EAL: Virtual area found at 0x7fffab200000 (size = 0x2000000) EAL: Ask a virtual area of 0x3ae00000 bytes EAL: Virtual area found at 0x7fff70200000 (size = 0x3ae00000) EAL: Requesting 512 pages of size 2MB from socket 0 [New Thread 0x7ffff0449700 (LWP 11965)] EAL: TSC frequency is ~2659984 KHz EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: Master lcore 0 is ready (tid=f7fd9b00;cpuset=[0,1,2,3]) EAL: Initializing pmd_bond for eth_bond0 PMD: Using mode 4, it is necessary to do TX burst and RX burst at least every 100ms. EAL: Create bonded device eth_bond0 on port 0 in mode 4 on socket 0. [New Thread 0x7fffefc48700 (LWP 11966)] EAL: lcore 1 is ready (tid=efc48700;cpuset=[0,1,2,3]) [New Thread 0x7fffef447700 (LWP 11967)] [New Thread 0x7fffeec46700 (LWP 11968)] [New Thread 0x7fffee445700 (LWP 11969)] EAL: lcore 8 is ready (tid=eec46700;cpuset=[0,1,2,3]) EAL: lcore 9 is ready (tid=ee445700;cpuset=[0,1,2,3]) EAL: lcore 2 is ready (tid=ef447700;cpuset=[0,1,2,3]) [New Thread 0x7fffedc44700 (LWP 11970)] EAL: lcore 10 is ready (tid=edc44700;cpuset=[2]) EAL: PCI device 0000:01:00.0 on NUMA socket -1 EAL: probe driver: 8086:105e rte_em_pmd EAL: PCI memory mapped at 0x7fffea800000 EAL: PCI memory mapped at 0x7fffea820000 PMD: eth_em_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x105e EAL: PCI device 0000:01:00.1 on NUMA socket -1 EAL: probe driver: 8086:105e rte_em_pmd EAL: PCI memory mapped at 0x7fffea840000 EAL: PCI memory mapped at 0x7fffea860000 PMD: eth_em_dev_init(): port_id 2 vendorID=0x8086 deviceID=0x105e 2017-04-21 10:35:37,148 VROUTER: Found 3 eth device(s) 2017-04-21 10:35:37,148 VROUTER: Using 1 forwarding lcore(s) 2017-04-21 10:35:37,148 VROUTER: Using 0 IO lcore(s) 2017-04-21 10:35:37,148 VROUTER: Using 5 service lcores 2017-04-21 10:35:37,148 VROUTER: Max HOLD flow entries set to 1000 2017-04-21 10:35:37,148 VROUTER: set fd limit to 4096 (prev 1024, max 4096) 2017-04-21 10:35:37,199 VROUTER: Starting NetLink... 2017-04-21 10:35:37,199 USOCK: usock_alloc[7fffee445700]: new socket FD 33 2017-04-21 10:35:37,199 USOCK: usock_alloc[7fffee445700]: setting socket FD 33 send buff size. Buffer size set to 18432000 (requested 9216000) 2017-04-21 10:35:37,199 VROUTER: NetLink TCP socket FD is 33 2017-04-21 10:35:37,199 VROUTER: uvhost Unix socket FD is 34 2017-04-21 10:35:37,201 UVHOST: Starting uvhost server... 2017-04-21 10:35:37,201 UVHOST: server event FD is 35 2017-04-21 10:35:37,201 UVHOST: server socket FD is 36 2017-04-21 10:35:37,202 VROUTER: Lcore 10: distributing MPLSoGRE packets to [] 2017-04-21 10:35:38,199 VROUTER: Retrying connection for socket 34... 2017-04-21 10:35:38,199 UVHOST: Handling connection FD 36... 2017-04-21 10:35:38,199 UVHOST: FD 36 accepted new NetLink connection FD 37 2017-04-21 10:36:00,501 VROUTER: Adding vif 0 (gen. 1) eth device 0 (PMD) MAC 00:00:00:00:00:00 (vif MAC 00:15:1a:6e:2e:ca) 2017-04-21 10:36:00,501 VROUTER: bond eth device 0 configured MAC 00:15:1a:6e:2e:ca 2017-04-21 10:36:00,501 VROUTER: bond member eth device 1 PCI 0000:01:00.0 MAC 00:15:1a:6e:2e:ca 2017-04-21 10:36:00,501 VROUTER: bond member eth device 1 promisc mode disabled 2017-04-21 10:36:00,501 VROUTER: bond member eth device 2 PCI 0000:01:00.1 MAC 00:15:1a:6e:2e:cb 2017-04-21 10:36:00,501 VROUTER: bond member eth device 2 promisc mode disabled 2017-04-21 10:36:00,501 VROUTER: setup 1 RSS queue(s) and 0 filtering queue(s) 2017-04-21 10:36:01,284 VROUTER: lcore 10 TX to HW queue 0 2017-04-21 10:36:01,284 VROUTER: lcore 8 TX queue 0 to SW ring in lcore 10 2017-04-21 10:36:01,284 VROUTER: lcore 10 now has 1 ring(s) to push 2017-04-21 10:36:01,284 VROUTER: lcore 9 TX queue 0 to SW ring in lcore 10 2017-04-21 10:36:01,284 VROUTER: lcore 10 now has 2 ring(s) to push 2017-04-21 10:36:01,284 VROUTER: lcore 10 RX from HW queue 0 2017-04-21 10:36:02,755 PMD: eth_em_interrupt_action(): Port 2: Link Up - speed 100 Mbps - full-duplex 2017-04-21 10:36:03,287 PMD: eth_em_interrupt_action(): Port 1: Link Up - speed 1000 Mbps - full-duplex 2017-04-21 10:36:13,244 VROUTER: Adding vif 1 (gen. 2) device vhost0 at eth device 0 MAC 00:15:1a:6e:2e:ca (vif MAC 00:15:1a:6e:2e:ca) 2017-04-21 10:36:13,244 VROUTER: using bond slave eth device 1 MAC 00:15:1a:6e:2e:ca 2017-04-21 10:36:13,244 VROUTER: initializing KNI with 5 maximum interfaces 2017-04-21 10:36:13,245 VROUTER: bind KNI kernel thread to CPU 1 2017-04-21 10:36:13,245 KNI: pci: 01:00:00 8086:105e 2017-04-21 10:36:13,245 VROUTER: lcore 10 TX to HW queue 0 2017-04-21 10:36:13,245 VROUTER: lcore 8 TX queue 0 to SW ring in lcore 10 2017-04-21 10:36:13,245 VROUTER: lcore 10 now has 3 ring(s) to push 2017-04-21 10:36:13,245 VROUTER: lcore 9 TX queue 0 to SW ring in lcore 10 2017-04-21 10:36:13,245 VROUTER: lcore 10 now has 4 ring(s) to push 2017-04-21 10:36:13,245 VROUTER: lcore 10 RX from HW queue 0 2017-04-21 10:37:00,572 VROUTER: Configuring eth device 0 UP 2. Vif --list output contrail@contrail-23:/opt/stack/contrail$ vif --list Vrouter Interface Table Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2 D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled Proxy=MAC Requests Proxied Always, Er=Etree Root vif0/0 PMD: 0 (Speed 100, Duplex 1) Type:Physical HWaddr:00:15:1a:6e:2e:ca IPaddr:0 Vrf:0 Flags:XL3L2VpDpdk MTU:1514 QOS:0 Ref:5 RX device packets:74 bytes:39140 errors:0 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 RX packets:0 bytes:0 errors:0 TX packets:54 bytes:8295 errors:0 Drops:0 TX port packets:0 errors:54 TX device packets:347 bytes:44416 errors:0 vif0/1 PMD: 0 Type:Host HWaddr:00:15:1a:6e:2e:ca IPaddr:0 Vrf:0 Flags:XL3L2Dpdk MTU:1514 QOS:0 Ref:5 RX port packets:54 errors:0 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 RX packets:54 bytes:8295 errors:0 TX packets:0 bytes:0 errors:0 Drops:0 3. ifconfig output contrail@contrail-23:/opt/stack/contrail$ ifconfig eth4 Link encap:Ethernet HWaddr bc:30:5b:cb:03:8d inet addr:192.168.1.24 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::be30:5bff:fecb:38d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11796 errors:0 dropped:0 overruns:0 frame:0 TX packets:7585 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2925311 (2.9 MB) TX bytes:492123 (492.1 KB) Interrupt:16 eth5 Link encap:Ethernet HWaddr 00:1c:f0:15:2d:91 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:46574357 errors:0 dropped:0 overruns:0 frame:0 TX packets:46574357 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5959097119 (5.9 GB) TX bytes:5959097119 (5.9 GB) vhost0 Link encap:Ethernet HWaddr 00:15:1a:6e:2e:ca inet addr:192.168.1.23 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::f8f1:93ff:fe08:1af8/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:60 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:8877 (8.8 KB) virbr0 Link encap:Ethernet HWaddr 1a:96:53:e4:2c:55 inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 4. rt --dump output contrail@contrail-23:/opt/stack/contrail$ rt --dump 0 Flags: L=Label Valid, P=Proxy ARP, T=Trap ARP, F=Flood ARP vRouter inet4 routing table 0/0/unicast Destination PPL Flags Label Nexthop Stitched MAC(Index) contrail@contrail-23:/opt/stack/contrail$ The route table is empty. However it was same case when I was trying without bond. But then ping was working. Br, Deepak "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus." "DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."
_______________________________________________ Dev mailing list Dev@lists.opencontrail.org http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
