The setting for qemu commands seems nothing wrong.
BTW, I also inspect the packet counter upon ovs rule "#ovs-ofctl dump-flows
br0".
The number is correct.
Is there other way to search the problem?
-----Original Message-----
From: ???
Sent: Friday, September 09, 2016 4:19 PM
To: Xu, Qian Q; users at dpdk.org
Subject: RE: 2VMs with dpdk app-testpmd effect each other
1.Are you using 2 10G port to inject traffic?
Ans: No. one for in, one for out. i used opeflow rule to separate the incoming
stream into two VMs.
2.What's your qemu launch commands for each VM?
Ans:
taskset -c 8,10 ./qemu-2.5.1.1/x86_64-softmmu/qemu-system-x86_64 -cpu host
-enable-kvm -m 4096M -smp 4 -nographic -k en-us -vnc :1 -pidfile
/usr/local/var/run/openvswitch/vm1.pid,snapshot=off -monitor
unix:/usr/local/var/run/openvswitch/vm1monitor,server,nowait\
-object
memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa
node,memdev=mem -mem-prealloc \
-chardev socket,id=char0,path=/usr/local/var/run/openvswitch/vhost-user-1 \
-netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \
-device virtio-net-pci,netdev=mynet1,mac=52:54:00:02:d9:01 \
-chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user-2 \
-netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \
-device virtio-net-pci,netdev=mynet2,mac=52:54:00:02:d9:02 \
vm1.qcow2
taskset -c 12,14 ./qemu-2.5.1.1/x86_64-softmmu/qemu-system-x86_64 -cpu host
-enable-kvm -m 4096M -smp 4 -nographic -k en-us -vnc :2 -pidfile
/usr/local/var/run/openvswitch/vm2.pid,snapshot=off -monitor
unix:/usr/local/var/run/openvswitch/vm1monitor,server,nowait\
-object
memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa
node,memdev=mem -mem-prealloc \
-chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-3 \
-netdev type=vhost-user,id=mynet3,chardev=char2,vhostforce \
-device virtio-net-pci,netdev=mynet3,mac=52:54:00:02:d9:03 \
-chardev socket,id=char3,path=/usr/local/var/run/openvswitch/vhost-user-4 \
-netdev type=vhost-user,id=mynet4,chardev=char3,vhostforce \
-device virtio-net-pci,netdev=mynet4,mac=52:54:00:02:d9:04 \
vm2.qcow2
3.Can you use "top -H" command to check the core usage when u running the case2
and case3?
Ans:
Case2:
105561 root 20 0 41.660g 27388 2300 R 99.9 0.0 1487:40 pmd39
105562 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:13 pmd38
105563 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:12 pmd37
105564 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:13 pmd36
119509 root 20 0 4929700 57736 5296 R 99.9 0.1 7:01.16
qemu-system-x86
Case3:
119509 root 20 0 4929700 57736 5296 R 99.9 0.1 8:44.03
qemu-system-x86
105561 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:23 pmd39
105562 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd38
105563 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd37
105564 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd36
case2 and case 3 seems nothing different~~
-----Original Message-----
From: Xu, Qian Q [mailto:[email protected]]
Sent: Wednesday, September 07, 2016 10:11 AM
To: ???; users at dpdk.org
Subject: RE: 2VMs with dpdk app-testpmd effect each other
It's not normal, and I think first we need figure out it's DPDK issue or OVS
issue or some configuration issue.
Are you using 2 10G port to inject traffic?
What's your qemu launch commands for each VM?
Can you use "top -H" command to check the core usage when u running the case2
and case3?
-----Original Message-----
From: ??? [mailto:[email protected]]
Sent: Wednesday, September 7, 2016 10:04 AM
To: Xu, Qian Q <qian.q.xu at intel.com>; users at dpdk.org
Subject: RE: 2VMs with dpdk app-testpmd effect each other
So my question is : is it normal ?
Is there any way i can solve the problem?
Is the problem related to qemu-img?
________________________________________
?: ???
????: 2016?9?2? ?? 01:37
?: Xu, Qian Q; users at dpdk.org
??: RE: 2VMs with dpdk app-testpmd effect each other
I am sure that VM1 and VM2 are using different core with different coremask
setting.
Yes, I know that default fwd enging is IO fwd. And fwd engine will start while
testpmd is started.
I used "stop" command to stop fwd engine inside testpmd.
The I can see effect ion on the other VM.
-----Original Message-----
From: Xu, Qian Q [mailto:[email protected]]
Sent: Thursday, September 01, 2016 9:36 AM
To: ???; users at dpdk.org
Subject: RE: 2VMs with dpdk app-testpmd effect each other
Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are
using the same core for fwd.
What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO
fwd.
-----Original Message-----
From: users [mailto:[email protected]] On Behalf Of ???
Sent: Thursday, September 1, 2016 8:47 AM
To: users at dpdk.org
Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other
Hi all
I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ).
Please see the following setup.
Hardware Information
System
Dell Inc. PowerEdge R630
?
?
BIOS
Dell Inc. version1.0.4
?
CPU
Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz
?
Processor(s): 16
?
?
Cache: 20480 KB
?
?
?
Memory
Size: 16 GB*6
?
?
?
?
Type: DDR4
?
?
Speed: 2133MHz
?
?
?
Ethernet Controller
Intel Corporation Ethernet 10G 2P X520 Adapter *2
Software information
Qemu = qemu-2.5.1.1
DPDK = dpdk-2.2.0
OVS = ovs-branch-2.5
Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is
generated by Spirent testcenter
| VM1 | | VM2 |
?1001? ?1002?
-----------------------
| OVS |
-----------------------
? ?
-----------------------
TestCenter |
-----------------------
ovs setup:
modprobe openvswitch
rm -rf /usr/local/var/run/openvswitch/*
rm -rf /opt/ovs-vswitchd.log
modprobe uio
insmod $DPDK_BUILD/kmod/igb_uio.ko
$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1
$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2
$DPDK_DIR/tools/dpdk_nic_bind.py --status
rm /usr/local/etc/openvswitch/conf.db -rf
mkdir -p /usr/local/etc/openvswitch
mkdir -p /usr/local/var/run/openvswitch
cd $OVS_DIR
./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db
./vswitchd/vswitch.ovsschema
./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock
--remote=db:Open_vSwitch,Open_vSwitch,manager_options
--private-key=db:Open_vSwitch,SSL,private_key
--certificate=db:Open_vSwitch,SSL,certificate
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
./utilities/ovs-vsctl --no-wait init
./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 --
unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile
--detach
Case 1:
VM1 start testpmd, fwd = iofwd
VM2 start testpmd,fwd = iofwd
Result = Spirent testcenter show output stream is reaching line speed 10G.
Case 2:
VM1 testpmd = started, fwd = iofwd
VM2 testpmd=stopped,fwd = none
Result = Spirent testcenter show output stream is reaching 5G.
Which is reasonable.
Case 3:
VM1 testpmd = started, fwd = iofwd
VM2 testpmd = started ,fwd = none
Result = Spirent testcenter show output stream is less than 200Mbps.
I am confused about the case 3. What is the reason VM2 effects VM1 so
significantly?
How can i solve such problem? I will appreciate for your answer.
Thx
============================
??? Kai-Mao Cheng
??????? ??????? 231??
??:03-4244093 ??:0912314847
E-mail:kmcheng at cht.com.tw
============================
Please be advised that this email message (including any attachments) contains
confidential information and may be legally privileged. If you are not the
intended recipient, please destroy this message and all attachments from your
system and do not further collect, process, or use them. Chunghwa Telecom and
all its subsidiaries and associated companies shall not be liable for the
improper or incomplete transmission of the information contained in this email
nor for any delay in its receipt or damage to your system. If you are the
intended recipient, please protect the confidential and/or personal information
contained in this email with due care. Any unauthorized use, disclosure or
distribution of this message in whole or in part is strictly prohibited. Also,
please self-inspect attachments and hyperlinks contained in this email to
ensure the information security and to protect personal information.
