Hello, Situation:
We are experiencing performance issues with OpenVswitch on Xenserver. We use vswitch to regulate traffic between customer machines on a shared platform. A customer can only connect to a machines that 'belong' to this customer. To get the needed information we created a script for internal use only.
Script function in a nutshell. * Get nodes that belong to customer X using a webcall * add generic rules ( gateways and such ) * add flows src_ -- dst_ accordingly Nothing fancy just automate the adding / deleting of flows within a datapath. During a regular run some flows get added/ some deleted. e.g.: Added 16 flow(s) Deleted 14 flow(s) Setup: Xen server 5.6 Service Pack 2 ovs-vswitchd (Open vSwitch) 1.0.2 OpenFlow versions 0x1:0x1 --- ovs-dpctl show: system@dp0: flows: cur:5, soft-max:8192, hard-max:1048576 ports: cur:2, max:1024 groups: max:16 lookups: frags:0, hit:264762685, missed:23143102, lost:52 queues: max-miss:100, max-action:100 port 0: xenbr0 (internal) port 1: eth0 system@dp1: flows: cur:255, soft-max:8192, hard-max:1048576 ports: cur:31, max:1024 groups: max:16 lookups: frags:0, hit:1792946153, missed:392494847, lost:1806195 queues: max-miss:100, max-action:100 port 0: xenbr1 (internal) port 1: eth1 port 2: xapi1 (internal) port 3: xapi12 (internal) port 4: vif147.0 port 5: xapi13 (internal) port 6: vif148.0 port 7: xapi14 (internal) port 8: vif149.0 port 9: xapi15 (internal) port 10: xapi16 (internal) port 11: vif151.0 port 12: xapi26 (internal) port 13: xapi17 (internal) port 14: xapi18 (internal) port 15: vif152.1 port 16: vif150.0 port 17: xapi19 (internal) port 18: xapi27 (internal) port 19: xapi33 (internal) port 20: xapi28 (internal) port 21: xapi34 (internal) port 22: xapi35 (internal) port 23: xapi40 (internal) port 24: xapi41 (internal) port 25: xapi42 (internal) port 26: xapi43 (internal) port 27: xapi44 (internal) port 28: xapi45 (internal) port 29: xapi46 (internal) port 30: vif153.0 system@dp2: flows: cur:19, soft-max:1024, hard-max:1048576 ports: cur:2, max:1024 groups: max:16 lookups: frags:0, hit:1667986286, missed:8951406, lost:13219 queues: max-miss:100, max-action:100 port 0: xenbr2 (internal) port 1: eth2 system@dp3: flows: cur:6, soft-max:1024, hard-max:1048576 ports: cur:2, max:1024 groups: max:16 lookups: frags:0, hit:7368959, missed:8706606, lost:13304 queues: max-miss:100, max-action:100 port 0: xenbr3 (internal) port 1: eth3 Issues: During adding or deleting rules in datapath: * packets are being dropped * 2 noticeable high pings * Flows in datapath are being dropped (it seems) * 1 min avg load on the Xenserver is high 3-5. ovs-vswitchd 101% CPU missed status: increases eventually lost: increases Current flows in datapath decreasing ovs-dpctl show system@dp1 system@dp1: flows: cur:306, soft-max:65536, hard-max:1048576 ports: cur:30, max:1024 groups: max:16 lookups: frags:0, hit:992564, missed:795173, lost:532618 queues: max-miss:100, max-action:100 Reproduce: Generate flows by issueing a hping to random nodes on the Xen host hping3 -S -L 0 -p 3389 --fast <node1> hping3 -S -L 0 -p 3389 --fast <node2> hping3 -S -L 0 -p 3389 --fast <node3> hping3 -S -L 0 -p 3389 --fast <node4> dump flows in system@dp1 <mailto:system@dp1> while true ; do ovs-dpctl dump-flows system@dp1 | grep <source host> | wc -l ; sleep 0.5 ; done IDLE: 19 19 19 Flows in system@dp1 <mailto:system@dp1>: while adding / removing rules: 367 337 367 343 319 355 60 ? drops 90 ? drops 126 ? drops 156 ? drops 186 ? drops 216 246 282 ^C ON HOST show datapaths : # while true; do ovs-dpctl show | egrep 'system|flows|missed:|lost:'; sleep 1 ; echo " " ; done system@dp0: flows: cur:12, soft-max:8192, hard-max:1048576 lookups: frags:0, hit:110947, missed:55395, lost:0 system@dp1: flows: cur:723, soft-max:65536, hard-max:1048576 lookups: frags:0, hit:1032970, missed:827870, lost:532618 system@dp2: flows: cur:4, soft-max:1024, hard-max:1048576 lookups: frags:0, hit:3110, missed:4807, lost:0 system@dp3: flows: cur:26, soft-max:1024, hard-max:1048576 lookups: frags:0, hit:2785444, missed:5148, lost:0 Anyone seen this behavior before ? Any hints on how to solve this problem ? --- Kind regards, Martin Willemsma -- This e-mail is intended exclusively for the addressee(s), and may not be passed on to, or made available for use by any person other than the addressee(s). SaaSplaza rules out any and every liability resulting from any electronic transmission.
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
