[ovs-discuss] QoS Rate-Limiting Problem
Dear Sir/Madam, Recently I met a problem when I used open vswitch. Can you help to analysis it. I install the open-vswitch 1.4.0 on Suse linux 11 sp2, the vlan test is successful. But when I test the rate limiting it failed. I start 2 guest Os, one is window server 2003 (VNIC:vif3.0,tap3.0),one is redhat 5(VNIC:vif2.0). then i execute the command as following: ovs-vsctl set Interface vif2.0 ingress_policing_rate=1000 ovs-vsctl set Interface vif3.0 ingress_policing_rate=1000 ovs-vsctl set Interface tap3.0 ingress_policing_rate=1000 then use iperf to test, the result both pass 1M/ps. I doubt if the kernel is not support the open vswitch rate limiting feature? the physical hypervisor kernel is : Linux 150 3.0.13-0.27-xen #1 SMP Wed Feb 15 13:33:49 UTC 2012 (d73692b) x86_64 x86_64 x86_64 GNU/Linux Best Regards, Gu Fuyang Beijing ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
Hi, Do you mean the setup of flows in cached flow table in kernel module? We are sending packets after setting up the flows. Therefore, this problem should be in the fast path or in other words, between the ovs-vswitchd and kernel module. About traffic passing from the switch, MTU size UDP packets are passed from the switch; received from one interface and sent out from the other interface of NIC. On Tue, Apr 24, 2012 at 2:55 AM, Ben Pfaff b...@nicira.com wrote: On Mon, Apr 23, 2012 at 03:54:09PM +0600, junaid khalid wrote: 1) Apr 23 13:19:27|00019|dpif|WARN|system@br0: recv failed (No buffer space available) Packets are arriving at your interfaces faster than ovs-vswitchd can set up flows. What traffic is going through the switch? 2) Apr 23 13:19:37|00033|pktbuf|WARN|cookie mismatch: 01fa != 02fa OVS is sending buffered packets to your OpenFlow controller. The OpenFlow controller is sending back replies to use those buffers, but the replies are arriving slowly enough that by the time that they arrive the switch has already discarded those buffers. Apr 23 13:19:37|00034|connmgr|INFO|br0-tcp:127.0.0.1:6633: sending type OFPET_BAD_REQUEST, code OFPBRC_BUFFER_UNKNOWN error reply to OFPT_FLOW_MOD message Also a consequence of the above. traffic type: MTU size UDP packets @7.3 Gbps, generating 100,000 flows by changing src IPs. NOX isn't going to be able to keep up with that rate. OVS 1.4.0 isn't, either, but OVS 1.6.90 from the tip of master should be able to handle it without a controller. Throwing in an OpenFlow controller that sees every packet will probably bog things down a lot. Apr 23 13:19:16|00010|ofp_util|WARN|received Nicira extension message of unknown type 8 That message has been obsolete for ages, why are you using it? ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
[ovs-discuss] problem in installing openvswitch on centos 6.2 using kvm
Dear all, i have installed openvswitch 1.4.0 on centos 6.2 and kvm when i bring up VMs they use bridge for connecting to host on the other hand, when i want to insert openvswitch module it is like this: insmod: error inserting 'datapath/linux/openvswitch_mod.ko': -1 Invalid module format i've searched and find out in order not to have this problem i should remove bridge with: rmmod bridge in this case, i can insert openvswitch module but i lose connection to my VMs so: 1-how can i insert openvswitch module(insmod datapath/linux/openvswitch_mod.ko) without removing bridge 2-is it possible to connect to VMs without using brdge? 3-what is equal to module-assistant command in ubuntu 11.04 in centos 6.2 thanks a lot ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
On Tue, Apr 24, 2012 at 07:02:37PM +0600, junaid khalid wrote: Do you mean the setup of flows in cached flow table in kernel module? We are sending packets after setting up the flows. Therefore, this problem should be in the fast path or in other words, between the ovs-vswitchd and kernel module. You said you have 100,000 flows. How many packets in each flow? ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
[ovs-discuss] ovs-vsctl, QoS, delay
Hello, is it possible to add a delay to network interfaces via the QoS-settings? We need to simulate a real network with delays. Thank you, Chris ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
[ovs-discuss] [BUG] multi-/broadcast counter, sFlow
Hello, I ran in a curious measurement when I looked at sFlowTool and sFlowTrend numbers. Several counters display 4,294,967,295. These are ifInBroadcastPkts, ifInUnknownProtos, ifOutMulticastPkts, ifOutBroadcastPkts. My setup uses the software at [1] and I start it with [2]. For enabling multicasting I run the following command [0] in each lxc-shell [0] route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0 Is there a possible workaround I can use to reset the sFlow database entries to numbers which are more useable? Looks like this uint(?) was set to -1.. Chris [1]Installed Software: openvswitch-1.4.0, sflowtool-3.25, Ubuntu 11.10 Kernel 3.0.0-12 in VirtualBox, connecting lxc-containers via brctl compatiblity mode. [2]Reproduce via: cd ~/openvswitch-1.4.0 sudo -s rmmod bridge insmod ./datapath/linux/openvswitch_mod.ko insmod ./datapath/linux/brcompat_mod.ko ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,manager_options --pidfile --detach ovs-vsctl --no-wait init ovs-vswitchd --pidfile --detach ovs-brcompatd --pidfile --detach brctl addbr br0 ovs-vsctl -- --id=@s create sFlow agent=lo target=\127.0.0.1:6343\ header=128 sampling=64 polling=1 -- set Bridge br0 sflow=@s lxc.conf: lxc.utsname = one lxc.network.type = veth lxc.network.flags = up lxc.network.link = br0 lxc.network.ipv4 = 10.0.0.1/24 mkdir /cgroup mount -t cgroup cgroup /cgroup lxc-create -n one -f lxc.conf lxc-start -n /bin/bash sflowtool ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
On Tue, Apr 24, 2012 at 9:06 PM, Ben Pfaff b...@nicira.com wrote: On Tue, Apr 24, 2012 at 07:02:37PM +0600, junaid khalid wrote: Do you mean the setup of flows in cached flow table in kernel module? We are sending packets after setting up the flows. Therefore, this problem should be in the fast path or in other words, between the ovs-vswitchd and kernel module. You said you have 100,000 flows. How many packets in each flow? we are generating packets in a round robin fashion, approx. 6 packets per flow per sec. ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
On Wed, Apr 25, 2012 at 10:14 AM, Ben Pfaff b...@nicira.com wrote: On Wed, Apr 25, 2012 at 09:43:50AM +0500, junaid khalid wrote: On Tue, Apr 24, 2012 at 9:06 PM, Ben Pfaff b...@nicira.com wrote: On Tue, Apr 24, 2012 at 07:02:37PM +0600, junaid khalid wrote: Do you mean the setup of flows in cached flow table in kernel module? We are sending packets after setting up the flows. Therefore, this problem should be in the fast path or in other words, between the ovs-vswitchd and kernel module. You said you have 100,000 flows. How many packets in each flow? we are generating packets in a round robin fashion, approx. 6 packets per flow per sec. You might want to increase the flow eviction threshold to 10 then. See the documentation in ovs-vswitchd.conf.db(5). We have also tried that. We set the flow eviction threshold to 100,000 and added a periodic print in the kernel module to check the number of entries in the flow table in kernel module. We noticed that, although the flow table size in userspace is 100,000 (from dump-aggregate command), but the flow table size in kernel module increases gradually and the error continues to come up till the kernel table is completely filled with 10 entries. ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
On Wed, Apr 25, 2012 at 10:28:05AM +0500, junaid khalid wrote: On Wed, Apr 25, 2012 at 10:14 AM, Ben Pfaff b...@nicira.com wrote: On Wed, Apr 25, 2012 at 09:43:50AM +0500, junaid khalid wrote: On Tue, Apr 24, 2012 at 9:06 PM, Ben Pfaff b...@nicira.com wrote: On Tue, Apr 24, 2012 at 07:02:37PM +0600, junaid khalid wrote: Do you mean the setup of flows in cached flow table in kernel module? We are sending packets after setting up the flows. Therefore, this problem should be in the fast path or in other words, between the ovs-vswitchd and kernel module. You said you have 100,000 flows. How many packets in each flow? we are generating packets in a round robin fashion, approx. 6 packets per flow per sec. You might want to increase the flow eviction threshold to 10 then. See the documentation in ovs-vswitchd.conf.db(5). We have also tried that. We set the flow eviction threshold to 100,000 and added a periodic print in the kernel module to check the number of entries in the flow table in kernel module. We noticed that, although the flow table size in userspace is 100,000 (from dump-aggregate command), but the flow table size in kernel module increases gradually and the error continues to come up till the kernel table is completely filled with 10 entries. I'm not sure what's the error. If it's the one about unknown buffers, then that's probably because you've got some kind of packet loss between the switch and your controller. Or your controller can't keep up with 100,000 packet-ins in the first 166 ms (1/6 of a second)? Is the controller in-band? ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
Re: [ovs-discuss] warning messages regarding buffer space and unknown buffer
Thanks a lot. I will update you after testing it On Wed, Apr 25, 2012 at 10:45 AM, Ben Pfaff b...@nicira.com wrote: On Wed, Apr 25, 2012 at 10:38:45AM +0500, junaid khalid wrote: we are talking about the recv failed problem (system@br0: recv failed (No buffer space available)). Try applying the following patch from master to your tree. It may help. --8--cut here--8-- From: Ben Pfaff b...@nicira.com Date: Thu, 15 Mar 2012 21:15:38 -0700 Subject: [PATCH] netlink-socket: Increase Netlink socket receive buffer size. Open vSwitch userspace can set up flows at a high rate, but it is somewhat bursty in opportunities to set up flows, by which I mean that OVS sets up a batch of flows, then goes off and does some other work for a while, then sets up another batch of flows, and so on. The result is that, if a large number of packets that need flow setups come in all at once, then some of them can overflow the relatively small kernel-to-user buffers. This commit increases the kernel-to-user buffers from the default of approximately 120 kB each to 1 MB each. In one somewhat synthetic test case that I ran based on an hping3 that generated a load of about 20,000 new flows per second (including both requests and replies), this reduced the packets dropped at the kernel-to-user interface from about 30% to none. I expect that it will similarly improve packet loss in workloads where flow arrival is not easily predictable. (This has little effect on workloads generated by ovs-benchmark rate because that benchmark is effectively self-clocking, that is, a new flow is triggered only by a reply to a request made earlier, which means that the number of buffered packets at any given has a known, constant upper limit.) Bug #10210. Signed-off-by: Ben Pfaff b...@nicira.com --- include/sparse/sys/socket.h |5 +++-- lib/netlink-socket.c| 10 +- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/include/sparse/sys/socket.h b/include/sparse/sys/socket.h index 89e3c2d..1ed195b 100644 --- a/include/sparse/sys/socket.h +++ b/include/sparse/sys/socket.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2011 Nicira Networks. + * Copyright (c) 2011, 2012 Nicira Networks. * * Licensed under the Apache License, Version 2.0 (the License); * you may not use this file except in compliance with the License. @@ -74,7 +74,8 @@ enum { SO_SNDBUF, SO_SNDLOWAT, SO_SNDTIMEO, -SO_TYPE +SO_TYPE, +SO_RCVBUFFORCE }; enum { diff --git a/lib/netlink-socket.c b/lib/netlink-socket.c index bc46235..df6f1d8 100644 --- a/lib/netlink-socket.c +++ b/lib/netlink-socket.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008, 2009, 2010, 2011 Nicira Networks. + * Copyright (c) 2008, 2009, 2010, 2011, 2012 Nicira Networks. * * Licensed under the Apache License, Version 2.0 (the License); * you may not use this file except in compliance with the License. @@ -89,6 +89,7 @@ nl_sock_create(int protocol, struct nl_sock **sockp) struct nl_sock *sock; struct sockaddr_nl local, remote; socklen_t local_size; +int rcvbuf; int retval = 0; if (!max_iovs) { @@ -122,6 +123,13 @@ nl_sock_create(int protocol, struct nl_sock **sockp) sock-protocol = protocol; sock-dump = NULL; +rcvbuf = 1024 * 1024; +if (setsockopt(sock-fd, SOL_SOCKET, SO_RCVBUFFORCE, + rcvbuf, sizeof rcvbuf)) { +VLOG_WARN_RL(rl, setting %d-byte socket receive buffer failed (%s), + rcvbuf, strerror(errno)); +} + retval = get_socket_rcvbuf(sock-fd); if (retval 0) { retval = -retval; -- 1.7.2.5 ___ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss