On 3 February 2017 at 14:26, Joe Stringer <[email protected]> wrote: > On 3 February 2017 at 11:25, Vijay Sampath via discuss > <[email protected]> wrote: >> We have a default rule in ovs which I assume makes it behave like a regular >> L2 switch >> >> cookie=0x0, duration=71407.425s, table=0, n_packets=33577078, >> n_bytes=38722336595, idle_age=0, hard_age=65534, priority=0 actions=NORMAL >> >> Through a traffic generator we are sending unknown unicast traffic/broadcast >> traffic to/from about 10000 hosts at say 500 pkts/sec. We see that this >> causes really high CPU utilization with the revalidator threads as shown: >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ >> COMMAND >> 1522 root 20 0 413360 55972 3916 S 85.4 0.7 603:29.96 >> revalidator8 >> 1521 root 20 0 413360 55972 3916 R 79.7 0.7 616:24.68 >> revalidator9 >> >> >> And the following logs are seen in ovs-vswitchd.log >> >> 2017-02-02T21:27:15.474Z|00009|poll_loop(revalidator23)|INFO|wakeup due to >> [POLLIN] on fd 52 (FIFO pipe:[23153437]) at lib/ovs-thread.c:306 (54% CPU >> usage) >> 2017-02-02T21:27:15.530Z|00014|poll_loop(revalidator22)|INFO|wakeup due to >> [POLLIN] on fd 50 (FIFO pipe:[23153436]) at lib/ovs-thread.c:306 (58% CPU >> usage) >> 2017-02-02T21:27:15.532Z|00015|poll_loop(revalidator22)|INFO|wakeup due to >> [POLLIN] on fd 50 (FIFO pipe:[23153436]) at lib/ovs-thread.c:306 (58% CPU >> usage) >> 2017-02-02T21:27:21.444Z|00016|poll_loop(revalidator22)|INFO|Dropped 242 log >> messages in last 5 seconds (most recently, 0 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:21.445Z|00017|poll_loop(revalidator22)|INFO|wakeup due to >> [POLLIN] on fd 50 (FIFO pipe:[23153436]) at lib/ovs-thread.c:306 (73% CPU >> usage) >> 2017-02-02T21:27:27.471Z|00010|poll_loop(revalidator23)|INFO|Dropped 190 log >> messages in last 6 seconds (most recently, 0 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:27.471Z|00011|poll_loop(revalidator23)|INFO|wakeup due to >> [POLLIN] on fd 52 (FIFO pipe:[23153437]) at lib/ovs-thread.c:306 (82% CPU >> usage) >> 2017-02-02T21:27:33.439Z|00012|poll_loop(revalidator23)|INFO|Dropped 195 log >> messages in last 6 seconds (most recently, 0 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:33.439Z|00013|poll_loop(revalidator23)|INFO|wakeup due to >> [POLLIN] on fd 52 (FIFO pipe:[23153437]) at lib/ovs-thread.c:306 (88% CPU >> usage) >> 2017-02-02T21:27:39.479Z|00014|poll_loop(revalidator23)|INFO|Dropped 203 log >> messages in last 6 seconds (most recently, 0 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:39.479Z|00015|poll_loop(revalidator23)|INFO|wakeup due to >> [POLLIN] on fd 52 (FIFO pipe:[23153437]) at lib/ovs-thread.c:306 (78% CPU >> usage) >> 2017-02-02T21:27:45.469Z|00016|poll_loop(revalidator23)|INFO|Dropped 239 log >> messages in last 6 seconds (most recently, 0 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:45.469Z|00017|poll_loop(revalidator23)|INFO|wakeup due to >> [POLLIN] on fd 52 (FIFO pipe:[23153437]) at lib/ovs-thread.c:306 (80% CPU >> usage) >> 2017-02-02T21:27:51.733Z|00018|poll_loop(revalidator22)|INFO|Dropped 213 log >> messages in last 7 seconds (most recently, 1 seconds ago) due to excessive >> rate >> 2017-02-02T21:27:51.733Z|00019|poll_loop(revalidator22)|INFO|wakeup due to >> 422-ms timeout at ofproto/ofproto-dpif-upcall.c:917 (71% CPU usage) >> >> Are there any tips to improve OVS performance under such traffic, where the >> kernel cache may be constantly thrashed? > > Revalidator threads are pretty much designed to keep as many flows in > the datapath as it can, consuming additional CPU if necessary to do > so, with a maximum number of datapath flows around 200,000. I see that > you've limited this to 1 core using the n-revalidator-threads, another > way would to be also set the flow-limit option in the same > other_config column. This limits the number of flows that will exist > in the datapath; if this is low enough, then revalidators won't need > to spend so much CPU maintaining the flows, their statistics, > correctness, etc. YMMV. > >> Is there a way to wildcard Layer 2 information in the packets and purely >> forward packets based on vlan, port, so that the kernel cache undergoes less >> thrashing? > > Yes, you should be able to wildcard the src/dst MACs.
Oh, I should clarify, I don't think this is compatible with the NORMAL action; that needs to specify MACs because it performs MAC learning. _______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
