Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)
Found the issue, I have idea of unit testing it, will experiment on it, On Tue, Sep 30, 2014 at 11:11 AM, Ben Pfaff wrote: > It's not the first time we've had this kind of bug. Last time, I > tried to think of a good unit test, but did not succeed. > > On Tue, Sep 30, 2014 at 10:32:15AM -0700, Alex Wang wrote: > > I can reproduce it, working on a fix, > > > > Thx for reporting~ > > > > On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff wrote: > > > > > There's still a 100% CPU bug in ovs-vswitchd. Run "make sandbox", then > > > run a second ovs-vswitchd inside the subshell (just run a bare > > > "ovs-vswitchd" without any parameters). You get output like this: > > > > > > 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on > > > NUMA node 0 > > > 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes > and 8 > > > CPU cores > > > > > > > 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > > > connecting... > > > > > > > 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > > > connected > > > 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process > is > > > running, disabling this process (pid 13812) until it goes away > > > 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms > timeout > > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > > > > The cited line in bridge.c is: > > > > > > seq_wait(connectivity_seq_get(), connectivity_seqno); > > > > > > Thanks, > > > > > > Ben. > > > > ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)
It's not the first time we've had this kind of bug. Last time, I tried to think of a good unit test, but did not succeed. On Tue, Sep 30, 2014 at 10:32:15AM -0700, Alex Wang wrote: > I can reproduce it, working on a fix, > > Thx for reporting~ > > On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff wrote: > > > There's still a 100% CPU bug in ovs-vswitchd. Run "make sandbox", then > > run a second ovs-vswitchd inside the subshell (just run a bare > > "ovs-vswitchd" without any parameters). You get output like this: > > > > 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on > > NUMA node 0 > > 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8 > > CPU cores > > > > 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > > connecting... > > > > 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > > connected > > 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is > > running, disabling this process (pid 13812) until it goes away > > 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout > > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > > > The cited line in bridge.c is: > > > > seq_wait(connectivity_seq_get(), connectivity_seqno); > > > > Thanks, > > > > Ben. > > ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)
I can reproduce it, working on a fix, Thx for reporting~ On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff wrote: > There's still a 100% CPU bug in ovs-vswitchd. Run "make sandbox", then > run a second ovs-vswitchd inside the subshell (just run a bare > "ovs-vswitchd" without any parameters). You get output like this: > > 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on > NUMA node 0 > 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8 > CPU cores > > 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > connecting... > > 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: > connected > 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is > running, disabling this process (pid 13812) until it goes away > 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout > at ../vswitchd/bridge.c:2723 (100% CPU usage) > > The cited line in bridge.c is: > > seq_wait(connectivity_seq_get(), connectivity_seqno); > > Thanks, > > Ben. > ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
[ovs-dev] 100% CPU in ovs-vswitchd (again)
There's still a 100% CPU bug in ovs-vswitchd. Run "make sandbox", then run a second ovs-vswitchd inside the subshell (just run a bare "ovs-vswitchd" without any parameters). You get output like this: 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on NUMA node 0 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8 CPU cores 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: connecting... 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock: connected 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is running, disabling this process (pid 13812) until it goes away 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout at ../vswitchd/bridge.c:2723 (100% CPU usage) The cited line in bridge.c is: seq_wait(connectivity_seq_get(), connectivity_seqno); Thanks, Ben. ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev