Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)

2014-09-30 Thread Alex Wang
Found the issue, I have idea of unit testing it, will experiment on it,

On Tue, Sep 30, 2014 at 11:11 AM, Ben Pfaff  wrote:

> It's not the first time we've had this kind of bug.  Last time, I
> tried to think of a good unit test, but did not succeed.
>
> On Tue, Sep 30, 2014 at 10:32:15AM -0700, Alex Wang wrote:
> > I can reproduce it, working on a fix,
> >
> > Thx for reporting~
> >
> > On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff  wrote:
> >
> > > There's still a 100% CPU bug in ovs-vswitchd.  Run "make sandbox", then
> > > run a second ovs-vswitchd inside the subshell (just run a bare
> > > "ovs-vswitchd" without any parameters).  You get output like this:
> > >
> > > 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on
> > > NUMA node 0
> > > 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes
> and 8
> > > CPU cores
> > >
> > >
> 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> > > connecting...
> > >
> > >
> 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> > > connected
> > > 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process
> is
> > > running, disabling this process (pid 13812) until it goes away
> > > 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > > 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms
> timeout
> > > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > >
> > > The cited line in bridge.c is:
> > >
> > > seq_wait(connectivity_seq_get(), connectivity_seqno);
> > >
> > > Thanks,
> > >
> > > Ben.
> > >
>
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev


Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)

2014-09-30 Thread Ben Pfaff
It's not the first time we've had this kind of bug.  Last time, I
tried to think of a good unit test, but did not succeed.

On Tue, Sep 30, 2014 at 10:32:15AM -0700, Alex Wang wrote:
> I can reproduce it, working on a fix,
> 
> Thx for reporting~
> 
> On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff  wrote:
> 
> > There's still a 100% CPU bug in ovs-vswitchd.  Run "make sandbox", then
> > run a second ovs-vswitchd inside the subshell (just run a bare
> > "ovs-vswitchd" without any parameters).  You get output like this:
> >
> > 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on
> > NUMA node 0
> > 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8
> > CPU cores
> >
> > 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> > connecting...
> >
> > 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> > connected
> > 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is
> > running, disabling this process (pid 13812) until it goes away
> > 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> > 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout
> > at ../vswitchd/bridge.c:2723 (100% CPU usage)
> >
> > The cited line in bridge.c is:
> >
> > seq_wait(connectivity_seq_get(), connectivity_seqno);
> >
> > Thanks,
> >
> > Ben.
> >
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev


Re: [ovs-dev] 100% CPU in ovs-vswitchd (again)

2014-09-30 Thread Alex Wang
I can reproduce it, working on a fix,

Thx for reporting~

On Tue, Sep 30, 2014 at 9:37 AM, Ben Pfaff  wrote:

> There's still a 100% CPU bug in ovs-vswitchd.  Run "make sandbox", then
> run a second ovs-vswitchd inside the subshell (just run a bare
> "ovs-vswitchd" without any parameters).  You get output like this:
>
> 2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on
> NUMA node 0
> 2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8
> CPU cores
>
> 2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> connecting...
>
> 2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
> connected
> 2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is
> running, disabling this process (pid 13812) until it goes away
> 2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
> 2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout
> at ../vswitchd/bridge.c:2723 (100% CPU usage)
>
> The cited line in bridge.c is:
>
> seq_wait(connectivity_seq_get(), connectivity_seqno);
>
> Thanks,
>
> Ben.
>
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev


[ovs-dev] 100% CPU in ovs-vswitchd (again)

2014-09-30 Thread Ben Pfaff
There's still a 100% CPU bug in ovs-vswitchd.  Run "make sandbox", then
run a second ovs-vswitchd inside the subshell (just run a bare
"ovs-vswitchd" without any parameters).  You get output like this:

2014-09-30T16:33:45Z|1|ovs_numa|INFO|Discovered 8 CPU cores on NUMA 
node 0
2014-09-30T16:33:45Z|2|ovs_numa|INFO|Discovered 1 NUMA nodes and 8 CPU 
cores

2014-09-30T16:33:45Z|3|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
 connecting...

2014-09-30T16:33:45Z|4|reconnect|INFO|unix:/home/blp/nicira/ovs/tutorial/sandbox/db.sock:
 connected
2014-09-30T16:33:45Z|5|bridge|ERR|another ovs-vswitchd process is 
running, disabling this process (pid 13812) until it goes away
2014-09-30T16:33:48Z|6|poll_loop|INFO|wakeup due to 1-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|7|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|8|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|9|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00010|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00011|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00012|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00013|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00014|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)
2014-09-30T16:33:48Z|00015|poll_loop|INFO|wakeup due to 0-ms timeout at 
../vswitchd/bridge.c:2723 (100% CPU usage)

The cited line in bridge.c is:

seq_wait(connectivity_seq_get(), connectivity_seqno);

Thanks,

Ben.
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev