On Mon, Sep 08, 2014 at 05:31:38PM -0700, Krishna Sagiraju wrote: > Flavio, > Appreciate the suggestion. If it was not clear, I was attempting packstack > install originally. It configures and the OpenStack networking related > bridge interfaces. This problem first show up when packstack/puppet tries to > bring up those bridge interfaces and it keeps happening everytime I attempt > to bring down/up the bridge interfaces manually. > > I am going to try the "HOTPLUG=no" options soon.
Ok, let us know if that helps. Anyway, it would be good to know more details of this issue, so please run the small hack below if you have a chance. fbl > > Thanks, > --Krishna/. > > On 9/8/14, 11:59 AM, Flavio Leitner wrote: > >On Mon, Sep 08, 2014 at 09:34:03AM -0700, Gurucharan Shetty wrote: > >>I saw something similar today. In my case, 'force-reload-kmod' would > >>trigger the udev event loop. Adding HOTPLUG=no in > >>/etc/sysconfig/network-scripts/ifcfg-* solved the problem for me. I do > >>not know the exact sequence of events that makes udev act that way. > >> > >>Here is one bug report where the end result is the same but the > >>trigger is different. > >>https://bugzilla.redhat.com/show_bug.cgi?id=855107 > > > >There might be some scenarios where the hotplug event can trigger > >a slave device which depends on the master which depends on something > >that causes a loop. The HOTPLUG=no on ports/slaves usually helps to > >break such loops. > > > >I don't recall anything apart of OpenStack monitoring for openvswitch > >module to load in the system. Even if you load openvswitch, it should > >not create any device by default. However, if you restart the service > >with 'stale' devices in the DB, the hotplug event will be fired. Maybe > >that is confusing the agent. > > > >My suggestion to understand this better would be to change > >/etc/rc.d/init.d/openvswitch to capture additional info about > >which pid is the parent, etc. For example, inserting those > >3 'echo' lines below. > > > >#!/bin/bash > ># > > > >echo "Parent: $(cat /proc/$PPID/cmdline) " >> /tmp/ovs.loop > >echo "pid: $PPID, command line $*" >> /tmp/ovs.loop > >echo "pstree: $(pstree -a -l -n)" >> /tmp/ovs.loop > > > ># openvswitch > ># > ># chkconfig: 2345 09 91 > ># description: Manage Open vSwitch kernel modules and user-space > ># daemons > > > > > >fbl > > > >>On Sun, Aug 24, 2014 at 12:16 PM, Ben Pfaff <[email protected]> wrote: > >>>No. > >>> > >>>I suggest you find out what program is invoking ovs-vsctl and go from > >>>there. > >>> > >>>On Sun, Aug 24, 2014 at 12:00:29PM -0700, Krishna Sagiraju wrote: > >>>>Thanks for the response, Ben. > >>>> > >>>>Actually I did try disabling all OpenStack Icehouse services - nova > >>>>and neutron. But, I still see the messages and events. Restarting > >>>>/etc/init.d/openvswitch service or executing "ovs-ctl > >>>>force-reload-kmod" almost always stops the spew, but that could be > >>>>for a totally different reason. I did say, "almost always" because, > >>>>sometimes force-reload-kmod doesn't help either unless I reboot the > >>>>box. And even after a reboot, I can reproduce the problem > >>>>consistently by just executing "ovs-ctl force-reload-kmod". > >>>> > >>>>Do you have any insights on what other services could be a suspect? > >>>> > >>>>--Krishna/. > >>>> > >>>>On 8/24/14, 11:39 AM, Ben Pfaff wrote: > >>>>>On Sun, Aug 24, 2014 at 06:02:06AM -0700, Krishna Sagiraju wrote: > >>>>>>I am running into a problem with openvswith and would greatly > >>>>>>appreciate help/suggestions from experts. > >>>>>> > >>>>>>I installed openvswitch-1.11.0_8ce28d-1.el6ost.x86_64 on CentOS 6.5 > >>>>>>running Icehouse and as soon as the openvswitch kernel module is > >>>>>>loaded, I get a flood of the following messages in > >>>>>>/var/log/messages: > >>>>>> > >>>>>>Aug 22 15:23:55 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --if-exists del-br br-eth1 > >>>>>>Aug 22 15:23:55 host kernel: device br-eth1 left promiscuous mode > >>>>>>Aug 22 15:23:55 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --may-exist add-br br-eth1 > >>>>>>Aug 22 15:23:55 host kernel: device br-eth1 entered promiscuous mode > >>>>>>Aug 22 15:23:56 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --if-exists del-br br-eth1 > >>>>>>Aug 22 15:23:56 host kernel: device br-eth1 left promiscuous mode > >>>>>>Aug 22 15:23:56 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --may-exist add-br br-eth1 > >>>>>>Aug 22 15:23:56 host kernel: device br-eth1 entered promiscuous mode > >>>>>>Aug 22 15:23:56 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --if-exists del-br br-eth1 > >>>>>>Aug 22 15:23:56 host kernel: device br-eth1 left promiscuous mode > >>>>>>Aug 22 15:23:56 host ovs-vsctl: ovs|00001|vsctl|INFO|Called as > >>>>>>ovs-vsctl -t 10 -- --may-exist add-br br-eth1 > >>>>>> > >>>>>>I traced those messages to udev trying to toggle the interface up > >>>>>>and down rapidly. Here's a sample of the events received by UDEV: > >>>>>> > >>>>>>[root@host ~]# udevadm monitor > >>>>>>monitor will print the received events for: > >>>>>>UDEV - the event which udev sends out after rule processing > >>>>>>KERNEL - the kernel uevent > >>>>>> > >>>>>>KERNEL[1408843389.801627] remove > >>>>>>/devices/virtual/net/br-eth1/queues/rx-0 (queues) > >>>>>>KERNEL[1408843389.801667] remove > >>>>>>/devices/virtual/net/br-eth1/queues/tx-0 (queues) > >>>>>>KERNEL[1408843389.801742] remove /devices/virtual/net/br-eth1 (net) > >>>>>>UDEV [1408843389.820986] remove /devices/virtual/net/br-eth1 (net) > >>>>>>KERNEL[1408843389.870369] add /devices/virtual/net/br-eth1 (net) > >>>>>>KERNEL[1408843389.870391] add > >>>>>>/devices/virtual/net/br-eth1/queues/rx-0 (queues) > >>>>>>KERNEL[1408843389.870401] add > >>>>>>/devices/virtual/net/br-eth1/queues/tx-0 (queues) > >>>>>>UDEV [1408843389.940706] add /devices/virtual/net/br-eth1 (net) > >>>>>>UDEV [1408843389.940944] add > >>>>>>/devices/virtual/net/br-eth1/queues/tx-0 (queues) > >>>>>>UDEV [1408843389.940959] add > >>>>>>/devices/virtual/net/br-eth1/queues/rx-0 (queues) > >>>>>>UDEV [1408843389.941067] remove > >>>>>>/devices/virtual/net/br-eth1/queues/tx-0 (queues) > >>>>>>UDEV [1408843389.941082] remove > >>>>>>/devices/virtual/net/br-eth1/queues/rx-0 (queues) > >>>>>>KERNEL[1408843390.088915] remove > >>>>>>/devices/virtual/net/br-eth1/queues/rx-0 (queues) > >>>>>>KERNEL[1408843390.088941] remove > >>>>>>/devices/virtual/net/br-eth1/queues/tx-0 (queues) > >>>>>>KERNEL[1408843390.089002] remove /devices/virtual/net/br-eth1 (net) > >>>>>>UDEV [1408843390.105143] remove /devices/virtual/net/br-eth1 (net) > >>>>>> > >>>>>> > >>>>>>I would appreciate any advice and/or any workarounds to alleviate > >>>>>>the issue. Once again, thanks for the help in advance > >>>>>The problem is that you have some software creating and deleting devices > >>>>>rapidly. Open vSwitch isn't doing that on its own, so you will have to > >>>>>figure out what is. > >>>_______________________________________________ > >>>discuss mailing list > >>>[email protected] > >>>http://openvswitch.org/mailman/listinfo/discuss > >>_______________________________________________ > >>discuss mailing list > >>[email protected] > >>http://openvswitch.org/mailman/listinfo/discuss > >> > >_______________________________________________ > >discuss mailing list > >[email protected] > >http://openvswitch.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
