OK - as per the decision in the meeting today - it is OK to backport to Boron since this is a bug fix not a feature.
On Thu, Feb 2, 2017 at 12:29 AM, Jozef Bacigál <[email protected]> wrote: > +Abhijit > > Hi Bertrand, > > It is question for community if we wish to make cherry pick this patch > into boron. Let's discuss on today's meeting. > > Jozef > > -----Original Message----- > From: Bertrand Low [mailto:[email protected]] > Sent: Thursday, February 2, 2017 5:20 AM > To: Jozef Bacigál <[email protected]>; Luis Gomez < > [email protected]>; Jamo Luhrsen <[email protected]> > Cc: M Vinoth <[email protected]>; [email protected] > Subject: RE: [openflowplugin-dev] handling of OVS restart > > Hi Jozef, > > I've ported your changes below to my local stable/boron repo: > - All the patches related to 6802. > - https://git.opendaylight.org/gerrit/#/c/51071/ > > With your patches, Bug 7462 (using stable/boron Legacy Netvirt) is no > longer reproduced - the flows are installed correctly in the OVS restart > scenario. Thanks. > > Do you plan to merge the changes over to the stable/boron branch as well? > If not, I can help with that. > > Thanks, > Bertrand > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Bertrand > Low > Sent: January 27, 2017 2:31 PM > To: Luis Gomez <[email protected]>; Jamo Luhrsen <[email protected]> > Cc: M Vinoth <[email protected]>; [email protected] > Subject: Re: [openflowplugin-dev] handling of OVS restart > > Thanks Jozef for the 6802 patch collection, we are testing with that for > the 7462 scenario - will let you know the results. > > Adding a minor delay between 'systemctl stop openvswitch' and 'systemctl > start openvswitch' helps, but 'systemctl restart openvswitch' by default > does not apply a sufficient delay between stopping and starting the process > so it would be good if OF also handles the no-delay use case. > > Thanks, > Bertrand > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Luis Gomez > Sent: January 27, 2017 10:52 AM > To: Jamo Luhrsen <[email protected]> > Cc: M Vinoth <[email protected]>; [email protected] > Subject: Re: [openflowplugin-dev] handling of OVS restart > > It is critical because only workaround is not acceptable :) > > > On Jan 27, 2017, at 10:45 AM, Jamo Luhrsen <[email protected]> wrote: > > > > > > > > On 01/26/2017 11:08 PM, Anil Vishnoi wrote: > >> Bertrand, > >> > >> Yes the reported issue is same as you are facing with the NetVirt. > >> It's happening because of the OVS don't reset/disconnect the > >> connection properly (Don't send FIN/RST packet ) and netty assume > >> that connection is just idle and not disconnected, so it does not > >> disconnect the connection. Meanwhile if device tries to connect back, > controller ignores that connection, because it thinks that device is > already connection. Simple workaround is to add some minor delay (sleep > .5), that will help with this situation. > > > > we can add the delays in our controlled environments to be nice, but > > we can't always count on that in the wild. The workaround in the BZ > > that Luis gave is to restart the controller. Seems like a drastic > > workaround, imo. The bug is marked critical though, so hopefully > something can be done on our side to protect this. > > > > Thanks, > > JamO > > > > > >> Anil > >> > >> On Thu, Jan 26, 2017 at 9:35 PM, Luis Gomez <[email protected] <mailto: > [email protected]>> wrote: > >> > >> Hi Bertrand, > >> > >> The issue I see is reproducible with just 1 controller node and it > is similar in the sense that to it requires an OF > >> channel block (e.g. as a result of an OF agent reboot) followed by > an OF channel restore. > >> > >> https://bugs.opendaylight.org/show_bug.cgi?id=7689 > >> <https://bugs.opendaylight.org/show_bug.cgi?id=7689> > >> > >> BR/Luis > >> > >> > >>> On Jan 26, 2017, at 12:39 PM, Bertrand Low <[email protected] > <mailto:[email protected]>> wrote: > >>> > >>> Hi Luis and Jozef,____ > >>> __ __ > >>> In our ODL cluster setup, if the device issues a “systemctl restart > openvswitch”, or “systemctl stop openvswitch; > >>> systemctl start openvswitch” (no delay between stopping and > starting), we’ve observed the issue of the br-int > >>> controller being unable to reconnect to the OF master node. The OF > master node’s role will be “other” on the device and > >>> it will keep toggling between “CONNECTING” and “BACKOFF” states. > The other OF nodes’ roles remains as “slave”. This > >>> inability to reconnect to the OF master node (and the other nodes > remaining as slaves and not assuming mastership) > >>> appears to persist until ODL is restarted.____ > >>> Luis, is this the “switch hanging” symptom you’ve mentioned?____ > >>> __ __ > >>> In the scenario I’ve described, we have some logs up on bug 7462 ( > https://bugs.opendaylight.org/show_bug.cgi?id=7462 > >>> <https://bugs.opendaylight.org/show_bug.cgi?id=7462>) with DEBUG > enabled for org.opendaylight.openflowplugin. This was > >>> observed while testing legacy netvirt; we will work on providing > updated logs for new netvirt as well.____ > >>> __ __ > >>> Thanks,____ > >>> Bertrand____ > >>> __ __ > >>> *From:* [email protected] <mailto: > [email protected]> > >>> [mailto:[email protected] > >>> <mailto:[email protected]>] *On > Behalf Of *Luis Gomez > >>> *Sent:* January 26, 2017 9:55 AM > >>> *To:* Jozef Bacigál <[email protected] <mailto: > [email protected]>> > >>> *Cc:* M Vinoth <[email protected] <mailto:[email protected]>>; > [email protected] > >>> <mailto:[email protected]> > >>> *Subject:* Re: [openflowplugin-dev] handling of OVS restart____ > >>> __ __ > >>> Jozef and all,____ > >>> __ __ > >>> FYI I have seen sporadic issues of switch hanging in controller > after switch connection is not properly closed, I am > >>> currently working to see if I can consistently reproduce these > issue in the ODL CI.____ > >>> __ __ > >>> BR/Luis____ > >>> __ __ > >>> __ __ > >>> > >>> On Jan 26, 2017, at 12:23 AM, Jozef Bacigál > <[email protected] <mailto:[email protected]>> > >>> wrote:____ > >>> __ __ > >>> May I ask you to run this non-working case again but with ____ > >>> __ __ > >>> org.opendaylight.openflowplugin.impl DEBUG____ > >>> __ __ > >>> and send me the logs please ?____ > >>> __ __ > >>> Thanks____ > >>> __ __ > >>> Jozef____ > >>> __ __ > >>> __ __ > >>> ------------------------------------------------------------ > ----------------------------------------------------------------- > >>> *Od:* M Vinoth <[email protected] <mailto:[email protected]>> > >>> *Odoslané:* 25. januára 2017 15:31 > >>> *Komu:* Jozef Bacigál > >>> *Kópia:* [email protected] <mailto: > [email protected]> > >>> *Predmet:* RE: handling of OVS restart____ > >>> ____ > >>> Hi Jozef,____ > >>> ____ > >>> As you suggested I have checkout the last patch of 6802. And > now I can see all the changes.____ > >>> ____ > >>> And I have tested this changes with new netvirt build.____ > >>> ____ > >>> But still the dump flows are not installed properly. And other > test cases are working fine (1. Node disconnect 2. > >>> Br-int deletion)____ > >>> ____ > >>> Please find my analysis in below for main difference between > working and failing cases,____ > >>> ____ > >>> *Working cases (attached in mail : try_1 document):*____ > >>> ____ > >>> 2017-01-25 21:14:49,043 | INFO | lt-dispatcher-17 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > Entity{type='ovsdb', > >>> id=/(urn:TBD:params:xml:ns:yang:network-topology? > revision=2013-10-21)network-topology/topology/topology[{( > urn:TBD:params:xml:ns:yang:network-topology?revision= > 2013-10-21)topology-id=ovsdb:1}]/node/node[{(urn:TBD: > params:xml:ns:yang:network-topology?revision=2013-10-21) > node-id=ovsdb://uuid/655c51fc-c86b-4cd7-bb0c-7c5d273b457b}]} > >>> has no owner, cleaning up the operational data store____ > >>> ____ > >>> 2017-01-25 21:14:49,574 | INFO | DBConnNotifSer-5 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > InstanceIdentifier > >>> KeyedInstanceIdentifier{targetType=interface > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.topology.Node, > >>> path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns. > yang.network.topology.rev131021.NetworkTopology, > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.Topology[key=TopologyKey > >>> [_topologyId=Uri [_value=ovsdb:1]]], > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.topology.Node[key=NodeKey > >>> [_nodeId=Uri [_value=ovsdb://uuid/655c51fc- > c86b-4cd7-bb0c-7c5d273b457b]]]]} generated for device connection > >>> ConnectionInfo [Remote-address=10.106.138.164, > Remote-port=32850, Local-address10.106.138.251, Local-port=6640, > >>> type=PASSIVE]____ > >>> 2017-01-25 21:14:49,574 | INFO | DBConnNotifSer-5 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > OVSDB entity Entity{type='ovsdb', > >>> id=/(urn:TBD:params:xml:ns:yang:network-topology? > revision=2013-10-21)network-topology/topology/topology[{( > urn:TBD:params:xml:ns:yang:network-topology?revision= > 2013-10-21)topology-id=ovsdb:1}]/node/node[{(urn:TBD: > params:xml:ns:yang:network-topology?revision=2013-10-21) > node-id=ovsdb://uuid/655c51fc-c86b-4cd7-bb0c-7c5d273b457b}]} > >>> is registered for ownership.____ > >>> 2017-01-25 21:14:49,692 | INFO | lt-dispatcher-17 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > handleOwnershipChanged: *this* southbound plugin instance > >>> is an OWNER of the device ConnectionInfo > [Remote-address=10.106.138.164, Remote-port=32850, > >>> Local-address10.106.138.251, Local-port=6640, type=PASSIVE]____ > >>> ____ > >>> *Failing Cases (attached in mail : try_2 document)*____ > >>> ____ > >>> 2017-01-25 21:15:38,751 | INFO | entLoopGroup-5-4 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > Library disconnected PASSIVE from /10.106.138.164:32850 > >>> <http://10.106.138.164:32850> to /10.106.138.251:6640 < > http://10.106.138.251:6640>. Cleaning up the operational > >>> data store____ > >>> ____ > >>> 2017-01-25 21:15:39,283 | INFO | DBConnNotifSer-4 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > InstanceIdentifier > >>> KeyedInstanceIdentifier{targetType=interface > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.topology.Node, > >>> path=[org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns. > yang.network.topology.rev131021.NetworkTopology, > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.Topology[key=TopologyKey > >>> [_topologyId=Uri [_value=ovsdb:1]]], > >>> org.opendaylight.yang.gen.v1.urn.tbd.params.xml.ns.yang. > network.topology.rev131021.network.topology.topology.Node[key=NodeKey > >>> [_nodeId=Uri [_value=ovsdb://uuid/655c51fc- > c86b-4cd7-bb0c-7c5d273b457b]]]]} generated for device connection > >>> ConnectionInfo [Remote-address=10.106.138.164, > Remote-port=32862, Local-address10.106.138.251, Local-port=6640, > >>> type=PASSIVE]____ > >>> 2017-01-25 21:15:39,284 | INFO | DBConnNotifSer-4 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > OVSDB entity Entity{type='ovsdb', > >>> id=/(urn:TBD:params:xml:ns:yang:network-topology? > revision=2013-10-21)network-topology/topology/topology[{( > urn:TBD:params:xml:ns:yang:network-topology?revision= > 2013-10-21)topology-id=ovsdb:1}]/node/node[{(urn:TBD: > params:xml:ns:yang:network-topology?revision=2013-10-21) > node-id=ovsdb://uuid/655c51fc-c86b-4cd7-bb0c-7c5d273b457b}]} > >>> is registered for ownership.____ > >>> 2017-01-25 21:15:39,309 | INFO | lt-dispatcher-17 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > handleOwnershipChanged: no change in ownership for > >>> ConnectionInfo [Remote-address=10.106.138.164, > Remote-port=32862, Local-address10.106.138.251, Local-port=6640, > >>> type=PASSIVE]. Ownership status is : NON-OWNER____ > >>> 2017-01-25 21:15:39,333 | INFO | lt-dispatcher-19 | > OvsdbConnectionManager | 308 - > >>> org.opendaylight.ovsdb.southbound-impl - 1.4.0.SNAPSHOT | > handleOwnershipChanged: *this* southbound plugin instance > >>> is an OWNER of the device ConnectionInfo > [Remote-address=10.106.138.164, Remote-port=32862, > >>> Local-address10.106.138.251, Local-port=6640, type=PASSIVE]____ > >>> ____ > >>> *Example:*____ > >>> [stack@control devstack]$ sudo systemctl stop openvswitch____ > >>> [stack@control devstack]$ sudo systemctl start openvswitch____ > >>> [stack@control devstack]$____ > >>> [stack@control devstack]$____ > >>> [stack@control devstack]$ sudo ovs-ofctl -O OpenFlow13 > dump-flows br-int____ > >>> OFPST_FLOW reply (OF1.3) (xid=0x2):____ > >>> ____ > >>> I will test more scenario’s and then I will let you know > further.____ > >>> ____ > >>> Regards,____ > >>> Vinoth____ > >>> ____ > >>> ____ > >>> *From:* Jozef Bacigál [mailto:[email protected] > <mailto:[email protected]>] > >>> *Sent:* 25 January 2017 15:53 > >>> *To:* M Vinoth <[email protected] <mailto:[email protected]>> > >>> *Cc:* [email protected] <mailto: > [email protected]> > >>> *Subject:* Re: handling of OVS restart____ > >>> ____ > >>> Hi Vinoth,____ > >>> ____ > >>> yes the patch for bug 6082 should prevent this kind of issues. > We trying to make some test with stop/start > >>> connection without delay on this patch. I'll let you know if we > were able to reproduce the issue. ____ > >>> ____ > >>> Anyway as I said this patch shall hold all information about > switches for few seconds/milliseconds (configurable) > >>> and tha just replace broken connection with correct one without > any changes in behavior.____ > >>> ____ > >>> I will let you know.____ > >>> ____ > >>> Jozef____ > >>> ____ > >>> ____ > >>> ------------------------------------------------------------ > ----------------------------------------------------------------- > >>> *Od:* M Vinoth <[email protected] <mailto:[email protected]>> > >>> *Odoslané:* 24. januára 2017 13:40 > >>> *Komu:* Jozef Bacigál > >>> *Kópia:* [email protected] <mailto: > [email protected]> > >>> *Predmet:* FW: handling of OVS restart____ > >>> ____ > >>> Hi Jozef,____ > >>> ____ > >>> I have seen your patch for bug:6082., and observe that > implemented related to the below issues.____ > >>> https://git.opendaylight.org/gerrit/#/q/topic:%22Bug+6802%22 > >>> <https://git.opendaylight.org/gerrit/#/q/topic:%22Bug+6802%22 > >____ > >>> ____ > >>> The issue is when OVS stop and start immediately without delay, > that time br-int flows were not available. And node > >>> also disconnect.____ > >>> And I have notice that life cycle service and Device context > service is not properly invoked when failed cases.____ > >>> In below mail I have highlighted comparatively with working as > well as failed cases.____ > >>> Whether the below issue is addressed in your patch.____ > >>> Please validate the below mail and share your inputs on > this.____ > >>> Thanks.____ > >>> ____ > >>> Regards,____ > >>> Vinoth____ > >>> *From:* [email protected] > >>> <mailto:[email protected]> [mailto: > [email protected] > >>> <mailto:[email protected]>] *On > Behalf Of *M Vinoth > >>> *Sent:* 24 January 2017 15:33 > >>> *To:* Bertrand Low <[email protected] <mailto: > [email protected]>>; Vishal Thapar <[email protected] > >>> <mailto:[email protected]>>; netvirt-dev@lists. > opendaylight.org <mailto:[email protected]> > >>> *Cc:* [email protected] > >>> <mailto:[email protected]>; > [email protected] > >>> <mailto:[email protected]> > >>> *Subject:* Re: [netvirt-dev] handling of OVS restart____ > >>> ____ > >>> Hi Vishal,____ > >>> ____ > >>> I have tested the following behaviour in new netvirt in 3-node > cluster,____ > >>> ____ > >>> The following commands requested without delay,____ > >>> [stack@control devstack]$ sudo systemctl stop openvswitch____ > >>> [stack@control devstack]$ sudo systemctl start openvswitch____ > >>> ____ > >>> And found that one of the node is disconnected from 3-node > cluster.____ > >>> ____ > >>> [stack@control devstack]$ sudo ovs-vsctl show____ > >>> 655c51fc-c86b-4cd7-bb0c-7c5d273b457b____ > >>> Manager "tcp:10.106.138.251:6640 < > http://10.106.138.251:6640>"____ > >>> is_connected: true____ > >>> Manager "tcp:10.106.138.248:6640 < > http://10.106.138.248:6640>"____ > >>> is_connected: true____ > >>> Manager "tcp:10.106.138.244:6640 < > http://10.106.138.244:6640>"____ > >>> is_connected: true____ > >>> Bridge br-int____ > >>> Controller "tcp:10.106.138.251:6653 < > http://10.106.138.251:6653>"____ > >>> Controller "tcp:10.106.138.248:6653 < > http://10.106.138.248:6653>"____ > >>> is_connected: true____ > >>> Controller "tcp:10.106.138.244:6653 < > http://10.106.138.244:6653>"____ > >>> is_connected: true____ > >>> fail_mode: secure____ > >>> Port br-int____ > >>> Interface br-int____ > >>> type: internal____ > >>> ovs_version: "2.5.0"____ > >>> ____ > >>> Verified that the internal bridge default flows are not > available.____ > >>> ____ > >>> [stack@control devstack]$ sudo ovs-ofctl -O OpenFlow13 > dump-flows br-int____ > >>> OFPST_FLOW reply (OF1.3) (xid=0x2):____ > >>> [stack@control devstack]$____ > >>> ____ > >>> The same behaviour in legacy netvirt too, but additionally one > more cases were failed in legacy that br-int is not > >>> available.____ > >>> ____ > >>> I will share the following execution flows based on the log > which is analysed in legacy netvirt.____ > >>> *_Working cases execution flows,_*____ > >>> *ß**------------------ ------ Stopping the service > ------------------------------------------**à** **(A)*____ > >>> SystemNotificationsListenerImpl - ConnectionEvent: Connection > closed by device, Device:/10.106.138.164:59630 > >>> <http://10.106.138.164:59630>, NodeId:openflow: > 35760623087526____ > >>> OvsdbConnectionService - Connection closed ConnectionInfo____ > >>> OvsdbConnectionManager - Library disconnected PASSIVE from / > 10.106.138.164:57042 <http://10.106.138.164:57042> to > >>> /10.106.138.251:6640 <http://10.106.138.251:6640>. Cleaning up > the operational data store____ > >>> OvsdbConnectionManager - handleOwnershipChanged: *this* > instance is elected as an owner of the device____ > >>> OvsdbConnectionManager - has no owner, cleaning up the > operational data store____ > >>> *ß**------------------ ------ Deleting the node and bridge > ------------------------------------------**à** **(B) *____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with > action : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with > action : DELETE for the OVS node :____ > >>> SouthboundImpl - deleteBridge node:____ > >>> SouthboundHandler - processOvsdb Node Delete :____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> *ß**------------------ ------ Setting Up the cluster service > ------------------------------------------**à** **(C)*____ > >>> ConnectionAdapterImpl - Hello received / branch____ > >>> DeviceManagerImpl - ConnectionEvent: Device connected to > controller,____ > >>> SalRoleServiceImpl - SetRole called with input:SetRoleInput____ > >>> SalRoleServiceImpl - Requesting state change to BECOMESLAVE____ > >>> SalRoleServiceImpl - RoleChangeTask called on > device:openflow:35760623087526 OFPRole:BECOMESLAVE____ > >>> RoleService - getGenerationIdFromDevice called for device: > openflow:35760623087526____ > >>> LifecycleServiceImpl - Registered clustering MASTER services > for node openflow:35760623087526____ > >>> RoleService - submitRoleChange called for device:Uri > [_value=openflow:35760623087526], role:BECOMESLAVE____ > >>> LoggingHandler - RECEIVED: [id: 0x2cd517d7, L:/ > 10.106.138.251:6640 <http://10.106.138.251:6640> - > >>> R:/10.106.138.164:57066 <http://10.106.138.164:57066>]____ > >>> RoleService - submitRoleChange onSuccess for device:Uri > [_value=openflow:35760623087526], role:BECOMESLAVE____ > >>> OvsdbConnectionManager - Library connected PASSIVE from / > 10.106.138.164:57066 <http://10.106.138.164:57066> to > >>> /10.106.138.251:6640 <http://10.106.138.251:6640>____ > >>> OvsdbConnectionManager - OVSDB Connection from / > 10.106.138.164:57066 <http://10.106.138.164:57066>____ > >>> OvsdbConnectionManager - generated for device connection > ConnectionInfo____ > >>> OvsdbConnectionManager - OVSDB entity Entity, is registered for > ownership.____ > >>> LifecycleServiceImpl - Starting clustering MASTER services for > node openflow:35760623087526____ > >>> DeviceContextImpl - Starting device context cluster services > for node openflow:35760623087526____ > >>> OvsdbConnectionManager - handleOwnershipChanged: *this* > southbound plugin instance is an OWNER of the device > >>> ConnectionInfo____ > >>> OvsdbConnectionInstance - Monitoring database: Open_vSwitch____ > >>> DeviceInitializationUtils - IP address of switch is: / > 10.106.138.164:59640 <http://10.106.138.164:59640>____ > >>> DeviceInitializationUtils - Static node Uri > >>> [_value=openflow:35760623087526] > info: OFPMPMETERFEATURES collected____ > >>> StringValueObjectFactory - Instantiated factory for class > >>> org.opendaylight.yang.gen.v1.urn.ietf.params.xml.ns.yang. > ietf.yang.types.rev130715.MacAddress____ > >>> DeviceInitializationUtils - Static node Uri > >>> [_value=openflow:35760623087526] > info: OFPMPPORTDESC collected____ > >>> SalRoleServiceImpl - SetRole called with input:____ > >>> SalRoleServiceImpl - Requesting state change to BECOMEMASTER____ > >>> RoleService - getGenerationIdFromDevice called for device: > openflow:35760623087526____ > >>> StatisticsContextImpl - Starting statistics context cluster > services for node openflow:35760623087526____ > >>> RoleService - submitRoleChange called for device:Uri > [_value=openflow:35760623087526], role:BECOMEMASTER____ > >>> RoleService - submitRoleChange onSuccess for device:Uri > [_value=openflow:35760623087526], role:BECOMEMASTER____ > >>> StatisticsManagerImpl - Scheduling statistics poll for device: > Uri [_value=openflow:35760623087526]____ > >>> *<-------- Updating the internal bridge > ----------------------------**à** **(D)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : ADD for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Create :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : ADD for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with > action : ADD for the OVS node :____ > >>> SouthboundHandler - notifyNode : action: ADD, Node :____ > >>> SouthboundHandler - notifyNode : action: ADD, Node :____ > >>> OF13Provider - initializeOFFlowRules: bridgeName: br-int____ > >>> ControllerUpdateCommand - Register ODL controllers : > ControllerEntry____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Update :____ > >>> NodeCacheManagerImpl - oldDpid == 0, newDpid == 0____ > >>> SouthboundHandler - notifyNode : action: UPDATE, Node :____ > >>> ____ > >>> *_Failure cases execution flows (Without delay),_*____ > >>> * *____ > >>> *ß**------------------ ------ Stopping the service > ------------------------------------------**à** **(A)*____ > >>> SystemNotificationsListenerImpl - ConnectionEvent: Connection > closed by device, Device:/10.106.138.164:59640 > >>> <http://10.106.138.164:59640>, NodeId:openflow: > 35760623087526____ > >>> OvsdbConnectionService - Connection closed ConnectionInfo____ > >>> OvsdbConnectionManager - Library disconnected PASSIVE from / > 10.106.138.164:57066 <http://10.106.138.164:57066> to > >>> /10.106.138.251:6640 <http://10.106.138.251:6640>. Cleaning up > the operational data store____ > >>> OvsdbConnectionManager - has no owner, cleaning up the > operational data store____ > >>> *ß**------------------ ------ Setting Up the cluster service > ------------------------------------------**à** **(C)*____ > >>> ConnectionAdapterImpl - Hello received / branch____ > >>> DeviceManagerImpl - ConnectionEvent: Device connected to > controller, Device:/10.106.138.164:59658 > >>> <http://10.106.138.164:59658>, NodeId:Uri [_value=openflow: > 35760623087526]____ > >>> SalRoleServiceImpl - SetRole called with input:SetRoleInput____ > >>> SalRoleServiceImpl - Requesting state change to BECOMESLAVE____ > >>> SalRoleServiceImpl - RoleChangeTask called on > device:openflow:35760623087526 OFPRole:BECOMESLAVE____ > >>> RoleService - submitRoleChange called for device:Uri > [_value=openflow:35760623087526], role:BECOMESLAVE____ > >>> RoleService - submitRoleChange onSuccess for device:Uri > [_value=openflow:35760623087526], role:BECOMESLAVE____ > >>> LoggingHandler - RECEIVED: [id: 0x2610f6bd, L:/ > 10.106.138.251:6640 <http://10.106.138.251:6640> - > >>> R:/10.106.138.164:57084 <http://10.106.138.164:57084>]____ > >>> OvsdbConnectionManager - Library connected PASSIVE from / > 10.106.138.164:57084 <http://10.106.138.164:57084> to > >>> /10.106.138.251:6640 <http://10.106.138.251:6640>____ > >>> OvsdbConnectionManager - OVSDB Connection from / > 10.106.138.164:57084 <http://10.106.138.164:57084>____ > >>> OvsdbConnectionManager - generated for device connection > ConnectionInfo____ > >>> OvsdbConnectionManager - OVSDB entity, is registered for > ownership.____ > >>> OvsdbConnectionManager - handleOwnershipChanged: *this* > southbound plugin instance is an OWNER of the device > >>> ConnectionInfo____ > >>> OvsdbConnectionInstance - Monitoring database: Open_vSwitch____ > >>> *ß**------------------ ------ Deleting the node and bridge > ------------------------------------------**à** **(B)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : DELETE for the OVS node :____ > >>> SouthboundImpl - deleteBridge node:____ > >>> SouthboundHandler - processOvsdb Node Delete :____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> ControllerUpdateCommand - Register ODL controllers : {} > bridges detail : {}____ > >>> ProtocolRemovedCommand - Removed ProtocolEntry : OpenFlow13 for > OVSDB Bridge : null____ > >>> BridgeRemovedCommand - Bridge Deleted:____ > >>> SystemNotificationsListenerImpl - ConnectionEvent: Connection > closed by device, Device:/10.106.138.164:59658 > >>> <http://10.106.138.164:59658>, NodeId:openflow: > 35760623087526____ > >>> OvsdbPortRemoveCommand - Bridge not found for port____ > >>> *<-------- Updating the internal bridge > ----------------------------**à** **(D)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : ADD for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : ADD for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : ADD for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Create :____ > >>> SouthboundImpl - addBridge: node:____ > >>> SouthboundImpl - addBridge: result: true____ > >>> SouthboundHandler - notifyNode : action: ADD, Node :____ > >>> SouthboundHandler - notifyNode : action: ADD, Node :____ > >>> OF13Provider - initializeOFFlowRules: bridgeName: br-int____ > >>> BridgeUpdateCommand - Added ovsdb Bridge name: OvsdbBridgeName > [_value=br-int] uuid: null____ > >>> ControllerUpdateCommand - Register ODL controllers : > ControllerEntry____ > >>> ProtocolUpdateCommand - Updated ProtocolEntry : OpenFlow13 for > OVSDB Bridge : br-int____ > >>> *ß**------------------ ------ Setting Up the cluster service > ------------------------------------------**à** **(C)*____ > >>> ConnectionAdapterImpl - Hello received / branch____ > >>> DeviceManagerImpl - ConnectionEvent: Device connected to > controller, Device:/10.106.138.164:59670 > >>> <http://10.106.138.164:59670>, NodeId:Uri [_value=openflow: > 35760623087526]____ > >>> SalRoleServiceImpl - Requesting state change to BECOMESLAVE____ > >>> SalRoleServiceImpl - RoleChangeTask called on > device:openflow:35760623087526 OFPRole:BECOMESLAVE____ > >>> RoleService - getGenerationIdFromDevice called for device: > openflow:35760623087526____ > >>> LifecycleServiceImpl - Registered clustering MASTER services > for node openflow:35760623087526____ > >>> RoleService - submitRoleChange called for device:Uri > [_value=openflow:35760623087526], role:BECOMESLAVE____ > >>> RoleService - submitRoleChange onSuccess for device:____ > >>> *ß**------------------ ------ Deleting the node and bridge > ------------------------------------------**à** **(B)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Update :____ > >>> NodeCacheManagerImpl - oldDpid == 0, newDpid == 0____ > >>> SouthboundHandler - notifyNode : action: UPDATE, Node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : DELETE for the OVS node :____ > >>> SouthboundImpl - deleteBridge node:____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> *<-------- Updating the internal bridge > ----------------------------**à** **(D)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : ADD for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : ADD for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Update :____ > >>> NodeCacheManagerImpl - oldDpid == 0, newDpid == 0____ > >>> SouthboundHandler - notifyNode : action: UPDATE, Node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : UPDATE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - notifyNode : action: ADD, Node :____ > >>> OF13Provider - initializeOFFlowRules: bridgeName: br-int____ > >>> ControllerUpdateCommand - Register ODL controllers : {} > bridges detail : {}____ > >>> ProtocolRemovedCommand - Removed ProtocolEntry : OpenFlow13 for > OVSDB Bridge : null____ > >>> BridgeRemovedCommand - Bridge Deleted:____ > >>> OvsdbPortRemoveCommand - Bridge not found for port Port :____ > >>> SystemNotificationsListenerImpl - ConnectionEvent: Connection > closed by device, Device:/10.106.138.164:59670 > >>> <http://10.106.138.164:59670>, NodeId:openflow: > 35760623087526____ > >>> *ß**------------------ ------ Deleting the node and bridge > ------------------------------------------**à** **(B)*____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Update :____ > >>> NodeCacheManagerImpl - oldDpid == 0, newDpid == 0____ > >>> SouthboundHandler - notifyNode : action: UPDATE, Node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : PORT with action > : DELETE for the OVS node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : BRIDGE with > action : DELETE for the OVS node :____ > >>> SouthboundImpl - deleteBridge node:____ > >>> SouthboundHandler - notifyNode : action: DELETE, Node :____ > >>> SouthboundHandler - Received ovsdbUpdate for : NODE with action > : UPDATE for the OVS node :____ > >>> SouthboundHandler - processOvsdb Node Update :____ > >>> NodeCacheManagerImpl - oldDpid == 0, newDpid == 0____ > >>> SouthboundHandler - notifyNode : action: UPDATE, Node :____ > >>> ____ > >>> I have observe that the above working cases are properly > executing the life cycle service and device context for > >>> cluster service, it means that properly maintain the context > information.____ > >>> ____ > >>> Please share your inputs.____ > >>> ____ > >>> Regards,____ > >>> Vinoth____ > >>> ____ > >>> ____ > >>> *From:* [email protected] > >>> <mailto:[email protected]> [mailto: > [email protected] > >>> <mailto:[email protected]>] *On > Behalf Of *Bertrand Low > >>> *Sent:* 23 January 2017 11:02 > >>> *To:* Vishal Thapar <[email protected] > >>> <mailto:[email protected]>>; netvirt-dev@lists. > opendaylight.org <mailto:[email protected]> > >>> *Subject:* Re: [netvirt-dev] handling of OVS restart____ > >>> ____ > >>> Thanks Vishal,____ > >>> ____ > >>> The bridge deletion upon disconnect does seem like a potential > issue in legacy netvirt. I’ll do some testing on > >>> this.____ > >>> ____ > >>> Cheers,____ > >>> Bertrand____ > >>> ____ > >>> *From:* Vishal Thapar [mailto:[email protected] > <mailto:[email protected]>] > >>> *Sent:* January 22, 2017 7:59 PM > >>> *To:* Bertrand Low <[email protected] <mailto: > [email protected]>>; [email protected] > >>> <mailto:[email protected]> > >>> *Subject:* RE: handling of OVS restart____ > >>> ____ > >>> Hi Bertrand,____ > >>> ____ > >>> Can you try with new netvirt? I am not that familiar with > legacy netvirt but new netvirt doesn’t have code to > >>> delete bridge on disconnect. https://bugs.opendaylight.org/ > show_bug.cgi?id=7488 > >>> <https://bugs.opendaylight.org/show_bug.cgi?id=7488> fixed an > issue with dpnid changing which was impacting cluster > >>> setups.____ > >>> ____ > >>> So, for now I’d say that this issue is not seen in new netvirt > unless proven otherwise J____ > >>> ____ > >>> Regards,____ > >>> Vishal.____ > >>> ____ > >>> *From:* [email protected] > >>> <mailto:[email protected]> [mailto: > [email protected] > >>> <mailto:[email protected]>] *On > Behalf Of *Bertrand Low > >>> *Sent:* 23 January 2017 08:36 > >>> *To:* [email protected] <mailto: > [email protected]> > >>> *Subject:* [netvirt-dev] handling of OVS restart____ > >>> ____ > >>> Hi all,____ > >>> ____ > >>> Has anyone run into issues with ODL handling OVS restart?____ > >>> ____ > >>> The scenario is like this:____ > >>> ____ > >>> 1) Connect device to an ODL cluster (i.e. at least 3), see > that br-int and the pipeline flows are installed > >>> successfully____ > >>> 2) Execute “restart openvswitch” on the device, or execute > “stop openvswitch; start openvswitch” (i.e. no > >>> delay between starting and stopping openvswitch)____ > >>> 3) Once the device reconnects to ODL, there are two > failure scenarios observed:____ > >>> a. There is no br-int (Bug 7461)____ > >>> 1. A connection flap is observed for br-int, sometimes > with br-int assigned a new dpnid____ > >>> OR____ > >>> b. br-int is installed, but without any pipeline flows > (Bug 7462)____ > >>> ____ > >>> Seems to me like the “restart openvswitch” use case should be > handled by ODL – please let me know if my > >>> understanding is incorrect.____ > >>> ____ > >>> I’ve been testing with legacy netvirt but this timing issue may > be relevant to new netvirt as well.____ > >>> If anyone has any insight on new netvirt’s handling of OVS > restart, please share. Thanks!____ > >>> ____ > >>> Bertrand____ > >>> > >>> > >>> ::DISCLAIMER:: > >>> ------------------------------------------------------------ > ------------------------------------------------------------ > ----------------------------____ > >>> The contents of this e-mail and any attachment(s) are > confidential and intended for the named recipient(s) only. > >>> E-mail transmission is not guaranteed to be secure or > error-free as information could be intercepted, corrupted, > >>> lost, destroyed, arrive late or incomplete, or may contain > viruses in transmission. The e mail and its contents > >>> (with or without referred errors) shall therefore not attach > any liability on the originator or HCL or its affiliates. > >>> Views or opinions, if any, presented in this email are solely > those of the author and may not necessarily reflect the > >>> views or opinions of HCL or its affiliates. Any form of > reproduction, dissemination, copying, disclosure, > >>> modification, > >>> distribution and / or publication of this message without the > prior written consent of authorized representative of > >>> HCL is strictly prohibited. If you have received this email in > error please delete it and notify the sender > >>> immediately. > >>> Before opening any email and/or attachments, please check them > for viruses and other defects.____ > >>> ------------------------------------------------------------ > ------------------------------------------------------------ > ----------------------------____ > >>> ____ > >>> Jozef*Bacigál*____ > >>> Software Engineer____ > >>> > >>> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia > >>> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / Slovakia > >>> +421 908 766 972 <tel:+421%20908%20766%20972> / > [email protected] <mailto:[email protected]> > >>> reception: +421 2 206 65 114 / www.pantheon.tech < > http://www.pantheon.tech/>____ > >>> logo____ > >>> ____ > >>> ____ > >>> Jozef*Bacigál*____ > >>> Software Engineer____ > >>> > >>> Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia > >>> R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / Slovakia > >>> +421 908 766 972 <tel:+421%20908%20766%20972> / > [email protected] <mailto:[email protected]> > >>> reception: +421 2 206 65 114 / www.pantheon.tech < > http://www.pantheon.tech/>____ > >>> logo____ > >>> ____ > >>> _______________________________________________ > >>> openflowplugin-dev mailing list > >>> [email protected] > >>> <mailto:[email protected]> > >>> > >>> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > >>> > >>> <https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev> > >> > >> > >> _______________________________________________ > >> openflowplugin-dev mailing list > >> [email protected] <mailto: > [email protected]> > >> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > >> > >> <https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev> > >> > >> > >> > >> > >> -- > >> Thanks > >> Anil > >> > >> > >> _______________________________________________ > >> openflowplugin-dev mailing list > >> [email protected] > >> https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > >> > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > JozefBacigál > Software Engineer > > Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia > R&D centrum / Janka Kráľa 9 / 974 01 Banská Bystrica / Slovakia > +421 908 766 972 / [email protected] > reception: +421 2 206 65 114 / www.pantheon.tech > > [logo] > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev >
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
