Prasanna, I beg to differ with you on the aspect that "Switch anyway is in unknown state of Reconciliation (pushing the flows) fails." Reconciliation is for a particular application only. In a multi-application scenario, it can fail for one application due to various reasons(2 for eg): 1. flow entities got messed up on the way downstairs 2. the datastore for the app was not ready
In that case, its an application issue and we would end up punishing the switch because of it. If you are concerned with the FSM design it could be as UNKNOWN( switch has just connected, and Recon has not started) INPROGRESS( Reconciliation in progress) READY( Reconciliation finished) Any failures *have* to be addressed by the applications. we need to design a framework that gives applications way to register error handlers, and let them handle errors. Since i had pointed out earlier as well that applications are responsible for their data , hence if there is an error, they should handle it and not penalise plugin for it. >>>>>>> 5. This model of reconciliation encourages the applications to be responsible of their own data, hope we are making a conscious choice. [Prasanna]: Yes, plugin should not be worried about application data. >>>>>>>>> And without bundles/ proprietary flags reconciliation is a "*best-effort*" service. Br,shuva On Wed, Jun 7, 2017 at 12:21 PM, Prasanna Huddar < [email protected]> wrote: > Hello Shuva, > > Switch anyway is in unknown state of Reconciliation (pushing the flows) > fails. > > If Reconciliation uses bundles based resync as discussed with Jan, there > is some hope of recovery. > > > > And switches need to handle the Reconciliation failures. > > > > Polling approach is needed because the user should at least will get to > know which applications are not responding. > > - This timer anyway is configurable, if its 0 no polling is > enabled. > > Thanks > > Prasanna > > > > *From:* Shuva Kar [mailto:[email protected]] > *Sent:* Tuesday, June 06, 2017 11:44 PM > *To:* Prasanna Huddar <[email protected]> > *Cc:* Muthukumaran K <[email protected]>; > [email protected]; Kanagasundaram K < > [email protected]> > > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > The first point does make sense. But not sure about the second one though. > Why do we want a timer/polling approach -- it's an extra overhead. We can > have the tasks report their completion to the framework instead (you can > refer to the job coordinator design). > > > > In case of an error/ an exception, log it and move ahead, please *do not > consider disconnecting the switch *that would be messy since you would > leave the switch with stale data (with no way of clean up) and no method of > determining what you have sent and what not. It's better to keep things > simple. > > > Br,shuva > > > > On Tue, Jun 6, 2017 at 9:28 PM, Prasanna Huddar < > [email protected]> wrote: > > Hello Muthu, > > As discussed with Arun and you. > > 1. We would provide priority API through infrastructure. > > 2. After timeout the infra would not disconnect the DPN at least in > Nitrogen release, we would allow the timer to run and keep logging the > application which have not responded back with reconciliation status. But > we might need to give kill command to make reconciliation framework to come > out of monitoring and may be just DC the switch. > > > > Thanks > > Prasanna > > > > *From:* Muthukumaran K > *Sent:* Tuesday, June 06, 2017 3:01 PM > *To:* Prasanna Huddar <[email protected]>; Shuva Kar < > [email protected]> > > > *Cc:* [email protected]; Kanagasundaram K < > [email protected]> > *Subject:* RE: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hi Prasanna, > > > > > > >>>> Applications or Vendors defined Resync algorithm. > > >>>Prioritized execution is again dependent on the Custom algorithm. > > >>> 1. Vendor “A” may opt for prioritized execution(Resync), in that > case he will register one Resync application in which he will give > prioritization provide required functionality. > > >>> 2. Vendor “B” is not bothered about prioritization he just wants > to push certain groups and flows down. > > >>> 3. The framework gives option to plug in the custom algorithms, > without getting into the implementation details of Resync. > > > > Priority may be decided by applications, but the orchestration of tasks > across priorities would have to centralized. Else, there would be two > different applications which can destructively cancel each other’s changes > to config of the switch. > > > > So, what exactly is the contract provided by framework so that > > a) Apps can advertise priorties and related tasks > > b) How framework ensures that the tasks (grouped across apps) can be > executed with only priority-level as key for orchestration > > > Regards > > Muthu > > > > > > > > > > *From:* [email protected] [ > mailto:[email protected] > <[email protected]>] *On Behalf Of *Prasanna > Huddar > *Sent:* Tuesday, June 06, 2017 2:06 PM > *To:* Shuva Kar > *Cc:* [email protected]; Kanagasundaram K > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hello Shuva, > > Comments inline > > Thanks > > Prasanna > > > > *From:* Shuva Kar [mailto:[email protected] > <[email protected]>] > *Sent:* Tuesday, June 06, 2017 10:29 AM > *To:* Prasanna Huddar <[email protected]> > *Cc:* Luis Gomez <[email protected]>; Kanagasundaram K < > [email protected]>; [email protected] > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > Sorry for missing out completely on this thread :) > > > > Have a few queries for your model,Prasanna, from my end : > > > > 1. When you say "vendors" you do mean applications right? If that is so > you would still require a prioritised execution of the tasks for each of > the applications. The apps would register themselves at a particular > priority and submit their task when the come-up. All the framework does is > to take care of executing these tasks. > > Am i correct? > > > > [Prasanna]: > > Applications or Vendors defined Resync algorithm. > > Prioritized execution is again dependent on the Custom algorithm. > > 1. Vendor “A” may opt for prioritized execution(Resync), in that > case he will register one Resync application in which he will give > prioritization provide required functionality. > > 2. Vendor “B” is not bothered about prioritization he just wants to > push certain groups and flows down. > > 3. The framework gives option to plug in the custom algorithms, > without getting into the implementation details of Resync. > > > > 2. Now we would have (in the future :)) tasks that depend on each other- > like ports coming up --> groups pushed on them and flows pointing to them, > a typical tunnelling flow? how would your framework handle such tasks? > > > > [Prasanna]: > > As discussed yesterday the framework waits for certain time for > application to tell the status back. > > 1. Application which are participating in Reconciliation will anyway > handle port up / down separately. > > > > > > 3. How do you define end of a task? Since "some" tasks that are network > resource dependent is a function of time. Now you might say we can wait for > a predefined amount of time, but that is a very very poor design and is a > can of worms and a pandorica of Heisenbugs > > [Prasanna]: > > What are the issues you see? because we are giving the option for > application to inform back the status of reconciliation? > > If they do not inform during certain amount of time than we timeout. > > > > If you can suggest any better approach, please do. > > Even OF uses timeouts for connection monitoring. > > > > 4. I *donot* like the idea of a *switch disconnect* when resync fails - > what is a resync failure ? flows/groups/meters not getting pushed for a > particular device , if thats so the applications *should* be designed to > take care of failures. > > > > Ideally, the framework should have hooks for applications to register for > failures and errors. > > > > [Prasanna]: > > The whole concept is if the resync fails the switch is in unknown > state, and for sure we do not want application to have access to a switch > which is in unknown state. > > > > Current design just does not take care of the situation where > Reconciliation fails, application continues using it even though some flows > are missing, which is still a bad state. > > > > If any UC comes where application can selfheal from unknown switch state, > using some kind intelligence than we might need to enhance the said > infrastructure, which can be done beyond Nitrogen. > > > > 5. This model of reconciliation encourages the applications to be > responsible of their own data, hope we are making a conscious choice. > > [Prasanna]: Yes, plugin should not be worried about application data. > > > > 6. Applications *should* take of group-> flow, group-group dependencies > and not the framework. The FRM/FRS shall be the last piece or default task > that runs for applications that are poor enough to run separate > reconciliation tasks. > > > > [Prasanna]: > > Infrastructure does not get into these details. > > > > My 2 cents here, given that the problem definition here is to have a > well-defined and well-tracked FSM of a switch that is unknown to one that > has been reconcile, it would be grat if we address that first.Also, let's > not push code before the design has been finalised, else we would end up > with tonnes of bugs from a new feature perspective. > > > > [Prasanna]: > > - FSM enhancement can happen in parallel and be plugged in into > this infra, and since we are moving towards bundles based resync. > > - This design is in the community for review, please do review > on priority, since this is targeted for Nitrogen release. > > - We can plan for design review, once we agree on the Design > published. > > > > > Br,shuva > > > > On Tue, May 16, 2017 at 10:37 AM, Prasanna Huddar < > [email protected]> wrote: > > Hello Luis, > > Agree, that’s why the framework, the framework just defines how > reconciliation call flow should happen, but does not define an algorithm. > > > > Vendors can plug-in their Reconciliation algorithm as needed into the > framework > > Thanks > > Prasanna > > > > *From:* Luis Gomez [mailto:[email protected]] > *Sent:* Wednesday, May 10, 2017 10:38 PM > *To:* Prasanna Huddar <[email protected]> > *Cc:* Anil Vishnoi <[email protected]>; Abhijit Kumbhare < > [email protected]>; [email protected]; > Kanagasundaram K <[email protected]> > > > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > I think the main point here is there is no unique way of doing > reconciliation and different applications may also have different > requirements on this, for example: > > > > - Some application may wipe out all switch data in DS when switch goes > down so they can replay everything when switch reconnects. In this case > reconciliation is as simple as sending a delete all when switch connects. > > - Some applications may delete flows but keep groups and meters when > switch is down, so they only need to replay flows after switch reconnects. > In this case reconciliation just need to synchronize groups/meters when > switch connects. > > - Some applications may react as "nothing happens" when switch goes down > so they expect the reconciliation to do all the work of synchronizing > flows/groups/meters. > > > > Unless we can get a very "smart" reconciliation module that can take care > of every possible scenario in an optimal way, I agree we need a framework > to let applications do the reconciliation. > > > > BR/Luis > > > > > > On May 10, 2017, at 8:35 AM, Prasanna Huddar <[email protected]> > wrote: > > > > Hello All, > > > > If you have any more comments, please share. > > > > Thanks > > Prasanna > > > > *From:* Prasanna Huddar > *Sent:* Friday, March 03, 2017 3:37 PM > *To:* Anil Vishnoi <[email protected]>; Abhijit Kumbhare < > [email protected]> > *Cc:* [email protected]; Kanagasundaram K < > [email protected]> > *Subject:* RE: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hello Anil, > > My inputs inline below > > > > Thanks, > > Prasanna > > > > *From:* Anil Vishnoi [mailto:[email protected] <[email protected]> > ] > *Sent:* 02 March 2017 13:22 > *To:* Abhijit Kumbhare <[email protected]> > *Cc:* Prasanna Huddar <[email protected]>; > [email protected]; Kanagasundaram K < > [email protected]> > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hi Prasanna, > > > > I liked the idea of notifying the applications once the node is reconciled > and ready for further configuration. I have few questions about the > proposed framework in the context of single node + current master-slave > implementation: > > > > (1) I think following is not the case, for sure not for FRM, and i believe > for FRM-Sync as well > > Slide-4: > > *"If Reconciliation fails the ODL disconnects the switch, and the whole > process is repeated by Re-sync and application* > > *Applications start acting on OF switch which is in “unknown” state, which > should be avoided."* > > > > [Prasanna]: Ok, Will reword/ update. If Re-sync fails only options is > reconnect the switch. > > > > (2) Slide-8: > > *"Application(s) registering with Reconciliation module is encouraged > since:* > > *Applications would know the right Flows and group which needs to be > replayed with write operation(Add / delete / update).* > > *FRM / FRS would not have application view of flows / group, it would > blindly replay the flows / groups.* > > *Also flows having idle / hard timeout can be gracefully handled by > application rather than FRM / FRS."* > > > > Are we assuming the application won't maintain the correct configuration > in the data store ? > > [Prasanna]: > > No, FRM / FRS would have the right information in data-store, but FRM > / FRS will push all the flows blindly, whereas application can push > optimized set of flows, as covered in couple of points below. > > > > I believe meter and group always need to be programmed before flow, so is > there any other variance of ordering where we want application to do the > reconciliation ? > > [Prasanna]: > > Yes, once FRM / FRS or application registers with Reconciliation > module, the algorithm implemented by application can handle all these > variances, algorithm related info are just reccomendations. > > > > If the flows are installed with idle/hard timeout, we should expect > application to remove it from data store. Unfortunately data store can't > help much here, so application will have to take this burden. > > > > [Prasanna]: > > Agree, but some flow timeouts can happen when Switch is Disconnected > from switch, timeout information is available only in application, using > this info application can re-program the flows as part of re-sync or > exclude it. > > > > So we really keep the clear contract with respect to data store, i don't > really see any reason, why we should leave the node reconciliation to > application. Plugin should do the reconciliation based on the config state > in the data store and once it's done with that reconciliation, it can > notify application to reconcile it's business logic. > > > > Also in case of controller restart, the only state application will have > is in config data store, and until and unless they run their business logic > they can't figure out the correct ordering. And they can't run the business > logic because there is no node advertized to them yet. > > > > [Prasanna]: > > Regarding Plugin implementing the Reconciliation; > > Agree, but plugin should run it as separate bundle, which would be > enabled / disabled through flag. > > 1. Since, OVS might support Bundles , but community plugin cannot > accept all other users to use OVS. > > a. So, we need to give them option to use their own > algorithm(separate bundle) > > b. If future a better algorithm comes up user should be able to > move to it without dependencies. > > c. Reconciliation framework infra should work irrespective of > Reconciliation algorithm. > > > > Regarding node advertisement: > > 1. Agree, node is not advertised to application. > > 2. But as part of Reconciliation call-back / notification, > reconciliation framework calls the call back function or notification with > switch information. > > a. This switch information can be used by application to decide > which flows has to be re-programmed. > > > i. Timer is one option to filter out flows. > > > > (3) If we leave the reconciliation to the applications, and assume that > two applications are using the same node, do we expect that both the > application will have to use the similar kind of reconciliation mechanism? > Assuming one wants to use bundle or one want to use the vanilla > reconciliation mechanism that we have? If not, who will win? If yes, how we > will implement it at the plugin level, because bundle's like features are > provided at the plugin level. So basically i don't see how you can > implement different reconciliation methods without depending on the plugin. > I can see that you can use one method for reconciling specific node, but > then all the application using that node need to use the same method. By > doing that we are basically pushing switch specific reconciliation to the > application level, and in my opinion that's not a good idea. > > > > [Prasanna]: > > We are completely dependent on plugin for all the contracts > to push the flows(meter/flows/groups/etc) down to the switch. > > That is why we say Plugin registers with RPC than invokes Reconciliation > framework. > > > > We do not accept the application to use the same method, > but anything related to algorithm are just recommendation by this framework. > > > > Scenario; > > 1. Consider App1 and App2 have registered to Reconciliation > framework. > > 2. Plugin gives different contracts to program > flows(flows/group/meter/etc). > > a. General flows. > > b. Flows with barrier message. > > c. Flows using bundles. > > 3. App1 and App2 can use any construct of plugin to program the > flows. > > > > We can discuss this more if needed in one of the upcoming calls. > > > > > > (4) slide-9 > > > > "Check if Reconciliation is needed, if yes, update the DS status of the > Switch and go to Step 6" > > > > Given that switch is not yet in data store, what really we want to update > in the data store? Also i didn't find the step 6 in the slides. > > [Prasanna] > > “ Given that switch is not yet in data store, what really we want to > update in the data store? Also i didn't find the step 6 in the slides.” > > > > Typo will be corrected, no DS reference is needed > > > > Sorry, Updated the slide just check, no idea with gdrive conversions. > > > > (5) slide-10 > > "9.*Application(s) would also wait for error from switch, for pre-defined > time." * > > > > Why do we need to wait for pre-define time?, given that the current API-s > do return the future and future will fail if the configuration on switch > fails. > > > > [Prasanna]: > > Ah, than not needed. > > > > (6) slide -10 > > > > *"10.Application(s) would inform the Reconciliation status to > Reconciliation module."* > > How long does plugin wait on the application to notify? > > > > [Prasanna]: > > We can make it configurable. > > > > (7) Proposal says that if any of the application is not able to reconcile > successfully, it should disconnect the device. Assume out of 2 application, > one application is able to reconcile and other one failed, how do > disconnecting the switch will help here ? is there any scenario where we > expect different outcome once switch re-connect? > > > > [Prasanna]: > > > > Are you suggesting: > > When Reconciliation fails, whether to disconnect or not should be left to > solution? > > - Is yes, agree. > > > > (8) If we push the node reconciliation to applications, are we assuming > that they will use the RPC for flow installation during the reconciliation > ? > > [Prasanna]: > > Yes, we are registering the RPC methods with before invoking the > Reconciliation framework. > > But this is not hard requirement, again this is just framework, which > should be flexible for future needs. > > > > > > We can discuss further in one of the calls after Carbon release(or > separate call early, since Carbon is on priority), which is round the > corner. > > > > Thanks > > Anil > > > > On Wed, Feb 22, 2017 at 9:34 AM, Abhijit Kumbhare <[email protected]> > wrote: > > Just to clarify - this reconciliation workflow as well as bundles based > reconciliation being discussed is for the next release (Nitrogen) and not > part of Carbon. The basic bundles support can be code reviewed & merged as > it is a stretch goal item in the Carbon release plan - however it will not > be used by reconciliation or any other application in Carbon. > > > > On Wed, Feb 22, 2017 at 4:35 AM, Prasanna Huddar < > [email protected]> wrote: > > Hello All, > > > > Please review the proposed new Reconciliation workflow and provide your > inputs. > > > > Link to the document(1st draft):https://wiki.opendaylight.org/view/ > OpenDaylight_OpenFlow_Plugin:Reconciliation#Future_Enhancements > > > > Thanks, > > Prasanna > > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > > -- > > Thanks > > Anil > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > >
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
