Br,shuva On Tue, Jun 6, 2017 at 2:05 PM, Prasanna Huddar < [email protected]> wrote:
> Hello Shuva, > > Comments inline > > Thanks > > Prasanna > > > > *From:* Shuva Kar [mailto:[email protected]] > *Sent:* Tuesday, June 06, 2017 10:29 AM > *To:* Prasanna Huddar <[email protected]> > *Cc:* Luis Gomez <[email protected]>; Kanagasundaram K < > [email protected]>; [email protected] > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > Sorry for missing out completely on this thread :) > > > > Have a few queries for your model,Prasanna, from my end : > > > > 1. When you say "vendors" you do mean applications right? If that is so > you would still require a prioritised execution of the tasks for each of > the applications. The apps would register themselves at a particular > priority and submit their task when the come-up. All the framework does is > to take care of executing these tasks. > > Am i correct? > > > > [Prasanna]: > > Applications or Vendors defined Resync algorithm. > > Prioritized execution is again dependent on the Custom algorithm. > > 1. Vendor “A” may opt for prioritized execution(Resync), in that > case he will register one Resync application in which he will give > prioritization provide required functionality. > > 2. Vendor “B” is not bothered about prioritization he just wants to > push certain groups and flows down. > > 3. The framework gives option to plug in the custom algorithms, > without getting into the implementation details of Resync. > [Shuva] the applications might be vendor specific, as in vendor A might opt for GBP/SFC/netvirt while vendor B might opt for only SFC across switches(tunneliing logic in this case).Given that dependency aspects of the flow entities -- flows, groups and meters are of critical importance for all the applications, the algorithm for orchestrating resync/reconciliation is a part of the framework, while the priorities might be centralized > > > 2. Now we would have (in the future :)) tasks that depend on each other- > like ports coming up --> groups pushed on them and flows pointing to them, > a typical tunnelling flow? how would your framework handle such tasks? > > > > [Prasanna]: > > As discussed yesterday the framework waits for certain time for > application to tell the status back. > > 1. Application which are participating in Reconciliation will anyway > handle port up / down separately. > > > [Shuva] correct . But there would be inter-application dependencies that needs to be handled at the framework level , how do you plan to do that ? > > > 3. How do you define end of a task? Since "some" tasks that are network > resource dependent is a function of time. Now you might say we can wait for > a predefined amount of time, but that is a very very poor design and is a > can of worms and a pandorica of Heisenbugs > > [Prasanna]: > > What are the issues you see? because we are giving the option for > application to inform back the status of reconciliation? > > If they do not inform during certain amount of time than we timeout. > > > > If you can suggest any better approach, please do. > > Even OF uses timeouts for connection monitoring. > [Shuva] constantly altering timeouts to ensure that the ports come up within that time. groups/flows pointing to those go missing mostly. Since the entire system is non-deterministic, using time is not an idea. you will definitely end up with entities missing during reconciliation Application has no direct control on the status of reconciliation for a switch. for them its just a set of tasks that's need to done at particular time intervals. Tasks can fail or succeed but those have to handled by applications. Applications have to report that status and that is primarily the driver for tasks being completed. if some tasks fail, let them log and fail, the reconciliation should not be affected by it. Its just that application that has to identify and rectify. In the odd case where a task has been going on for hours, that's probably a bug that the application should take care > > 4. I *donot* like the idea of a *switch disconnect* when resync fails - > what is a resync failure ? flows/groups/meters not getting pushed for a > particular device , if thats so the applications *should* be designed to > take care of failures. > > > > Ideally, the framework should have hooks for applications to register for > failures and errors. > > > > [Prasanna]: > > The whole concept is if the resync fails the switch is in unknown > state, and for sure we do not want application to have access to a switch > which is in unknown state. > > > > Current design just does not take care of the situation where > Reconciliation fails, application continues using it even though some flows > are missing, which is still a bad state. > > > > If any UC comes where application can selfheal from unknown switch state, > using some kind intelligence than we might need to enhance the said > infrastructure, which can be done beyond Nitrogen. > > [shuva] if i take resync failure as control and forwarding plane not in sync with the control plane then errors can happen due to multiple factors -- flow/group ordering mismatch, applications not able to push flows(since the store is not ready),but it can happily happen for other applications , we have seen that in the past (in a multi-application scenario). What we can do is have error handlers that the applications can register and handle the errors accordingly . Why do we want to disconnect the switch and retrigger reconciliation ? Also if the switch does reboot while in between reconciliation we should be able to abort the ongoing reconciliation and start over fresh. Thats a framework req. Reason: currently we donot have this and more often than not we could see the switch coming up midway and having a skewed up configuration. > 5. This model of reconciliation encourages the applications to be > responsible of their own data, hope we are making a conscious choice. > > [Prasanna]: Yes, plugin should not be worried about application data. > [shuva] cool > > > 6. Applications *should* take of group-> flow, group-group dependencies > and not the framework. The FRM/FRS shall be the last piece or default task > that runs for applications that are poor enough to run separate > reconciliation tasks. > > > > [Prasanna]: > > Infrastructure does not get into these details. > [shuva] I agree but the applications should be aware of that, and as we have the ofplugin, we can consciously make it as the last task. > > > My 2 cents here, given that the problem definition here is to have a > well-defined and well-tracked FSM of a switch that is unknown to one that > has been reconcile, it would be grat if we address that first.Also, let's > not push code before the design has been finalised, else we would end up > with tonnes of bugs from a new feature perspective. > > > > [Prasanna]: > > - FSM enhancement can happen in parallel and be plugged in into > this infra, and since we are moving towards bundles based resync. > [shuva] yes . that can come in as a last or final task > - This design is in the community for review, please do review > on priority, since this is targeted for Nitrogen release. > [shuva] request you to please update the wiki based on the discussion > - We can plan for design review, once we agree on the Design > published. > > Br,shuva > > > > On Tue, May 16, 2017 at 10:37 AM, Prasanna Huddar < > [email protected]> wrote: > > Hello Luis, > > Agree, that’s why the framework, the framework just defines how > reconciliation call flow should happen, but does not define an algorithm. > > > > Vendors can plug-in their Reconciliation algorithm as needed into the > framework > > Thanks > > Prasanna > > > > *From:* Luis Gomez [mailto:[email protected]] > *Sent:* Wednesday, May 10, 2017 10:38 PM > *To:* Prasanna Huddar <[email protected]> > *Cc:* Anil Vishnoi <[email protected]>; Abhijit Kumbhare < > [email protected]>; [email protected]; > Kanagasundaram K <[email protected]> > > > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > I think the main point here is there is no unique way of doing > reconciliation and different applications may also have different > requirements on this, for example: > > > > - Some application may wipe out all switch data in DS when switch goes > down so they can replay everything when switch reconnects. In this case > reconciliation is as simple as sending a delete all when switch connects. > > - Some applications may delete flows but keep groups and meters when > switch is down, so they only need to replay flows after switch reconnects. > In this case reconciliation just need to synchronize groups/meters when > switch connects. > > - Some applications may react as "nothing happens" when switch goes down > so they expect the reconciliation to do all the work of synchronizing > flows/groups/meters. > > > > Unless we can get a very "smart" reconciliation module that can take care > of every possible scenario in an optimal way, I agree we need a framework > to let applications do the reconciliation. > > > > BR/Luis > > > > > > On May 10, 2017, at 8:35 AM, Prasanna Huddar <[email protected]> > wrote: > > > > Hello All, > > > > If you have any more comments, please share. > > > > Thanks > > Prasanna > > > > *From:* Prasanna Huddar > *Sent:* Friday, March 03, 2017 3:37 PM > *To:* Anil Vishnoi <[email protected]>; Abhijit Kumbhare < > [email protected]> > *Cc:* [email protected]; Kanagasundaram K < > [email protected]> > *Subject:* RE: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hello Anil, > > My inputs inline below > > > > Thanks, > > Prasanna > > > > *From:* Anil Vishnoi [mailto:[email protected] <[email protected]> > ] > *Sent:* 02 March 2017 13:22 > *To:* Abhijit Kumbhare <[email protected]> > *Cc:* Prasanna Huddar <[email protected]>; > [email protected]; Kanagasundaram K < > [email protected]> > *Subject:* Re: [openflowplugin-dev] Proposed Reconciliation framework > > > > Hi Prasanna, > > > > I liked the idea of notifying the applications once the node is reconciled > and ready for further configuration. I have few questions about the > proposed framework in the context of single node + current master-slave > implementation: > > > > (1) I think following is not the case, for sure not for FRM, and i believe > for FRM-Sync as well > > Slide-4: > > *"If Reconciliation fails the ODL disconnects the switch, and the whole > process is repeated by Re-sync and application* > > *Applications start acting on OF switch which is in “unknown” state, which > should be avoided."* > > > > [Prasanna]: Ok, Will reword/ update. If Re-sync fails only options is > reconnect the switch. > > > > (2) Slide-8: > > *"Application(s) registering with Reconciliation module is encouraged > since:* > > *Applications would know the right Flows and group which needs to be > replayed with write operation(Add / delete / update).* > > *FRM / FRS would not have application view of flows / group, it would > blindly replay the flows / groups.* > > *Also flows having idle / hard timeout can be gracefully handled by > application rather than FRM / FRS."* > > > > Are we assuming the application won't maintain the correct configuration > in the data store ? > > [Prasanna]: > > No, FRM / FRS would have the right information in data-store, but FRM > / FRS will push all the flows blindly, whereas application can push > optimized set of flows, as covered in couple of points below. > > > > I believe meter and group always need to be programmed before flow, so is > there any other variance of ordering where we want application to do the > reconciliation ? > > [Prasanna]: > > Yes, once FRM / FRS or application registers with Reconciliation > module, the algorithm implemented by application can handle all these > variances, algorithm related info are just reccomendations. > > > > If the flows are installed with idle/hard timeout, we should expect > application to remove it from data store. Unfortunately data store can't > help much here, so application will have to take this burden. > > > > [Prasanna]: > > Agree, but some flow timeouts can happen when Switch is Disconnected > from switch, timeout information is available only in application, using > this info application can re-program the flows as part of re-sync or > exclude it. > > > > So we really keep the clear contract with respect to data store, i don't > really see any reason, why we should leave the node reconciliation to > application. Plugin should do the reconciliation based on the config state > in the data store and once it's done with that reconciliation, it can > notify application to reconcile it's business logic. > > > > Also in case of controller restart, the only state application will have > is in config data store, and until and unless they run their business logic > they can't figure out the correct ordering. And they can't run the business > logic because there is no node advertized to them yet. > > > > [Prasanna]: > > Regarding Plugin implementing the Reconciliation; > > Agree, but plugin should run it as separate bundle, which would be > enabled / disabled through flag. > > 1. Since, OVS might support Bundles , but community plugin cannot > accept all other users to use OVS. > > a. So, we need to give them option to use their own > algorithm(separate bundle) > > b. If future a better algorithm comes up user should be able to > move to it without dependencies. > > c. Reconciliation framework infra should work irrespective of > Reconciliation algorithm. > > > > Regarding node advertisement: > > 1. Agree, node is not advertised to application. > > 2. But as part of Reconciliation call-back / notification, > reconciliation framework calls the call back function or notification with > switch information. > > a. This switch information can be used by application to decide > which flows has to be re-programmed. > > > i. Timer is one option to filter out flows. > > > > (3) If we leave the reconciliation to the applications, and assume that > two applications are using the same node, do we expect that both the > application will have to use the similar kind of reconciliation mechanism? > Assuming one wants to use bundle or one want to use the vanilla > reconciliation mechanism that we have? If not, who will win? If yes, how we > will implement it at the plugin level, because bundle's like features are > provided at the plugin level. So basically i don't see how you can > implement different reconciliation methods without depending on the plugin. > I can see that you can use one method for reconciling specific node, but > then all the application using that node need to use the same method. By > doing that we are basically pushing switch specific reconciliation to the > application level, and in my opinion that's not a good idea. > > > > [Prasanna]: > > We are completely dependent on plugin for all the contracts > to push the flows(meter/flows/groups/etc) down to the switch. > > That is why we say Plugin registers with RPC than invokes Reconciliation > framework. > > > > We do not accept the application to use the same method, > but anything related to algorithm are just recommendation by this framework. > > > > Scenario; > > 1. Consider App1 and App2 have registered to Reconciliation > framework. > > 2. Plugin gives different contracts to program > flows(flows/group/meter/etc). > > a. General flows. > > b. Flows with barrier message. > > c. Flows using bundles. > > 3. App1 and App2 can use any construct of plugin to program the > flows. > > > > We can discuss this more if needed in one of the upcoming calls. > > > > > > (4) slide-9 > > > > "Check if Reconciliation is needed, if yes, update the DS status of the > Switch and go to Step 6" > > > > Given that switch is not yet in data store, what really we want to update > in the data store? Also i didn't find the step 6 in the slides. > > [Prasanna] > > “ Given that switch is not yet in data store, what really we want to > update in the data store? Also i didn't find the step 6 in the slides.” > > > > Typo will be corrected, no DS reference is needed > > > > Sorry, Updated the slide just check, no idea with gdrive conversions. > > > > (5) slide-10 > > "9.*Application(s) would also wait for error from switch, for pre-defined > time." * > > > > Why do we need to wait for pre-define time?, given that the current API-s > do return the future and future will fail if the configuration on switch > fails. > > > > [Prasanna]: > > Ah, than not needed. > > > > (6) slide -10 > > > > *"10.Application(s) would inform the Reconciliation status to > Reconciliation module."* > > How long does plugin wait on the application to notify? > > > > [Prasanna]: > > We can make it configurable. > > > > (7) Proposal says that if any of the application is not able to reconcile > successfully, it should disconnect the device. Assume out of 2 application, > one application is able to reconcile and other one failed, how do > disconnecting the switch will help here ? is there any scenario where we > expect different outcome once switch re-connect? > > > > [Prasanna]: > > > > Are you suggesting: > > When Reconciliation fails, whether to disconnect or not should be left to > solution? > > - Is yes, agree. > > > > (8) If we push the node reconciliation to applications, are we assuming > that they will use the RPC for flow installation during the reconciliation > ? > > [Prasanna]: > > Yes, we are registering the RPC methods with before invoking the > Reconciliation framework. > > But this is not hard requirement, again this is just framework, which > should be flexible for future needs. > > > > > > We can discuss further in one of the calls after Carbon release(or > separate call early, since Carbon is on priority), which is round the > corner. > > > > Thanks > > Anil > > > > On Wed, Feb 22, 2017 at 9:34 AM, Abhijit Kumbhare <[email protected]> > wrote: > > Just to clarify - this reconciliation workflow as well as bundles based > reconciliation being discussed is for the next release (Nitrogen) and not > part of Carbon. The basic bundles support can be code reviewed & merged as > it is a stretch goal item in the Carbon release plan - however it will not > be used by reconciliation or any other application in Carbon. > > > > On Wed, Feb 22, 2017 at 4:35 AM, Prasanna Huddar < > [email protected]> wrote: > > Hello All, > > > > Please review the proposed new Reconciliation workflow and provide your > inputs. > > > > Link to the document(1st draft):https://wiki.opendaylight.org/view/OpenD > aylight_OpenFlow_Plugin:Reconciliation#Future_Enhancements > > > > Thanks, > > Prasanna > > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > > -- > > Thanks > > Anil > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > >
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
