Hi Muthu, Sorry for delayed response. Please see inline ..
On Fri, Feb 23, 2018 at 2:44 AM, Muthukumaran K <[email protected] > wrote: > Hi all, > > > > @Arun, @Gobi, please feel free to add anything which I might have missed > out from our discussions > > > > Context : > > ======= > > During last OFPlugin weekly call, we were discussing about how > bundle-based resync flow would behave in upgrade context. In normal resync > scenarios – ie. a) switch reconnecting and/or b) controller reboots (and > subsequent switch reconnects), flows and groups which can be dispatched via > resync bundle is sourced from the inventory Config DS. In case of upgrade, > inventory Config DS would be completely nuked and hence a different > handling is required. During this discussion, a concern was raised as to > why OFPlugin should become extensively cognizant of “upgrade” as a special > case. The thought process was more to expose the bundle service generically > from OFPlugin’s end so that upgrade-orchestration becomes a higher level > functionality without switching behaviors from with OFP > > > > So, following this mail boostraps the discussions with following > approaches and choosing an approach which is less invasive. We can continue > to discuss the merits, demerits and fine-tune the approach to arrive at an > agreeable mechanism. > > > > Premises : > > ======= > 1) Bundle construct is currently used in Reconciliation and Resync context > 2) The objective is to expose bundle as a generic construct for > applications to use whenever a set of flows and/groups have to be > provisioned to the switch in "atomic" manner > 3) Usage scenarios > a) Common bundle shared across applications - usage example - > coordinated resync > b) Enabling applications to use bundles with specific set of > constraints if the merit of atomicity provided by bundles offsets the > constraints (see approach 2) > NOTE: Bundle is transactional concept, so to expose these generically we need to make sure that the API control loop is properly expose, so when user configure any bundle, it can also fetch status of the bundle some way. > > > Bundle lifecycle : > > =========== > Approach 1 : > > ======== > Tying up bundle permanently with a flow/group at DS level optionally. In > other words, a given flow is always associated with a fixed bundle-id. This > can > > tantamount to adding non-persistable (config = false) bundle-id attribute > to flow and group models. This appears to be a very rigid approach which > would not be flexible to associate flows and groups with different bundles > for different scenarios (normal operations and resync – to be specific) > > This approach breaks the transactional nature of the bundle because now flows will be tied to the bundle-id and it needs to be persisted till flow/groups lifetime. Also if we really want to tag the flow/group we will have to user the config data store, because from northbound side, operational data store can not be modified. If we put in the config data store that means once FRM need to use the same bundle-id, and that can be a major limitation because now application also need to make sure that they are using the unique bundle-id. It can be done in plugin as well, but point is somebody need to do this logistic handling and that won't be simple. > > Approach 2 : > > ======== > Allowing applications to create and close the bundle with not more than > one bundle active per switch. > All groups and flows provisioned during the window of create and close of > the bundle by a given application can be pushed in context of the > current-active > bundle. This approach has a benefit that flow / group object in DS need > not explicitly refer the bundle. But separate flavor of flow and group > add/mod/del API > needs to be invoked by apps to indicate that corresponding flows and group > actions have to be encapsulated within a bundle. It would be the > responsibility of > the application to close the bundle explictly > I think this make sense to me as a first cut implementation. Although we need to provide programming interface (trough rpc, datastore) so that internal and external application both can consume this functionality. Given that at any point of time there is only active bundle for a switch, it should provide the following interface to the applications (1) Created/update/close bundle -- Already exist, but we need to (2) Get bundles with the status (with current implementation it will return only one bundle). (3) Report proper error if application is trying to modify the bundle that is already committed. (4) notification/rpc for caching the last error message, because bundle can throw error while staging as well and application should have some way to make sure that the bundle they are staging is in error-free condition. > > Approach 3 : > > ======== > Keeping bundle(s) as floating, leasable, non-persistent resouce(s) which > can be associated with one or more flows/groups combination. Application > functionality - resync being one of them, can lease one or more bundle(s) > as required from a pool of bundles, associate the same with a switch and > start using the same. Some involved yang-modeling and service-logic would > be required for providing such leasing service and corresponding validation > logic (eg. handling normal releasing and abnormal releasing - lease-onwer > node dies etc.) would require a non-trivial service implementation and > testing for various failure scenarios. But this approach would be provide > more flexibility from usage perspective. > In my humble opinion, this probably is not a good implementation given the transactional nature of the bundles. I see bundle service as a platform feature that user can leverage and it's should be pretty-much without any business logic. Just like a normal finite state machine type API's where user create a resource and take care of it's state during it's lifetime. Plugin should just throw an error message whenever user try to do the operation that is not allowed as per the openflowplugin specification, like using the already committed bundle. So in this scenario, application is the one that need to handle/coordinate the bundle lifetime, like the other resource flow/group. That i think give more freedom to applications and it probably make the programming model more simpler. > > > Approach 2 appears to be less invasive and is nothing but a special case > of Approach 3 (of course we can discuss on any other approaches and agree > upon a more acceptable way forward) > I would like to throw one more approach here, which probably can be more simpler and have same control flow for most of the scenarios. Plugin should provide a "reconcile" rpc that user can use to trigger reconciliation for individual switch. So FRM won't do any reconciliation till application sends a rpc request to trigger the reconciliation. This functionality will be enabled when user enable the "orchestrator" profile in the openflowplugin config file (we can choose any other name, it's just for discussion), otherwise everything will work the way it works in the existing implementation. This is how it should work in the two scenario you mentioned above (1) Switch connection/disconnection : Configuration is already present in the data store, so whenever application see the switch connects to the controller, it can trigger the reconciliation. If bundle based reconciliation is enabled it will use the bundle based reconciliation otherwise it will use the normal reconciliation. In clustered environment, even if switch disconnect from one of the controller, it won't notify the application about disconnection, so there is no action required from the application. But if switch is disconnected from all the controller, that will send notification to application and only "owner" controller will react to the "reconcile" request. (2) Controller reboot : This will work the same way as above. Only "owner" controller will react to the "reconcile" request. (3) Controller Upgrade : Same as above, but in this case user might want to push all of the configuration to the switch and fire the reconcile request for that switch. Rest works the same. So from plugin perspective, it's very simple contract to the application -- Plugin will reconcile the configuration present in the config data store at the time of reconcile request and then open it for general device configuration. Although this approach does not give a control over the bundles, but it provides a control to user on when to start the reconciliation. This mechanism can be further enhanced when openflowplugin will expose bundle to the application. Applications can use the bundles to reconcile in that scenario, but this "reconcile" rpc will still be require to notify plugin to open the gate for general configuration. On a side note, we can think of using the data store as well to provide a reconcile flag, but we need to discuss how we are going to maintain it's state in data store for various event like switch connection/disconnection. Let me know your thoughts. > Common model change across these approaches is following (assuming we > choose Approach 2) > > ============================================================= > 1) A separate bunde-list to be added as child to flowcapablenode - to > represent generic 1:MANY relationship between switch and bundle for future > extensions > 2) Each element of bundle-list shall be a bundle object which has > attributes - bundle-id, bundle-status (OPEN|CLOSE), bundle-type (REGULAR > for now and can > be added with additional enum values to provide one more level of > indirection in future if we want to add types like RESYNC - for example in > future) > 3) Especially for supporting approach-2, there must not be more than 1 > bundle active for a DPN at any point in time. This should be imposed as a > constraint in > the bundle-list. If we have more than 1 bundle active, then there > must be explicit mapping of flow and group entities to respective > bundles > I was wondering if we really want to maintain the bundle related data in config data store? Given that bundle is transactional in nature and they does not make sense across switch reconnect, controller reboot, i think providing the rpc's for CRUD to maintain the lifecycle of bundle probably make more sense here ? We can maintain the status of bundles in operational data store as well because we will require that for clustering environment (to avoid routed rpc). > > CRUD Handling of new bundle in bundle-list (assuming we choose Approach 2) > > ================================================== > 1) Applications create a common shared bundle (bundle-id=<bundle-id;bundle- > status=OPEN;bundle-type=REGULAR> under bundle-list and subsequently > invoke special flowmod and groupmod APIs (to indicate that corresponding > flow-mods and group-mods are to be added in context of OPEN'ed bundle) > By special means to add a leaf in the flow/group mod that we write to config data store? > 2) Bundle created under bundle-list can be disposed as part of nodal > reboot (since the bundle-list is operational in nature) > Yeah, I think using rpc here will help, because then we can just maintain the status in operational data store and that's non-persistent anyways. > 3) Bundle gets updated only for close status. ie. when applications decide > to close bundle which they have opened > 4) Bundle is read mainly by OFPlugin via DTCN to get update events on > bundle OPENing and CLOSing (for initial case, this would be mainly for > triggering resync) > > > > E2E flow (assuming Approach 2) : > > ====================== > > The proposal is to use > > a) Reconciliation API which is present in OFPlugin to announce the > available of switch to apps > > b) In reaction, apps OPEN a bundle using the approach 2 and push > flows and/or groups > > c) When completed, application can CLOSE > > > > Please see “app” referred above as normal resync or upgrade-specific > resync functionality and not in general notion of visualizing apps as ITM, > IFM or L3VPN service > > > > Let us continue the discussion during next week’s call as well as over > this mail chain > > > > Regards > > Muthu > > > > > > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > -- Thanks Anil
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
