On Wed, Jul 5, 2017 at 3:53 AM, Chathura Kulasinghe <[email protected]> wrote:
> > > Chathura Asanga Kulasinghe | + > > 4 > 4 > 77 2920 2448 | Senior Solutions Engineer | WSO₂ > > On 4 July 2017 at 18:02, Lakmal Warusawithana <[email protected]> wrote: > >> >> >> On Tue, Jul 4, 2017 at 10:07 PM, Chathura Kulasinghe <[email protected]> >> wrote: >> >>> >>> >>> Chathura Asanga Kulasinghe | + >>> >>> 4 >>> 4 >>> 77 2920 2448 | Senior Solutions Engineer | WSO₂ >>> >>> On 30 June 2017 at 10:17, Lakmal Warusawithana <[email protected]> wrote: >>> >>>> Hi Maninda, >>>> >>>> On Fri, Jun 30, 2017 at 12:02 PM, Maninda Edirisooriya < >>>> [email protected]> wrote: >>>> >>>>> Hi Dinusha, >>>>> >>>>> Why do we think it is the responsibility of the CEP to pull poilcies >>>>> from the APIM? Just forgetting the APIM case, if we think this as a >>>>> general >>>>> feature of CEP, here the CEP is periodically pulling artifacts from a >>>>> given >>>>> set of APIs. There are 2 drawbacks I see in this approach. >>>>> >>>>> 1. Updating the CEP with latest artifacts is restricted by the given >>>>> polling time interval. (realtime CEP artifact updates are not possible >>>>> from >>>>> an external party which is important as CEP is dealing with realtime >>>>> requirements) >>>>> >>>>> 2. Dynamically adding unplanned artifacts to a running CEP server is >>>>> not possible as the polling APIs has to be pre configured. For example >>>>> this >>>>> can happen when the existing APIM server is stopped and a different APIM >>>>> server is started with a different host address. In that case the CEP also >>>>> has to be restarted to update the policy API location, which would lose >>>>> some data in the running CEP. Correct me if I am wrong. >>>>> >>>>> Therefore I think the current policy push based approach is correct. >>>>> We should be able to correct the syncing policies across the CEP nodes >>>>> with >>>>> a different approach like deploment synchronizing, registry based >>>>> approach, >>>>> with a shared database based approach or with a clustering solution like >>>>> Hazelcast. >>>>> >>>> >>>> This is exactly what we want to avoid. :) >>>> >>> >>> Would it be practical to think of spinning-up the Traffic-Manager >>> instances with updated policies as an immutable-node-group? (...more a >>> dev-ops oriented approach). Because, a poll-and-pull type of mechanism >>> anyway will have a delay in applying the policy updates, which could have >>> become a concern for some industries (even if that may be accepted in >>> general case). >>> >>> >> If someone want immutability, yes they can bern policy to image and >> spin-up. Having pull method not harm for them as well. VM scenario and some >> container cases, they don't want to build docker image or VM with every >> policy update. So default it pull, but they can avoid that as well. >> > > I am sorry for narrowing my comment down to containers based deployment. > However, what I wanted to mention is was a set of reasons for thinking of > sticking to the push model, based on a few concerns that may be raised by > customers. In order to describe this, let me take a sample use-case. > > If we consider Telco business for example, what is allowed & > restricted/controlled under a certain policy is critical as every > millisecond counts from the finance departments' point-of-view. Therefore, > those customers like that may expect the policy updates to become available > in the runtime immediately after modifying/publishing them. In order to > comply with this requirement at the expected level, with the poll-and-pull > method, we need to configure a polling interval of a few milliseconds. > However, such policy updates are not that frequent in a typical business > scenario (even if it is crucial to apply it immediately once a change is > made). > > For example, a policy update may happen once a month, but the organisation > expect to apply changes immediately as they are made (without waiting a few > minutes until the Traffic Manager fetches the updates with a noticeable > delay). > > In such a scenario, if we ask the customer to configure a polling interval > of a few milliseconds, many EAs, infra/network staff may consider that as > an unnecessary overhead as the system is supposed to make frequent network > polls in order to fetch any updates that possibly may happen once a month. > > Therefore, wouldn't it be advisable to perform this following a > 'recipient-list' style push mechanism? Typically, the traffic manager > wouldn't scale into a number of instances that exceeds 4-6. And if we > introduce a 'recipient-list' style policy updates distribution mechanism, > that wouldn't have any dependency over clustering and hazelcast as well. > > I understand that you guys may have many other reasons/concerns to > consider while selecting the best possible option. But, I just thought of > adding thoughts to be considered in further brainstorming and to make sure > that no doubts/concerns were left unspoken. :) > I am seen some contradiction within your usecase. IMO, if customer updating policies in months and they want to update runtime in milliseconds? Sorry, to me this is not valid use case. Please apply 80/20 rule. Most of the case these policies not going to be change after deploying. So in these cases no need to have pull model as well. Few cases, they will change these policies and that totally fine with pull model or manually handle artifact syncing. (may be puppet etc). Pull model is one reference implementation. What we providing is an API in the core to retrieve updated siddhi artifacts. They can run this in TM to get the new artifacts. It can be manual, shell script, puppet with m-collective, cron etc. If someone wan't realtime updates, after creating policies in TM, they can should trigger any update mechanism IMO. > >> >>> >>>> #1 is not an valid concern. We are not going to solve generic case >>>> here. In the APIM, throttling policy update not a frequent thing. >>>> Sometimes it only doing one time for a deployment. >>>> >>>> I can't understand #2 concern. No need CEP to restart anytime. >>>> >>>> >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> *Maninda Edirisooriya* >>>>> Senior Software Engineer >>>>> >>>>> *WSO2, Inc.*lean.enterprise.middleware. >>>>> >>>>> *Blog* : http://maninda.blogspot.com/ >>>>> *E-mail* : [email protected] >>>>> *Skype* : @manindae >>>>> *Twitter* : @maninda >>>>> >>>>> On Thu, Jun 29, 2017 at 5:10 PM, Dinusha Dissanayake < >>>>> [email protected]> wrote: >>>>> >>>>>> +Adding architecture >>>>>> >>>>>> On Thu, Jun 29, 2017 at 5:05 PM, Nuwan Dias <[email protected]> wrote: >>>>>> >>>>>>> Let's discuss publicly please, this thread is internal only. >>>>>>> >>>>>>> On Thu, Jun 29, 2017 at 5:01 PM, Dinusha Dissanayake < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> In earlier versions of APIM, throttle policy deployment for CEP was >>>>>>>> handled using push mechanism meaning APIM itself had to deploy throttle >>>>>>>> policies in CEP. If there are multiple CEP nodes, then the Siddhi >>>>>>>> apps/execution plans syncing would be an issue here. >>>>>>>> >>>>>>>> We were thinking of pull based mechanism from CEP side to overcome >>>>>>>> this. To do that, from APIM side we need to add an API to the core >>>>>>>> which >>>>>>>> allows CEP to get all Siddhi apps for existing throttle policies. Then >>>>>>>> every CEP node will call APIM (once or periodically) and then it will >>>>>>>> check >>>>>>>> the DB and generate a zip containing all the Siddhi Apps for existing >>>>>>>> throttle policies. >>>>>>>> >>>>>>>> >>>>>>>> This API will only need GET resource. The resource path for this >>>>>>>> API would be *"export/policies/throttle"*. As mentioned above, >>>>>>>> this will return a zip. Mutual SSL would be used to secure the API >>>>>>>> since if >>>>>>>> we use OAuth2, then the token may expire in a while. >>>>>>>> >>>>>>>> Exported zip can be manually deployed in the CEP or deployment can >>>>>>>> be done using e a curl command. If this needs to be happened >>>>>>>> periodically, >>>>>>>> then a cron job can be written. >>>>>>>> >>>>>>>> Please provide suggestions for improvements. >>>>>>>> >>>>>>>> Thanks & Regards. >>>>>>>> -- >>>>>>>> Dinusha Dissanayake >>>>>>>> Software Engineer >>>>>>>> WSO2 Inc >>>>>>>> Mobile: +94712939439 <+94%2071%20293%209439> >>>>>>>> <https://wso2.com/signature> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Nuwan Dias >>>>>>> >>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>> email : [email protected] >>>>>>> Phone : +94 777 775 729 <+94%2077%20777%205729> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dinusha Dissanayake >>>>>> Software Engineer >>>>>> WSO2 Inc >>>>>> Mobile: +94712939439 <+94%2071%20293%209439> >>>>>> <https://wso2.com/signature> >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Lakmal Warusawithana >>>> Director - Cloud Architecture; WSO2 Inc. >>>> Mobile : +94714289692 <+94%2071%20428%209692> >>>> Blogs : https://medium.com/@lakwarus/ >>>> http://lakmalsview.blogspot.com/ >>>> >>>> >>> >> >> >> -- >> Lakmal Warusawithana >> Director - Cloud Architecture; WSO2 Inc. >> Mobile : +94714289692 <+94%2071%20428%209692> >> Blogs : https://medium.com/@lakwarus/ >> http://lakmalsview.blogspot.com/ >> >> > -- Lakmal Warusawithana Director - Cloud Architecture; WSO2 Inc. Mobile : +94714289692 Blogs : https://medium.com/@lakwarus/ http://lakmalsview.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
