On Fri, Apr 28, 2017 at 2:10 PM, Manoj Gunawardena <[email protected]> wrote:
> This is a suggestion, > > How about a REST API instead of a JMS service. > > GW nodes pull changes from APIMCore via REST service in APIMCore with > configurable intervals. > > The GW nodes fetch changes based on a timestamp (last fetch time) and api > labels.(apis which hosted in that gw) > The REST api will response meta data of changes as JSON payloads. > > The worry of connectivity breaking between core and GW is not with this > mechanism. > Well, I don't think intervals work. If an update is not reflected soon enough, it becomes a frustrating user experience. So we have to keep this interval to a small value. But in practice, API updates are very rare (almost never happens in prod). So its a waste to have to keep running a process on such a small interval. > > WDYT? > > > > On Fri, Apr 28, 2017 at 1:29 PM, Lakmal Warusawithana <[email protected]> > wrote: > >> Hi Kasun, >> >> On Fri, Apr 28, 2017 at 4:47 AM, Kasun Indrasiri <[email protected]> wrote: >> >>> Hi, >>> >>> Few questions related to the above design. >>> >>> >>> - Here we are using pub-sub messaging to receive notification of the >>> event of different API lifecycle states and use that to apply those changes >>> (deploy, redeploy etc.) at the Ballerina service level. This is inherently >>> subjective to lost the event notifications in case of a communication >>> failure between the broker and GWs. For instance, if there is a >>> communication failure, some API may not deploy/update on a given GW node (I >>> doubt that we will face the same set of issues as we have in the old >>> dep-sync approach). >>> I'd rather think a push mechanism from the APIMCore to GW would be more >>> solid compared to pub-sub. >>> >>> WDYT? >>> >> >> Even push model APIMCore to some of gateway communication can be broken. >> Then we need to replay that push messages otherwise they will not get them >> at all. But why we want to have MB and the pub-sub is for reliable message >> delivery. Second, this is very loosely couple and we can scale GWs without >> updating the topology in the APIMCore. >> >> But, yes you have a point. Like NuwanD said, we may need to use durable >> topics to handle the communication issues. I think we can have unique >> identifiers for GWs and use in durable topic subscriptions. Only catch is >> in container scenario, GWs can come and go very frequently and we need to >> have some mechanism to clean these dead subscriptions. Hope from MB we may >> have some way to clean them up. (or else we may need to have some clean up >> process in the core) >> >> >> >>> >>> From the Ballerina perspective: >>> >>> - A pre-built JMS service can subscribe to the respective topics (during >>> the deployment time) and as the event updates are received it has to deploy >>> (create/update new Ballerina service file?) the respective Ballerina >>> service. However, I feel we need to clarify the above issues prior to the >>> implementation. >>> >>> On Wed, Apr 26, 2017 at 9:58 PM, Thilini Shanika <[email protected]> >>> wrote: >>> >>>> Hi Manoj, >>>> >>>> Please find my comments inline. >>>> >>>> Fetching new apis from APIM core do via database call or REST api? >>>> >>>> We are providing a set of REST apis (in API core) for ballerina >>>> gateways to access and fetch API conf, subscription details etc. Following >>>> diagram depicts how the communication happens in between the APIM core, the >>>> gateway, and the broker. >>>> >>>> >>>> >>>> I think, we have to think about time to fetch all apis and starting >>>> time of a node. If num of apis increases, the startup time of a node may >>>> increase. >>>> This may critical for a containerized environments. >>>> We found some production instances which has around 1000 apis. >>>> >>>> Yes. As per the new design, there is a plan of deploying APIs on demand >>>> and this will eliminate the overhead of API deployment delays at server >>>> startup. >>>> >>>> On Thu, Apr 27, 2017 at 9:58 AM, Manoj Gunawardena <[email protected]> >>>> wrote: >>>> >>>>> Fetching new apis from APIM core do via database call or REST api? >>>>> I think, we have to think about time to fetch all apis and starting >>>>> time of a node. If num of apis increases, the startup time of a node may >>>>> increase. >>>>> This may critical for a containerized environments. >>>>> We found some production instances which has around 1000 apis. >>>>> >>>>> On Wed, Apr 26, 2017 at 1:56 PM, Sanjeewa Malalgoda <[email protected] >>>>> > wrote: >>>>> >>>>>> In fully automated containerized environment can we assume we can see >>>>>> node restarts. As i understood nodes come and go(each time we require we >>>>>> start new instance and when its not in use it will terminate). In such >>>>>> scenario having durable subscription or persisting data to local node do >>>>>> not make much difference. >>>>>> >>>>>> But if we are thinking of traditional deployment then we may be able >>>>>> to restart instances and reuse them. >>>>>> @Lakmal, Pubudu any thoughts? >>>>>> >>>>>> Thanks, >>>>>> sanjeewa. >>>>>> >>>>>> On Wed, Apr 26, 2017 at 11:34 AM, Pubudu Gunatilaka <[email protected] >>>>>> > wrote: >>>>>> >>>>>>> On Wed, Apr 26, 2017 at 11:02 AM, Lakmal Warusawithana < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Apr 26, 2017 at 10:51 AM, Manoj Gunawardena < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Are we going to use durable subscription here? When ever gw node >>>>>>>>> down and re start, >>>>>>>>> it should fetch messages which add to the topic during the down >>>>>>>>> time. >>>>>>>>> In case durable subscription what is the mechanism to create >>>>>>>>> client id for each gw.Is that discussed and confirm? >>>>>>>>> >>>>>>>> >>>>>>>> No, we are not going to use durable topic. New gateways (scaling) >>>>>>>> or if we restart a gateway, existing APIs will fetch via API Manager >>>>>>>> core. >>>>>>>> Only new API (after boot up) notifications are getting via the topic >>>>>>>> and >>>>>>>> actual API will fetch from the core. ( we can try what sanjeewa >>>>>>>> mention - >>>>>>>> getting API from the topic - but previous APIs need to fetch from the >>>>>>>> core) >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> When we restart a gateway node, it would be better to fetch only the >>>>>>> updated APIs as well as the new APIs. Restarted gateway has the already >>>>>>> fetched APIs and it is not worth to fetch all the APIs again. We will >>>>>>> have >>>>>>> to use durable subscription for this. >>>>>>> >>>>>>> If we consider the container scenario, are we persisting gateway >>>>>>> artifacts? If we are not persisting gateway artifacts and restart the >>>>>>> gateway container, then we need to fetch all the APIs from the core. >>>>>>> IMHO >>>>>>> it would better to persist data and load that data at restart as it >>>>>>> would >>>>>>> take time to fetch all the APIs from the core. >>>>>>> >>>>>>> Thank you! >>>>>>> -- >>>>>>> *Pubudu Gunatilaka* >>>>>>> Committer and PMC Member - Apache Stratos >>>>>>> Software Engineer >>>>>>> WSO2, Inc.: http://wso2.com >>>>>>> mobile : +94774078049 <%2B94772207163> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Sanjeewa Malalgoda* >>>>>> WSO2 Inc. >>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>> >>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Manoj Gunawardena >>>>> Tech Lead >>>>> WSO2, Inc.: http://wso2.com >>>>> lean.enterprise.middleware >>>>> Mobile : +94 77 2291643 >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thilini Shanika >>>> Senior Software Engineer >>>> WSO2, Inc.; http://wso2.com >>>> 20, Palmgrove Avenue, Colombo 3 >>>> >>>> E-mail: [email protected] >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Kasun Indrasiri >>> Director - Integration Architecture >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> cell: +1 650 450 2293 <(650)%20450-2293> >>> Blog : http://kasunpanorama.blogspot.com/ >>> >> >> >> >> -- >> Lakmal Warusawithana >> Director - Cloud Architecture; WSO2 Inc. >> Mobile : +94714289692 <+94%2071%20428%209692> >> Blogs : https://medium.com/@lakwarus/ >> http://lakmalsview.blogspot.com/ >> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Manoj Gunawardena > Tech Lead > WSO2, Inc.: http://wso2.com > lean.enterprise.middleware > Mobile : +94 77 2291643 > -- Nuwan Dias Software Architect - WSO2, Inc. http://wso2.com email : [email protected] Phone : +94 777 775 729
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
