Re: [Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi Mahesh and all, I have another suggestion. We should keep the dev, test, prod subscribers in the respective Stratos controllers. This is because it will reduce the load on the network. The architecture stays the same. thanks, dimuthu On Fri, Dec 19, 2014 at 2:16 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Mahesh, What is the possibility of removing the second REST call. If we are putting an additional moving part we need to justify why. For example we could say in this point you can put a LB and horizontally scale (Just one of many possible answers). In this case I don't see such a use case unless there is no Java API. Does Stratos provide Java API? If so that would be a better way, IMO. thanks, dimuthu On Wed, Dec 17, 2014 at 12:42 PM, Manjula Rathnayake manju...@wso2.com wrote: Hi Mahesh, +1 for the current design. I assume you have multiple instances of durable subscribers for each subscription made to Stratos cartridges. For example, if we have multiple tenant aware cartridges(App Server, UES Server, BPS Server etc), the durable subscriber should be able to instantiate based on subscription details for each cartridges in all the environments. Further we can consider unsubscribing from cartridges etc. Based on the message received by the topic subscribers, topic subscriber can decide the action plan(subscribe to cartridge, unsubscribe,etc). In case of failure, lets assume subscription to production app server cartridges is failed. Are we going to stop further actions in that particular tenant such as avoiding promoting to production? IMO, notifying to the tenant administrator and system administrators would be sufficient via notification mechanisms(wall,email). thank you. On Wed, Dec 17, 2014 at 3:16 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi Harsha, I see your point. If you follow tenant creation bpel you may see initializeBuildManager and initializeRepositoryManager happens one after the other in a row. If it got failed in any step the process won't continue. But in last step of bpel you can see that initializeCloudManager is called for all 3 environments at the same time. In order to consider tenant creation as successfull we need to have all 3 subscriptions successful. I think thats why this fault tolerance mechanism is more important in this step than others. Anyway I will discuss more with the team and will update this thread. On Wed, Dec 17, 2014 at 2:45 PM, Harsha Thirimanna hars...@wso2.com wrote: Hi Mahesh, What about the other failure situation within BPEL, like initializeBuildManager and initializeRepositoryManager. Can we do that also consuming this topics ? Because that also mandatory for the tenant creation flow. WDYT ? thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* *email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770 , +94 * *774617784twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimannlinked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Wed, Dec 17, 2014 at 1:31 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber will not ack. and MB will take care of sending that request until it becomes successful. At the moment we have 3
Re: [Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi Dimuthu, Noted. thanks. On Mon, Jan 5, 2015 at 10:34 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Mahesh and all, I have another suggestion. We should keep the dev, test, prod subscribers in the respective Stratos controllers. This is because it will reduce the load on the network. The architecture stays the same. thanks, dimuthu On Fri, Dec 19, 2014 at 2:16 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Mahesh, What is the possibility of removing the second REST call. If we are putting an additional moving part we need to justify why. For example we could say in this point you can put a LB and horizontally scale (Just one of many possible answers). In this case I don't see such a use case unless there is no Java API. Does Stratos provide Java API? If so that would be a better way, IMO. thanks, dimuthu On Wed, Dec 17, 2014 at 12:42 PM, Manjula Rathnayake manju...@wso2.com wrote: Hi Mahesh, +1 for the current design. I assume you have multiple instances of durable subscribers for each subscription made to Stratos cartridges. For example, if we have multiple tenant aware cartridges(App Server, UES Server, BPS Server etc), the durable subscriber should be able to instantiate based on subscription details for each cartridges in all the environments. Further we can consider unsubscribing from cartridges etc. Based on the message received by the topic subscribers, topic subscriber can decide the action plan(subscribe to cartridge, unsubscribe,etc). In case of failure, lets assume subscription to production app server cartridges is failed. Are we going to stop further actions in that particular tenant such as avoiding promoting to production? IMO, notifying to the tenant administrator and system administrators would be sufficient via notification mechanisms(wall,email). thank you. On Wed, Dec 17, 2014 at 3:16 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi Harsha, I see your point. If you follow tenant creation bpel you may see initializeBuildManager and initializeRepositoryManager happens one after the other in a row. If it got failed in any step the process won't continue. But in last step of bpel you can see that initializeCloudManager is called for all 3 environments at the same time. In order to consider tenant creation as successfull we need to have all 3 subscriptions successful. I think thats why this fault tolerance mechanism is more important in this step than others. Anyway I will discuss more with the team and will update this thread. On Wed, Dec 17, 2014 at 2:45 PM, Harsha Thirimanna hars...@wso2.com wrote: Hi Mahesh, What about the other failure situation within BPEL, like initializeBuildManager and initializeRepositoryManager. Can we do that also consuming this topics ? Because that also mandatory for the tenant creation flow. WDYT ? thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* *email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770 , +94 * *774617784twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimannlinked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Wed, Dec 17, 2014 at 1:31 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber
Re: [Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi Mahesh, What is the possibility of removing the second REST call. If we are putting an additional moving part we need to justify why. For example we could say in this point you can put a LB and horizontally scale (Just one of many possible answers). In this case I don't see such a use case unless there is no Java API. Does Stratos provide Java API? If so that would be a better way, IMO. thanks, dimuthu On Wed, Dec 17, 2014 at 12:42 PM, Manjula Rathnayake manju...@wso2.com wrote: Hi Mahesh, +1 for the current design. I assume you have multiple instances of durable subscribers for each subscription made to Stratos cartridges. For example, if we have multiple tenant aware cartridges(App Server, UES Server, BPS Server etc), the durable subscriber should be able to instantiate based on subscription details for each cartridges in all the environments. Further we can consider unsubscribing from cartridges etc. Based on the message received by the topic subscribers, topic subscriber can decide the action plan(subscribe to cartridge, unsubscribe,etc). In case of failure, lets assume subscription to production app server cartridges is failed. Are we going to stop further actions in that particular tenant such as avoiding promoting to production? IMO, notifying to the tenant administrator and system administrators would be sufficient via notification mechanisms(wall,email). thank you. On Wed, Dec 17, 2014 at 3:16 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi Harsha, I see your point. If you follow tenant creation bpel you may see initializeBuildManager and initializeRepositoryManager happens one after the other in a row. If it got failed in any step the process won't continue. But in last step of bpel you can see that initializeCloudManager is called for all 3 environments at the same time. In order to consider tenant creation as successfull we need to have all 3 subscriptions successful. I think thats why this fault tolerance mechanism is more important in this step than others. Anyway I will discuss more with the team and will update this thread. On Wed, Dec 17, 2014 at 2:45 PM, Harsha Thirimanna hars...@wso2.com wrote: Hi Mahesh, What about the other failure situation within BPEL, like initializeBuildManager and initializeRepositoryManager. Can we do that also consuming this topics ? Because that also mandatory for the tenant creation flow. WDYT ? thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* *email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770 , +94 * *774617784twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimannlinked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Wed, Dec 17, 2014 at 1:31 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber will not ack. and MB will take care of sending that request until it becomes successful. At the moment we have 3 stratos environments therefore we will have 3 subscribers listening to that particular topic(create_teanant) in MB. In a case of AF having more stratos environments in future, it is possible to add more subscribers. Durable topic subscribers[2] will be used here since it will persist the subscription on behalf of the client
[Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber will not ack. and MB will take care of sending that request until it becomes successful. At the moment we have 3 stratos environments therefore we will have 3 subscribers listening to that particular topic(create_teanant) in MB. In a case of AF having more stratos environments in future, it is possible to add more subscribers. Durable topic subscribers[2] will be used here since it will persist the subscription on behalf of the client subscription when client is not online. This step is not synchronous therefore above changes will not effect the tenant creation flow. *Architectural diagram * *=* Architectural diagram can be found in [3] [1] - https://wso2.org/jira/browse/APPFAC-2436 [2] - https://docs.wso2.com/display/MB220/Creating+a+Durable+Topic+Subscription [3] - https://docs.google.com/a/wso2.com/drawings/d/15CmM3rCTwjUUu4V6lp2WX32nRzufraDXawALNYIoTek/edit -- *Mahesh Chinthaka Vidanagama* | Software Engineer WSO2, Inc | lean. enterprise. middleware. #20, Palm Grove, Colombo 03, Sri Lanka Mobile: +94 71 63 63 083 | Work: +94 112 145 345 Email: mahe...@wso2.com | Web: www.wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi Harsha, I see your point. If you follow tenant creation bpel you may see initializeBuildManager and initializeRepositoryManager happens one after the other in a row. If it got failed in any step the process won't continue. But in last step of bpel you can see that initializeCloudManager is called for all 3 environments at the same time. In order to consider tenant creation as successfull we need to have all 3 subscriptions successful. I think thats why this fault tolerance mechanism is more important in this step than others. Anyway I will discuss more with the team and will update this thread. On Wed, Dec 17, 2014 at 2:45 PM, Harsha Thirimanna hars...@wso2.com wrote: Hi Mahesh, What about the other failure situation within BPEL, like initializeBuildManager and initializeRepositoryManager. Can we do that also consuming this topics ? Because that also mandatory for the tenant creation flow. WDYT ? thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* *email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770 , +94 * *774617784twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimannlinked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Wed, Dec 17, 2014 at 1:31 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber will not ack. and MB will take care of sending that request until it becomes successful. At the moment we have 3 stratos environments therefore we will have 3 subscribers listening to that particular topic(create_teanant) in MB. In a case of AF having more stratos environments in future, it is possible to add more subscribers. Durable topic subscribers[2] will be used here since it will persist the subscription on behalf of the client subscription when client is not online. This step is not synchronous therefore above changes will not effect the tenant creation flow. *Architectural diagram * *=* Architectural diagram can be found in [3] [1] - https://wso2.org/jira/browse/APPFAC-2436 [2] - https://docs.wso2.com/display/MB220/Creating+a+Durable+Topic+Subscription [3] - https://docs.google.com/a/wso2.com/drawings/d/15CmM3rCTwjUUu4V6lp2WX32nRzufraDXawALNYIoTek/edit -- *Mahesh Chinthaka Vidanagama* | Software Engineer WSO2, Inc | lean. enterprise. middleware. #20, Palm Grove, Colombo 03, Sri Lanka Mobile: +94 71 63 63 083 | Work: +94 112 145 345 Email: mahe...@wso2.com | Web: www.wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Mahesh Chinthaka Vidanagama* | Software Engineer WSO2, Inc | lean. enterprise. middleware. #20, Palm Grove, Colombo 03, Sri Lanka Mobile: +94 71 63 63 083 | Work: +94 112 145 345 Email: mahe...@wso2.com | Web: www.wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [AF] [MB] Make Appfactory tenant subscription to stratos environments fault tolerant
Hi Mahesh, +1 for the current design. I assume you have multiple instances of durable subscribers for each subscription made to Stratos cartridges. For example, if we have multiple tenant aware cartridges(App Server, UES Server, BPS Server etc), the durable subscriber should be able to instantiate based on subscription details for each cartridges in all the environments. Further we can consider unsubscribing from cartridges etc. Based on the message received by the topic subscribers, topic subscriber can decide the action plan(subscribe to cartridge, unsubscribe,etc). In case of failure, lets assume subscription to production app server cartridges is failed. Are we going to stop further actions in that particular tenant such as avoiding promoting to production? IMO, notifying to the tenant administrator and system administrators would be sufficient via notification mechanisms(wall,email). thank you. On Wed, Dec 17, 2014 at 3:16 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi Harsha, I see your point. If you follow tenant creation bpel you may see initializeBuildManager and initializeRepositoryManager happens one after the other in a row. If it got failed in any step the process won't continue. But in last step of bpel you can see that initializeCloudManager is called for all 3 environments at the same time. In order to consider tenant creation as successfull we need to have all 3 subscriptions successful. I think thats why this fault tolerance mechanism is more important in this step than others. Anyway I will discuss more with the team and will update this thread. On Wed, Dec 17, 2014 at 2:45 PM, Harsha Thirimanna hars...@wso2.com wrote: Hi Mahesh, What about the other failure situation within BPEL, like initializeBuildManager and initializeRepositoryManager. Can we do that also consuming this topics ? Because that also mandatory for the tenant creation flow. WDYT ? thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* *email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770 , +94 * *774617784twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimannlinked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Wed, Dec 17, 2014 at 1:31 PM, Mahesh Chinthaka mahe...@wso2.com wrote: Hi all, Im working on Making the tenants subscription to stratos environments fault tolerant [1] This has been discussed in mail thread with subject - [Architecture] [AF] Tenant subscribing to stratos environments - making it fault tolerant *Issue* ** There are some possible failure points in AF, hence in App Cloud. One such critical point is making the tenants subscribed to the dev, test and prod stratos environments. Due to any reason, if this step fails, then, thats the end of story for that tenant. Actually in appfactory tenant creation, When a tenant creation request is made, appfactory sends REST calls to stratos environments (dev,prod,test) to create tenants. At the moment there is no way to overcome a situation where tenant creation fails in one or more stratos environments. *Proposal* *===* Here what I'm going to do is, integrate WSO2 Message Broker to take tenant creation requests to a durable topic and manipulate the process in a fault tolerant manner. The durable topic will keep messages around persistently for any suitable consumer to consume them. Appfactory will have a publisher who publishes messages in the topic inside MB when a tenant creation is requested. There will be a subscriber for each stratos environment who is listening to above topic(create_teanant). If subscription is successfull then subscriber will acknowledge to MB and MB will not send that particular request to same subscriber again. If subscription got failed subscriber will not ack. and MB will take care of sending that request until it becomes successful. At the moment we have 3 stratos environments therefore we will have 3 subscribers listening to that particular topic(create_teanant) in MB. In a case of AF having more stratos environments in future, it is possible to add more subscribers. Durable topic subscribers[2] will be used here since it will persist the subscription on behalf of the client subscription when client is not online. This step is not synchronous therefore above changes will not effect the tenant creation flow. *Architectural diagram * *=* Architectural diagram can be found in [3] [1] - https://wso2.org/jira/browse/APPFAC-2436 [2] - https://docs.wso2.com/display/MB220/Creating+a+Durable+Topic+Subscription [3] - https://docs.google.com/a/wso2.com/drawings/d/15CmM3rCTwjUUu4V6lp2WX32nRzufraDXawALNYIoTek/edit -- *Mahesh Chinthaka Vidanagama* |
