[Architecture] [C5] [Registry] Using registry in platform components
Hi all, Now we exposed the all types of registries (will be Repository in future) to the outside registry kernel (Carbon kernel) via RegistryService. Here are the types of registries we have. systemRegistry configSystemRegistry governanceSystemRegistry userRegistry configUserRegistry governanceUserRegistry But if we take a close look we should be only exposing configUserRegistry, governanceUserRegistry which actually should be use in the platform level components. Other than that if we need to save something to local registry or as systemRegistry we should specifically give methods for them. For example if we want to retrieve last indexed time of a server we need to have a separate method. There are several advantages over this modification. 1. No one allowed to modify local registry + we may not need to show that in the resource browser. 2. There can be security issues when we expose system registries over an API 3. We have used systemRegistries all over our components for some operations which is incorrect because its overriding the authorization model + people won't find the correct logs. 4. People can use the system registry in their custom handlers which we should not allowed WDYT? thanks Eranda -- *Eranda Sooriyabandara*Senior Software Engineer; Integration Technologies Team; WSO2 Inc.; http://wso2.com Lean . Enterprise . Middleware E-mail: eranda AT wso2.com Mobile: +94 716 472 816 Linked-In: http://www.linkedin.com/in/erandasooriyabandara Blog: http://emsooriyabandara.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.com wrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
Hi Janaka, On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. Yes, correct. But if retrying fails means, something is wrong in the setup. We need to fix that before creating apps again. However, yes, roll back helps to get the appId back, so you can create the application with same appId even after it failed. However retrying will help in case of intermittent issues. thank you. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.com wrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.com wrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.com wrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com P: +94 776715671 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [C5] [Registry] Using registry in platform components
On Mon, Jan 6, 2014 at 10:28 AM, Eranda Sooriyabandara era...@wso2.comwrote: Hi all, Now we exposed the all types of registries (will be Repository in future) to the outside registry kernel (Carbon kernel) via RegistryService. Here are the types of registries we have. systemRegistry configSystemRegistry governanceSystemRegistry userRegistry configUserRegistry governanceUserRegistry Can you please explain what each of the above does? But if we take a close look we should be only exposing configUserRegistry, governanceUserRegistry which actually should be use in the platform level components. Other than that if we need to save something to local registry or as systemRegistry we should specifically give methods for them. For example if we want to retrieve last indexed time of a server we need to have a separate method. There are several advantages over this modification. 1. No one allowed to modify local registry + we may not need to show that in the resource browser. 2. There can be security issues when we expose system registries over an API 3. We have used systemRegistries all over our components for some operations which is incorrect because its overriding the authorization model + people won't find the correct logs. 4. People can use the system registry in their custom handlers which we should not allowed It sounds as if, we are going to re-factor what we have in C5. Should we not rather have a fresh look and re-design this to get the story right? In other words, how about disregarding what we have and have a holistic new look! WDYT? thanks Eranda -- *Eranda Sooriyabandara*Senior Software Engineer; Integration Technologies Team; WSO2 Inc.; http://wso2.com Lean . Enterprise . Middleware E-mail: eranda AT wso2.com Mobile: +94 716 472 816 Linked-In: http://www.linkedin.com/in/erandasooriyabandara Blog: http://emsooriyabandara.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.com wrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.com wrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com P: +94 776715671 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Factory] Per Developer Repos
Hi all, According to the diagram in Sanjiva's mail, I have come up with a sequence diagram of the scenarios that would cover in this user story. Please refer [1] For M11, we are hoping to implement the scenarios up to Add Developer user story. [1]. http://www.websequencediagrams.com/?lz=dGl0bGUgUGVyIERldmVsb3BlciBSZXBvIEltcGwgd2l0aCBHaXRIdWIKCm9wdCBDcmVhdGUgTmV3IFRlbmFudAogAAIHQWRtaW4gLT4AKAc6ACMIYSBHaXQgb3JnYW5pemF0aW9uACESQUYAJgluAD8bQUY6IFByb3ZpZGUgZwBGDyBjb250ZXh0ICYgY3JlZGVudGlhbHMAfCJuIGFwcCBvd25lciB0ZWFtAIFtBnB1c2ggcGVybWlzcwCBHRVHaXRodWI6YwCBXAhkZXYAOQZpZiBub3QgZXhpc3RpbmcgYW5kIGFkZCB1c2VyIHRvACAJCmVuZACCQhFBcHAKICBBcHBPAIELBQCCCBIAGgdGAIJVE25ldyByZXBvIGluIHRoZSBvcmcuAIINCAogAIM-BwCCaAUAWAc6IFJlc3BvbnNlAINhBgCDcQVkZXRhaQCCMwUAXwZKZW5raW5zOiBBZGQAgxcFYnVpbGQgdGFzayBmb3IAgjgFcmVwbwCBRQpBZGQAhEEKAIE_DwCEWgk6IGludml0AIM0BnRvIGpvaW4gcHJvamVjdAogAIUACwCEFgdTdXJlIACBaRFGb3JrAIFnBW1haW4AgQEGAIFdDABjC0hlcmUncyB5b3VyIGZvcmtlZAAnCACBQSYALQdkZXYAgV8PAIYrCldyaXRlcyBDb2RlAIEyEACGCghHaXQgUHVsbACBUhAAggwLACsUAIIrC0xvY2FsIElERQCDAgYASx5zaACDaw0Ag0IJQgA0BwCDVQcAgxIHIENsb3VkOiBEZXBsb3kgZACIBQkAVRUAgz8LAIguCnRlc3QvZGVidWcAhVQKUHVsbCBSZXF1ZXMAg0UVABsFcgAXCQCFVQYAhRkKABMMIHJlY2VpdmVkAIYUEACFRgt2aQCGAgUARgkAhkALAIkQCE1lcmdlAIR_Gk5vdGlmeSBhY2NlcHQAhyEFs=vs2010 Thanks Manisha On Sun, Jan 5, 2014 at 3:07 AM, Dimuthu Leelarathne dimut...@wso2.comwrote: Hi Manisha and all, Please see my comments inline. On Friday, January 3, 2014, Manisha Gayathri wrote: Hi all, For the M11 of App Factory, we are implementing per developer repo feature to give a github like experience for the developers. The user story for this federated development will be as follows: - Once an app-owner creates an application, the application will be getting a repo in gitblit - When a developer logs in, he gets an option to fork the main repo of the app Fork option should not be something done by developer when he logs in. Here is the user story we want from Sanjiva's mail titled per-developer git repos for App Factory. Here is the link from the mail [1]. The diagram here is very precise. Basically the fork request should be sent by AF when a developer is added to the project. And it should be done by AF on a listener when add user to application event is fired, so the developer do not do it. And as per the mail states GitBlit does not support PR(pull request) git workflow. Then thinking along this line, IMO we should NOT implement per developer repos using Gitblit, because it would require major hacking. I believe we should integrate with GitHub for this feature. Basically our story should be if underlying repo provider supports PR workflow then we do [1]. So lets provide this feature using GitHub. And if one day GitBlit provides PR git workflow, lets make it available for Git repositories as well. thanks, dimuthu [1] http://www.websequencediagrams.com/?lz=dGl0bGUgQXBwIEZhY3RvcnkgR2l0IFJlcG9zCgpvcHQgQ3JlYXRlIE5ldyBBcHAKICBBcHBPd25lciAtPiBBRjoAGQhuABcKRiAtPiBHaXRibGl0ABMNcmVwbwogABMIADwFAEYHOiBIZXJlJ3MgeW91ciBhcHAAJAgARwZKZW5raW5zOiBBZGQAZQVidWlsZCB0YXNrIGZvACYLZW5kAIExBkFkZCBEZXZlbG9wZXIAgScPAA8JOiBpbnZpdGF0aW9uIHRvIGpvaW4gcHJvamVjdAogADYKAIFjCFN1cmUgAIFJGXJlcG8AgQkFZGV2AIFYDgBmCwCBWwxwcml2ADEIAIFCKWRldgCBXA8AgR0KV3JpdGVzIENvZGUAgS8QAIMGCUdpdCBQdWxsAIFQEACCCgsALBQAgikLTG9jYWwgSURFAIMABgBLH3NoAINkDgCDQglCADYHAINVBwCDEgcgQ2xvdWQ6IERlcGxveSBkAIJ6CQBXFQCDPwsAgyMKdGVzdC9kZWJ1ZwCEBQpQdWxsIFJlcXVlcwCDRRUAGwVyABcJAIU9BgCFEwoAEwwgcmVjZWl2ZWQAhXwQAIVCCVJldmkAhWkFAEYJAIYoCwCGDwlNZXJnZQCFABpOb3RpZnkgYWNjZXB0AIVTBQs=vs2010 - The developer should not be permitted to directly commit to the main repo - Developer can fork the main repo and have it as his remote repo. Developer commits his changes to his remote repo - Once the changes are committed he sends a pull request with the committed changes - App owner can login to the main repo and check what are the available pull requests. He can review the commits and reject the problamatic ones and merge the accepted changes with the main repo There are couple of things to consider. We are using Gitblit as the repo provider in AF. In Gitblit only the fork option is available. The pull merge capability of Gitblit is still in work in progress state according to their discussions. They are hoping to make it available in the next release but still could not find even a trunk build of those components. In that case we will need to implement that capability in AF side. Therefore, we will give all these repo releated options from AF side. Thanks Manisha -- ~Regards *Manisha Eleperuma* Software Engineer WSO2, Inc.: http://wso2.com lean.enterprise.middleware *blog: http://manisha-eleperuma.blogspot.com/ http://manisha-eleperuma.blogspot.com/* *mobile: +94 71 8279777 %2B94%2071%208279777* -- You received this message because you are subscribed to the Google Groups WSO2 Engineering Group group. To unsubscribe from this group and stop
Re: [Architecture] Resilient application creation process
@Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.com wrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.com wrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com P: +94 776715671 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Resilient application creation process
What are the error situations that we roll back on? Are they numerous or are they handful? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com On Mon, Jan 6, 2014 at 11:04 AM, Ashansa Perera asha...@wso2.com wrote: @Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.com wrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.comwrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com P: +94 776715671 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org
Re: [Architecture] [App Factory] Per Developer Repos
Hi Manisha, On Mon, Jan 6, 2014 at 11:02 AM, Manisha Gayathri mani...@wso2.com wrote: Hi all, According to the diagram in Sanjiva's mail, I have come up with a sequence diagram of the scenarios that would cover in this user story. Please refer [1] For M11, we are hoping to implement the scenarios up to Add Developer user story. [1]. http://www.websequencediagrams.com/?lz=dGl0bGUgUGVyIERldmVsb3BlciBSZXBvIEltcGwgd2l0aCBHaXRIdWIKCm9wdCBDcmVhdGUgTmV3IFRlbmFudAogAAIHQWRtaW4gLT4AKAc6ACMIYSBHaXQgb3JnYW5pemF0aW9uACESQUYAJgluAD8bQUY6IFByb3ZpZGUgZwBGDyBjb250ZXh0ICYgY3JlZGVudGlhbHMAfCJuIGFwcCBvd25lciB0ZWFtAIFtBnB1c2ggcGVybWlzcwCBHRVHaXRodWI6YwCBXAhkZXYAOQZpZiBub3QgZXhpc3RpbmcgYW5kIGFkZCB1c2VyIHRvACAJCmVuZACCQhFBcHAKICBBcHBPAIELBQCCCBIAGgdGAIJVE25ldyByZXBvIGluIHRoZSBvcmcuAIINCAogAIM-BwCCaAUAWAc6IFJlc3BvbnNlAINhBgCDcQVkZXRhaQCCMwUAXwZKZW5raW5zOiBBZGQAgxcFYnVpbGQgdGFzayBmb3IAgjgFcmVwbwCBRQpBZGQAhEEKAIE_DwCEWgk6IGludml0AIM0BnRvIGpvaW4gcHJvamVjdAogAIUACwCEFgdTdXJlIACBaRFGb3JrAIFnBW1haW4AgQEGAIFdDABjC0hlcmUncyB5b3VyIGZvcmtlZAAnCACBQSYALQdkZXYAgV8PAIYrCldyaXRlcyBDb2RlAIEyEACGCghHaXQgUHVsbACBUhAAggwLACsUAIIrC0xvY2FsIElERQCDAgYASx5zaACDaw0Ag0IJQgA0BwCDVQcAgxIHIENsb3VkOiBEZXBsb3kgZACIBQkAVRUAgz8LAIguCnRlc3QvZGVidWcAhVQKUHVsbCBSZXF1ZXMAg0UVABsFcgAXCQCFVQYAhRkKABMMIHJlY2VpdmVkAIYUEACFRgt2aQCGAgUARgkAhkALAIkQCE1lcmdlAIR_Gk5vdGlmeSBhY2NlcHQAhyEFs=vs2010 The given diagram only contains up to committing the changes. How are we going to handle build and deployment. Can developers have their own build and own isolated deployment with their own repo? Or are they only allowed to test once they merge with the central repo. Thanks, Janaka Thanks Manisha On Sun, Jan 5, 2014 at 3:07 AM, Dimuthu Leelarathne dimut...@wso2.comwrote: Hi Manisha and all, Please see my comments inline. On Friday, January 3, 2014, Manisha Gayathri wrote: Hi all, For the M11 of App Factory, we are implementing per developer repo feature to give a github like experience for the developers. The user story for this federated development will be as follows: - Once an app-owner creates an application, the application will be getting a repo in gitblit - When a developer logs in, he gets an option to fork the main repo of the app Fork option should not be something done by developer when he logs in. Here is the user story we want from Sanjiva's mail titled per-developer git repos for App Factory. Here is the link from the mail [1]. The diagram here is very precise. Basically the fork request should be sent by AF when a developer is added to the project. And it should be done by AF on a listener when add user to application event is fired, so the developer do not do it. And as per the mail states GitBlit does not support PR(pull request) git workflow. Then thinking along this line, IMO we should NOT implement per developer repos using Gitblit, because it would require major hacking. I believe we should integrate with GitHub for this feature. Basically our story should be if underlying repo provider supports PR workflow then we do [1]. So lets provide this feature using GitHub. And if one day GitBlit provides PR git workflow, lets make it available for Git repositories as well. thanks, dimuthu [1] http://www.websequencediagrams.com/?lz=dGl0bGUgQXBwIEZhY3RvcnkgR2l0IFJlcG9zCgpvcHQgQ3JlYXRlIE5ldyBBcHAKICBBcHBPd25lciAtPiBBRjoAGQhuABcKRiAtPiBHaXRibGl0ABMNcmVwbwogABMIADwFAEYHOiBIZXJlJ3MgeW91ciBhcHAAJAgARwZKZW5raW5zOiBBZGQAZQVidWlsZCB0YXNrIGZvACYLZW5kAIExBkFkZCBEZXZlbG9wZXIAgScPAA8JOiBpbnZpdGF0aW9uIHRvIGpvaW4gcHJvamVjdAogADYKAIFjCFN1cmUgAIFJGXJlcG8AgQkFZGV2AIFYDgBmCwCBWwxwcml2ADEIAIFCKWRldgCBXA8AgR0KV3JpdGVzIENvZGUAgS8QAIMGCUdpdCBQdWxsAIFQEACCCgsALBQAgikLTG9jYWwgSURFAIMABgBLH3NoAINkDgCDQglCADYHAINVBwCDEgcgQ2xvdWQ6IERlcGxveSBkAIJ6CQBXFQCDPwsAgyMKdGVzdC9kZWJ1ZwCEBQpQdWxsIFJlcXVlcwCDRRUAGwVyABcJAIU9BgCFEwoAEwwgcmVjZWl2ZWQAhXwQAIVCCVJldmkAhWkFAEYJAIYoCwCGDwlNZXJnZQCFABpOb3RpZnkgYWNjZXB0AIVTBQs=vs2010 - The developer should not be permitted to directly commit to the main repo - Developer can fork the main repo and have it as his remote repo. Developer commits his changes to his remote repo - Once the changes are committed he sends a pull request with the committed changes - App owner can login to the main repo and check what are the available pull requests. He can review the commits and reject the problamatic ones and merge the accepted changes with the main repo There are couple of things to consider. We are using Gitblit as the repo provider in AF. In Gitblit only the fork option is available. The pull merge capability of Gitblit is still in work in progress state according to their discussions. They are hoping to make it available in the next release but still could not find even a trunk build of those components. In that case we will need to implement that capability in AF side. Therefore, we will give all these repo releated options from AF side. Thanks Manisha -- ~Regards
Re: [Architecture] [App Factory] Per Developer Repos
After the offline discussion with Dimuthu, we decided to go ahead with the per developer repo for now. Per developer build, deploy and test will be considered later. Thanks Manisha On Mon, Jan 6, 2014 at 11:45 AM, Manisha Gayathri mani...@wso2.com wrote: Thanks for bringing this up Janaka. Was going to address this separately. What I thought was, the ideal scenario would be, the developer should be able to build, deploy and test in their own isolated environment. But this incorporates a lot of work and changes. In that case, the user story will be like once the developer does the Pull Request he waits to test until the Pull Merge Accept Notification comes from the app owner. This is inconvenient from the developer's perspective. Dimuthu, how can we address this? Thanks Manisha On Mon, Jan 6, 2014 at 11:32 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manisha, On Mon, Jan 6, 2014 at 11:02 AM, Manisha Gayathri mani...@wso2.comwrote: Hi all, According to the diagram in Sanjiva's mail, I have come up with a sequence diagram of the scenarios that would cover in this user story. Please refer [1] For M11, we are hoping to implement the scenarios up to Add Developer user story. [1]. http://www.websequencediagrams.com/?lz=dGl0bGUgUGVyIERldmVsb3BlciBSZXBvIEltcGwgd2l0aCBHaXRIdWIKCm9wdCBDcmVhdGUgTmV3IFRlbmFudAogAAIHQWRtaW4gLT4AKAc6ACMIYSBHaXQgb3JnYW5pemF0aW9uACESQUYAJgluAD8bQUY6IFByb3ZpZGUgZwBGDyBjb250ZXh0ICYgY3JlZGVudGlhbHMAfCJuIGFwcCBvd25lciB0ZWFtAIFtBnB1c2ggcGVybWlzcwCBHRVHaXRodWI6YwCBXAhkZXYAOQZpZiBub3QgZXhpc3RpbmcgYW5kIGFkZCB1c2VyIHRvACAJCmVuZACCQhFBcHAKICBBcHBPAIELBQCCCBIAGgdGAIJVE25ldyByZXBvIGluIHRoZSBvcmcuAIINCAogAIM-BwCCaAUAWAc6IFJlc3BvbnNlAINhBgCDcQVkZXRhaQCCMwUAXwZKZW5raW5zOiBBZGQAgxcFYnVpbGQgdGFzayBmb3IAgjgFcmVwbwCBRQpBZGQAhEEKAIE_DwCEWgk6IGludml0AIM0BnRvIGpvaW4gcHJvamVjdAogAIUACwCEFgdTdXJlIACBaRFGb3JrAIFnBW1haW4AgQEGAIFdDABjC0hlcmUncyB5b3VyIGZvcmtlZAAnCACBQSYALQdkZXYAgV8PAIYrCldyaXRlcyBDb2RlAIEyEACGCghHaXQgUHVsbACBUhAAggwLACsUAIIrC0xvY2FsIElERQCDAgYASx5zaACDaw0Ag0IJQgA0BwCDVQcAgxIHIENsb3VkOiBEZXBsb3kgZACIBQkAVRUAgz8LAIguCnRlc3QvZGVidWcAhVQKUHVsbCBSZXF1ZXMAg0UVABsFcgAXCQCFVQYAhRkKABMMIHJlY2VpdmVkAIYUEACFRgt2aQCGAgUARgkAhkALAIkQCE1lcmdlAIR_Gk5vdGlmeSBhY2NlcHQAhyEFs=vs2010 The given diagram only contains up to committing the changes. How are we going to handle build and deployment. Can developers have their own build and own isolated deployment with their own repo? Or are they only allowed to test once they merge with the central repo. Thanks, Janaka Thanks Manisha On Sun, Jan 5, 2014 at 3:07 AM, Dimuthu Leelarathne dimut...@wso2.comwrote: Hi Manisha and all, Please see my comments inline. On Friday, January 3, 2014, Manisha Gayathri wrote: Hi all, For the M11 of App Factory, we are implementing per developer repo feature to give a github like experience for the developers. The user story for this federated development will be as follows: - Once an app-owner creates an application, the application will be getting a repo in gitblit - When a developer logs in, he gets an option to fork the main repo of the app Fork option should not be something done by developer when he logs in. Here is the user story we want from Sanjiva's mail titled per-developer git repos for App Factory. Here is the link from the mail [1]. The diagram here is very precise. Basically the fork request should be sent by AF when a developer is added to the project. And it should be done by AF on a listener when add user to application event is fired, so the developer do not do it. And as per the mail states GitBlit does not support PR(pull request) git workflow. Then thinking along this line, IMO we should NOT implement per developer repos using Gitblit, because it would require major hacking. I believe we should integrate with GitHub for this feature. Basically our story should be if underlying repo provider supports PR workflow then we do [1]. So lets provide this feature using GitHub. And if one day GitBlit provides PR git workflow, lets make it available for Git repositories as well. thanks, dimuthu [1] http://www.websequencediagrams.com/?lz=dGl0bGUgQXBwIEZhY3RvcnkgR2l0IFJlcG9zCgpvcHQgQ3JlYXRlIE5ldyBBcHAKICBBcHBPd25lciAtPiBBRjoAGQhuABcKRiAtPiBHaXRibGl0ABMNcmVwbwogABMIADwFAEYHOiBIZXJlJ3MgeW91ciBhcHAAJAgARwZKZW5raW5zOiBBZGQAZQVidWlsZCB0YXNrIGZvACYLZW5kAIExBkFkZCBEZXZlbG9wZXIAgScPAA8JOiBpbnZpdGF0aW9uIHRvIGpvaW4gcHJvamVjdAogADYKAIFjCFN1cmUgAIFJGXJlcG8AgQkFZGV2AIFYDgBmCwCBWwxwcml2ADEIAIFCKWRldgCBXA8AgR0KV3JpdGVzIENvZGUAgS8QAIMGCUdpdCBQdWxsAIFQEACCCgsALBQAgikLTG9jYWwgSURFAIMABgBLH3NoAINkDgCDQglCADYHAINVBwCDEgcgQ2xvdWQ6IERlcGxveSBkAIJ6CQBXFQCDPwsAgyMKdGVzdC9kZWJ1ZwCEBQpQdWxsIFJlcXVlcwCDRRUAGwVyABcJAIU9BgCFEwoAEwwgcmVjZWl2ZWQAhXwQAIVCCVJldmkAhWkFAEYJAIYoCwCGDwlNZXJnZQCFABpOb3RpZnkgYWNjZXB0AIVTBQs=vs2010 - The developer should not be permitted to directly commit to the main repo - Developer can fork the main repo and have it as his
Re: [Architecture] Resilient application creation process
The main error situations would be the failures on any of the following actions - Repository creation - Build job creation - Issue repository creation - Authorize roles - Add users to application On Mon, Jan 6, 2014 at 11:12 AM, Samisa Abeysinghe sam...@wso2.com wrote: What are the error situations that we roll back on? Are they numerous or are they handful? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com On Mon, Jan 6, 2014 at 11:04 AM, Ashansa Perera asha...@wso2.com wrote: @Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.com wrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.comwrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E: ram...@wso2.com P: +94 776715671 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc -- Thanks Regards, Ashansa Perera
Re: [Architecture] Resilient application creation process
Hi, Is there any possibility of showing the user that what step of this is exactly failed ?. So that user would know that due to that failure the application creation process is cancelled and rolled back. Regards On Mon, Jan 6, 2014 at 11:57 AM, Ashansa Perera asha...@wso2.com wrote: The main error situations would be the failures on any of the following actions - Repository creation - Build job creation - Issue repository creation - Authorize roles - Add users to application On Mon, Jan 6, 2014 at 11:12 AM, Samisa Abeysinghe sam...@wso2.comwrote: What are the error situations that we roll back on? Are they numerous or are they handful? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com On Mon, Jan 6, 2014 at 11:04 AM, Ashansa Perera asha...@wso2.com wrote: @Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.comwrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.comwrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Ramith Jayasinghe Technical Lead WSO2 Inc., http://wso2.com lean.enterprise.middleware E:
Re: [Architecture] Resilient application creation process
Hi, On Mon, Jan 6, 2014 at 12:17 PM, Janaka Ranabahu jan...@wso2.com wrote: On Mon, Jan 6, 2014 at 12:10 PM, Shamika Ariyawansa sham...@wso2.comwrote: Hi, Is there any possibility of showing the user that what step of this is exactly failed ?. So that user would know that due to that failure the application creation process is cancelled and rolled back. +1. That would help the user to get an understanding on why(or where) the app creation failed. +1 since this would also help to understand where it failed when fixing the setup. Thanks, Janaka Regards On Mon, Jan 6, 2014 at 11:57 AM, Ashansa Perera asha...@wso2.com wrote: The main error situations would be the failures on any of the following actions - Repository creation - Build job creation - Issue repository creation - Authorize roles - Add users to application Can the activities of the creation process be drawn diagrammatically where the various error situations are also included - we might find more error situations or different flavors while doing this. We should also identify what errors can be lived with and what cannot be. A retry mechanism for each activity should be included and the number can be configurable. On failure rollback should be done. When saying rollback, again we should ideally clearly identify what the rollback steps will be at each activity point. Thank you, Shiro On Mon, Jan 6, 2014 at 11:12 AM, Samisa Abeysinghe sam...@wso2.comwrote: What are the error situations that we roll back on? Are they numerous or are they handful? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com On Mon, Jan 6, 2014 at 11:04 AM, Ashansa Perera asha...@wso2.comwrote: @Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.comwrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.comwrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.com wrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org
Re: [Architecture] Resilient application creation process
Hi, As a user I would like to continue app creation even when some steps are failed such as Issue repository creation, And then I can try those steps later to complete the app creation. On Mon, Jan 6, 2014 at 12:10 PM, Shamika Ariyawansa sham...@wso2.comwrote: Hi, Is there any possibility of showing the user that what step of this is exactly failed ?. So that user would know that due to that failure the application creation process is cancelled and rolled back. Regards On Mon, Jan 6, 2014 at 11:57 AM, Ashansa Perera asha...@wso2.com wrote: The main error situations would be the failures on any of the following actions - Repository creation - Build job creation - Issue repository creation - Authorize roles - Add users to application On Mon, Jan 6, 2014 at 11:12 AM, Samisa Abeysinghe sam...@wso2.comwrote: What are the error situations that we roll back on? Are they numerous or are they handful? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com On Mon, Jan 6, 2014 at 11:04 AM, Ashansa Perera asha...@wso2.comwrote: @Ramith Application creation process do - repository creation - jenkins job creation - publish application creation ( which calls all the application event listeners ) On Mon, Jan 6, 2014 at 10:55 AM, Ashansa Perera asha...@wso2.comwrote: Agree with Janaka's idea of having rollback mechanism first and of course we can have retry logic in each operation as well. On Mon, Jan 6, 2014 at 10:48 AM, Ramith Jayasinghe ram...@wso2.comwrote: Shall we list down what are the steps involved in creating a application. then what needs to be to undo each step that was performed? On Mon, Jan 6, 2014 at 10:45 AM, Gayan Dhanushka gay...@wso2.comwrote: Hi Janaka, IMO it is ok for us to have a mechanism for retrying. If it was an intermittent issue that interrupted the system from creating the application this will solve it. If the application creation failed after retrying for a few times we can roll back the entire process. If it is some other serious issue rolling back and trying to start the application creation process from the begining still won't work. WDYT? GayanD Gayan Dhanuska Software Engineer http://wso2.com/ Lean Enterprise Middleware Mobile 071 666 2327 Office Tel : 94 11 214 5345 Fax : 94 11 214 5300 Twitter : https://twitter.com/gayanlggd On Mon, Jan 6, 2014 at 10:33 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.com wrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.comwrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
