Hi, I will be setting up a meeting (tomorrow 11am) to discuss and finalize the above mentioned component categorization. Can we have at least one person from each product team in that meeting who has knowledge of the component structure relevant to their products?
Thanks Shevan On Mon, Jan 20, 2014 at 12:19 AM, Senaka Fernando <[email protected]> wrote: > Hi all, > > Azeez is correct. But even with #2, there are productive ways of managing > that. Read [1] for example. Oh and BTW, our git repositories better have > some good conventions. Right now, [2] is a little messy. > > [1] > http://www.lshift.net/blog/2013/06/27/managing-multiple-github-repositories > [2] https://github.com/wso2 > > Thanks, > Senaka. > > > On Sun, Jan 19, 2014 at 11:47 PM, Afkham Azeez <[email protected]> wrote: > >> >> >> >> On Sun, Jan 19, 2014 at 11:34 PM, Sameera Jayasoma <[email protected]>wrote: >> >>> >>> >>> >>> On Sun, Jan 19, 2014 at 9:53 AM, Selvaratnam Uthaiyashankar < >>> [email protected]> wrote: >>> >>>> >>>> >>>> >>>> On Sun, Jan 19, 2014 at 11:02 PM, Sameera Jayasoma <[email protected]>wrote: >>>> >>>>> Hi Azeez, >>>>> >>>>> +1 for this model. This will solve most of the issues that we are >>>>> facing in the current system. >>>>> >>>>> >>>>> Senaka, see my comment below. >>>>> >>>>> On Sat, Jan 18, 2014 at 10:48 AM, Senaka Fernando <[email protected]>wrote: >>>>> >>>>>> Hi Azeez, Eranda, >>>>>> >>>>>> +1 for the proposed changes. So, as discussed we have two options. >>>>>> >>>>>> 1. In the git repo, go for wso2/carbon/feature or wso2/carbon/product >>>>>> model and then have features such as registry, governance and products >>>>>> such >>>>>> as esb. >>>>>> >>>>> >>>>> >>>>> I am + for the Option 1 if possible. >>>>> >>>> >>>> >>>> With this option, shouldn't we checkout the whole components? AFAIK, >>>> GIt doesn't support checking out part of the repo. Also, the release, >>>> branching etc. will be at the repo level, can't be at a sub repo level. >>>> Above structure is what we have now and is the reason for most of the >>>> trouble we are having, isn't it? >>>> >>> >>> No Shankar, all the components will be separate projects on their own. >>> But this is just a matter of organizing them in a proper order without >>> putting everything to the top level. (only if this is possible.) >>> >>> e.g. https://github.com/wso2/carbon/components/identity >>> >> >> You can't do that. It will create your repo as, >> carbon-components-identity if you try to do that. >> >> >>> >>> If this is not possible, then I prefer *carbon-component-* instead of >>> carbon-feature-*. * Because IMHO, components are what we develop and >>> features are just a way of installing these components to projects. We may >>> end up multiple features for a given component or features which combine >>> multiple components. >>> >>> >>> >>>> >>>> >>>> >>>>> That will allows developers to easily navigate to the relevant >>>>> component. >>>>> >>>>> How about using wso2/carbon/components instead of features. What we >>>>> really develop is components? I would prefer to use the word component >>>>> rather than using the word feature. >>>>> >>>>> WDYT? >>>>> >>>>> >>>>>> 2. Go with the model Azeez wrote in e-mail. >>>>>> >>>>> >>>>> With option 2, we will end up with many top level projects under >>>>> wso2/. >>>>> >>>>>> >>>>>> We need to figure out whether git supports #1 and if not go for #2. >>>>>> >>>>>> Also, wanted to highlight that we will no longer have service-stubs, >>>>>> and separate features categories. There will be no concept of parent, >>>>>> since >>>>>> each feature will define its own dependencies *excluding transitive >>>>>> dependencies*. The development governance system that we put in >>>>>> place will ensure we reuse the same versions and a separate pom is >>>>>> unwanted. >>>>>> >>>>>> Further, all branched dependencies and orbits will reside outside of >>>>>> our code structure, and those should follow separate release models as >>>>>> discussed previously. >>>>>> >>>>>> Next, we will integrate GerritHub for code reviews. >>>>>> >>>>>> Thanks, >>>>>> Senaka. >>>>>> >>>>>> On Sat, Jan 18, 2014 at 10:31 AM, Afkham Azeez <[email protected]>wrote: >>>>>> >>>>>>> Shall we name those as; >>>>>>> >>>>>>> - carbon-feature-registry >>>>>>> - carbon-feature-governance >>>>>>> - carbon-feature-identity >>>>>>> - carbon-feature-mediation >>>>>>> - carbon-feature-analytics >>>>>>> - carbon-feature-data >>>>>>> - carbon-feature-apis >>>>>>> - carbon-feature-business-process >>>>>>> - carbon-feature-business-messaging >>>>>>> - carbon-feature-rules >>>>>>> - carbon-feature-deployment >>>>>>> - carbon-feature-qos >>>>>>> - carbon-feature-utils >>>>>>> >>>>>>> and also have; >>>>>>> * carbon-product-appserver >>>>>>> * carbon-product-esb >>>>>>> >>>>>>> and so on. >>>>>>> >>>>>>> Also; >>>>>>> carbon-p2-repo >>>>>>> >>>>>>> carbon-platform-integration-tests >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sat, Jan 18, 2014 at 9:12 AM, Eranda Sooriyabandara < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Infra, >>>>>>>> Can we have following projects created in the git repo. >>>>>>>> Additionally >>>>>>>> >>>>>>>> - registry >>>>>>>> - governance >>>>>>>> - identity >>>>>>>> - mediation >>>>>>>> - analytics >>>>>>>> - data >>>>>>>> - apis >>>>>>>> - business-process >>>>>>>> - business-messaging >>>>>>>> - rules >>>>>>>> - deployment >>>>>>>> - qos >>>>>>>> - utils >>>>>>>> >>>>>>>> Additionally please add me (erandasooriyabandara) to WSO2 member >>>>>>>> list. >>>>>>>> >>>>>>>> thanks >>>>>>>> Eranda >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jan 17, 2014 at 9:26 PM, Afkham Azeez <[email protected]>wrote: >>>>>>>> >>>>>>>>> [Sorry for the very long mail. I want to document all that I had >>>>>>>>> in mind & the stuff we discussed. I would recommend all devs to take >>>>>>>>> some >>>>>>>>> time to read this] >>>>>>>>> >>>>>>>>> >>>>>>>>> I would like to summarize he discussion we had a couple of days >>>>>>>>> back. >>>>>>>>> >>>>>>>>> *The Problems* >>>>>>>>> The problems we are trying to solve are as follows: >>>>>>>>> >>>>>>>>> 1. Trunk & branches structures being completely different >>>>>>>>> 2. Branches containing directories with version numbers >>>>>>>>> 3. It is impossible to move to GitHub with the current structure >>>>>>>>> because of #2 >>>>>>>>> 4. It is very easy to break the build by changing already released >>>>>>>>> code. The room for human error is high. >>>>>>>>> 5. Bamboo builds are eternally broken because the build fails at >>>>>>>>> some point & Bamboo cannot continue any further >>>>>>>>> 6. When we branch, the trunk quickly becomes obsolete, and remains >>>>>>>>> in that broken state until the next major platform release. >>>>>>>>> 7. Everybody has to build all components/features, even if those >>>>>>>>> are not related to their products >>>>>>>>> 8. Fixed versions in branches instead of using SNAPSHOT versions. >>>>>>>>> This makes it impossible to upload build artifacts to Maven/Nexus >>>>>>>>> repos. >>>>>>>>> This leads to #7. >>>>>>>>> 9. Impossible to integrate code quality tools such as EraInsight >>>>>>>>> because of #5 >>>>>>>>> >>>>>>>>> *Proposed solution* >>>>>>>>> We have come up with the following solution after much >>>>>>>>> deliberation & thought. >>>>>>>>> >>>>>>>>> Rationale: >>>>>>>>> We started looking at other open source projects out there. We >>>>>>>>> took Axis2 as an example. Axis2 had many dependencies including Axiom, >>>>>>>>> XmlSchema, Woden, WSS4J etc. Those 3rd party dependencies were also >>>>>>>>> developed by some Axis2 contributors, but we never branched all of >>>>>>>>> those >>>>>>>>> together and brought them into the same code branch. We used to start >>>>>>>>> what >>>>>>>>> we used to call a release train, where the upstream code would have >>>>>>>>> to be >>>>>>>>> released first before the downstream code such as Axis2 & Synapse >>>>>>>>> could be >>>>>>>>> released. This way, we never had any of the problems outlined above. >>>>>>>>> >>>>>>>>> If you look at the WSO2 product releases, again, the scenario is >>>>>>>>> not that much different from the Axis2/Synapse releases. Components & >>>>>>>>> features are simply dependencies of the products. In order to get >>>>>>>>> product >>>>>>>>> releases out, we first need releases of those >>>>>>>>> dependencies (components/features). So the proposal is to identify >>>>>>>>> the top >>>>>>>>> level components/features, and first release that code before doing >>>>>>>>> product >>>>>>>>> releases. Each of those components will have a GitHub repo. All active >>>>>>>>> development will be done on the main branch of those components, and >>>>>>>>> will >>>>>>>>> be branched when they are close to the release. Instead of granting >>>>>>>>> commit >>>>>>>>> rights to all, we could only allow the primary developers of those >>>>>>>>> components to commit, and others can send pull requests, which will be >>>>>>>>> merged in by the primary owners after reviewing. These component teams >>>>>>>>> could even have an internal Git repo or clone, and commit to that, >>>>>>>>> run all >>>>>>>>> tests, and when they are satisfied that the code is in a good state >>>>>>>>> to be >>>>>>>>> merged into the main branch, they can send a pull request & merge the >>>>>>>>> changes. We would run Bamboo which would build the components several >>>>>>>>> times >>>>>>>>> a day & deploy them to the Nexus repo. That way, you would not have to >>>>>>>>> build code that is not directly related to what you are working on, >>>>>>>>> which >>>>>>>>> would save 100s of valuable dev hours. >>>>>>>>> >>>>>>>>> In the majority of the cases, the P2 features correspond to one or >>>>>>>>> more related components. So we would keep the feature close to the >>>>>>>>> component. There are some cases where these components & features >>>>>>>>> belong in >>>>>>>>> the product itself. AppFactory components are a good example. >>>>>>>>> >>>>>>>>> Products will also have their own GitHub repos. We will have >>>>>>>>> separate GitHub repos for platform integration tests. Products such >>>>>>>>> as API >>>>>>>>> Manager may opt to have the API Management feature in the product >>>>>>>>> itself. >>>>>>>>> We would have a separate P2-repo GitHub repo which would build & >>>>>>>>> deploy the >>>>>>>>> compatible set of P2 features. >>>>>>>>> >>>>>>>>> We will have Bamboo plans at multiple levels; components, >>>>>>>>> products, platform, p2-repo and so on. >>>>>>>>> >>>>>>>>> We will not change any package structures at this time. We would >>>>>>>>> defer that to Carbon 5 (if necessary). We will get rid of the chunk >>>>>>>>> based >>>>>>>>> release model. Continuous delivery is our ultimate aim & for that to >>>>>>>>> happen, we have to release a compatible set of features. We are going >>>>>>>>> to >>>>>>>>> rely heavily on automation, automated builds & deploys to Maven. The >>>>>>>>> developers will do the majority of the QA (just like the Apache >>>>>>>>> Stratos >>>>>>>>> team is doing now). >>>>>>>>> >>>>>>>>> We will use the Maven release plugin to create releases & upload >>>>>>>>> them to the Maven repo. >>>>>>>>> >>>>>>>>> *Implications to WSO2 code developers* >>>>>>>>> 1. Life becomes easier because you don't have to spend a lot of >>>>>>>>> time building unrelated stuff >>>>>>>>> 2. General development would happen in the main branch. >>>>>>>>> 3. Close to a release, branches would be cut. >>>>>>>>> 4. Once release branches are cut, fixes would be done in the main >>>>>>>>> branch, and pull requests would be sent to the branch & merged in. >>>>>>>>> This >>>>>>>>> will be easy because we no longer have the trunk & branch structure >>>>>>>>> being >>>>>>>>> different. >>>>>>>>> >>>>>>>>> >>>>>>>>> *Implications to users* >>>>>>>>> Users will not see any difference in the components & features >>>>>>>>> because we are not changing packages or binary structure. In fact, we >>>>>>>>> would >>>>>>>>> have a P2 repo with compatible features which all work together. >>>>>>>>> >>>>>>>>> Senaka, Isuruwan & Harshana have already started working on a PoC. >>>>>>>>> >>>>>>>>> Thoughts welcome. Those who were in these discussions, please add >>>>>>>>> anything I may have missed. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Afkham Azeez* >>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>> * <http://www.apache.org/>* >>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>> *twitter: >>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>> >>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Eranda Sooriyabandara*Senior Software Engineer; >>>>>>>> Integration Technologies Team; >>>>>>>> WSO2 Inc.; http://wso2.com >>>>>>>> Lean . Enterprise . Middleware >>>>>>>> >>>>>>>> E-mail: eranda AT wso2.com >>>>>>>> Mobile: +94 716 472 816 >>>>>>>> Linked-In: http://www.linkedin.com/in/erandasooriyabandara >>>>>>>> Blog: http://emsooriyabandara.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Afkham Azeez* >>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>> * <http://www.apache.org/>* >>>>>>> *email: **[email protected]* <[email protected]> >>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>> *twitter: >>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>> >>>>>>> *Lean . Enterprise . Middleware* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Senaka Fernando* >>>>>> Senior Technical Lead; WSO2 Inc.; http://wso2.com >>>>>> >>>>>> >>>>>> >>>>>> * Member; Apache Software Foundation; http://apache.org >>>>>> <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: >>>>>> +1 408 754 7388 <%2B1%20408%20754%207388>; ext: 51736*; >>>>>> >>>>>> >>>>>> *M: +94 77 322 1818 <%2B94%2077%20322%201818> Linked-In: >>>>>> http://linkedin.com/in/senakafernando >>>>>> <http://linkedin.com/in/senakafernando>* >>>>>> Lean . Enterprise . Middleware >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Sameera Jayasoma, >>>>> Architect, >>>>> >>>>> WSO2, Inc. (http://wso2.com) >>>>> email: [email protected] >>>>> blog: http://sameera.adahas.org >>>>> twitter: https://twitter.com/sameerajayasoma >>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >>>>> Mobile: 0094776364456 >>>>> >>>>> >>>>> Lean . Enterprise . Middleware >>>>> >>>> >>>> >>>> >>>> -- >>>> S.Uthaiyashankar >>>> VP Engineering >>>> WSO2 Inc. >>>> http://wso2.com/ - "lean . enterprise . middleware" >>>> >>>> Phone: +94 714897591 >>>> >>> >>> >>> >>> -- >>> Sameera Jayasoma, >>> Architect, >>> >>> WSO2, Inc. (http://wso2.com) >>> email: [email protected] >>> blog: http://sameera.adahas.org >>> twitter: https://twitter.com/sameerajayasoma >>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >>> Mobile: 0094776364456 >>> >>> Lean . Enterprise . Middleware >>> >> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> > > > > -- > > > *[image: http://wso2.com] <http://wso2.com> Senaka Fernando* > Senior Technical Lead; WSO2 Inc.; http://wso2.com > > > > * Member; Apache Software Foundation; http://apache.org > <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: +1 > 408 754 7388 <%2B1%20408%20754%207388>; ext: 51736*; > > > *M: +94 77 322 1818 <%2B94%2077%20322%201818> Linked-In: > http://linkedin.com/in/senakafernando > <http://linkedin.com/in/senakafernando>*Lean . Enterprise . Middleware > -- Shevan Goonetilleke Director of Engineering WSO2, Inc. lean.enterprise.middleware m: +94777340680 w: http://wso2.com
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
