Hi all, Azeez is correct. But even with #2, there are productive ways of managing that. Read [1] for example. Oh and BTW, our git repositories better have some good conventions. Right now, [2] is a little messy.
[1] http://www.lshift.net/blog/2013/06/27/managing-multiple-github-repositories [2] https://github.com/wso2 Thanks, Senaka. On Sun, Jan 19, 2014 at 11:47 PM, Afkham Azeez <[email protected]> wrote: > > > > On Sun, Jan 19, 2014 at 11:34 PM, Sameera Jayasoma <[email protected]>wrote: > >> >> >> >> On Sun, Jan 19, 2014 at 9:53 AM, Selvaratnam Uthaiyashankar < >> [email protected]> wrote: >> >>> >>> >>> >>> On Sun, Jan 19, 2014 at 11:02 PM, Sameera Jayasoma <[email protected]>wrote: >>> >>>> Hi Azeez, >>>> >>>> +1 for this model. This will solve most of the issues that we are >>>> facing in the current system. >>>> >>>> >>>> Senaka, see my comment below. >>>> >>>> On Sat, Jan 18, 2014 at 10:48 AM, Senaka Fernando <[email protected]>wrote: >>>> >>>>> Hi Azeez, Eranda, >>>>> >>>>> +1 for the proposed changes. So, as discussed we have two options. >>>>> >>>>> 1. In the git repo, go for wso2/carbon/feature or wso2/carbon/product >>>>> model and then have features such as registry, governance and products >>>>> such >>>>> as esb. >>>>> >>>> >>>> >>>> I am + for the Option 1 if possible. >>>> >>> >>> >>> With this option, shouldn't we checkout the whole components? AFAIK, GIt >>> doesn't support checking out part of the repo. Also, the release, branching >>> etc. will be at the repo level, can't be at a sub repo level. Above >>> structure is what we have now and is the reason for most of the trouble we >>> are having, isn't it? >>> >> >> No Shankar, all the components will be separate projects on their own. >> But this is just a matter of organizing them in a proper order without >> putting everything to the top level. (only if this is possible.) >> >> e.g. https://github.com/wso2/carbon/components/identity >> > > You can't do that. It will create your repo as, carbon-components-identity > if you try to do that. > > >> >> If this is not possible, then I prefer *carbon-component-* instead of >> carbon-feature-*. * Because IMHO, components are what we develop and >> features are just a way of installing these components to projects. We may >> end up multiple features for a given component or features which combine >> multiple components. >> >> >> >>> >>> >>> >>>> That will allows developers to easily navigate to the relevant >>>> component. >>>> >>>> How about using wso2/carbon/components instead of features. What we >>>> really develop is components? I would prefer to use the word component >>>> rather than using the word feature. >>>> >>>> WDYT? >>>> >>>> >>>>> 2. Go with the model Azeez wrote in e-mail. >>>>> >>>> >>>> With option 2, we will end up with many top level projects under wso2/. >>>> >>>> >>>>> >>>>> We need to figure out whether git supports #1 and if not go for #2. >>>>> >>>>> Also, wanted to highlight that we will no longer have service-stubs, >>>>> and separate features categories. There will be no concept of parent, >>>>> since >>>>> each feature will define its own dependencies *excluding transitive >>>>> dependencies*. The development governance system that we put in place >>>>> will ensure we reuse the same versions and a separate pom is unwanted. >>>>> >>>>> Further, all branched dependencies and orbits will reside outside of >>>>> our code structure, and those should follow separate release models as >>>>> discussed previously. >>>>> >>>>> Next, we will integrate GerritHub for code reviews. >>>>> >>>>> Thanks, >>>>> Senaka. >>>>> >>>>> On Sat, Jan 18, 2014 at 10:31 AM, Afkham Azeez <[email protected]> wrote: >>>>> >>>>>> Shall we name those as; >>>>>> >>>>>> - carbon-feature-registry >>>>>> - carbon-feature-governance >>>>>> - carbon-feature-identity >>>>>> - carbon-feature-mediation >>>>>> - carbon-feature-analytics >>>>>> - carbon-feature-data >>>>>> - carbon-feature-apis >>>>>> - carbon-feature-business-process >>>>>> - carbon-feature-business-messaging >>>>>> - carbon-feature-rules >>>>>> - carbon-feature-deployment >>>>>> - carbon-feature-qos >>>>>> - carbon-feature-utils >>>>>> >>>>>> and also have; >>>>>> * carbon-product-appserver >>>>>> * carbon-product-esb >>>>>> >>>>>> and so on. >>>>>> >>>>>> Also; >>>>>> carbon-p2-repo >>>>>> >>>>>> carbon-platform-integration-tests >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Jan 18, 2014 at 9:12 AM, Eranda Sooriyabandara < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Infra, >>>>>>> Can we have following projects created in the git repo. Additionally >>>>>>> >>>>>>> - registry >>>>>>> - governance >>>>>>> - identity >>>>>>> - mediation >>>>>>> - analytics >>>>>>> - data >>>>>>> - apis >>>>>>> - business-process >>>>>>> - business-messaging >>>>>>> - rules >>>>>>> - deployment >>>>>>> - qos >>>>>>> - utils >>>>>>> >>>>>>> Additionally please add me (erandasooriyabandara) to WSO2 member >>>>>>> list. >>>>>>> >>>>>>> thanks >>>>>>> Eranda >>>>>>> >>>>>>> >>>>>>> On Fri, Jan 17, 2014 at 9:26 PM, Afkham Azeez <[email protected]>wrote: >>>>>>> >>>>>>>> [Sorry for the very long mail. I want to document all that I had in >>>>>>>> mind & the stuff we discussed. I would recommend all devs to take some >>>>>>>> time >>>>>>>> to read this] >>>>>>>> >>>>>>>> >>>>>>>> I would like to summarize he discussion we had a couple of days >>>>>>>> back. >>>>>>>> >>>>>>>> *The Problems* >>>>>>>> The problems we are trying to solve are as follows: >>>>>>>> >>>>>>>> 1. Trunk & branches structures being completely different >>>>>>>> 2. Branches containing directories with version numbers >>>>>>>> 3. It is impossible to move to GitHub with the current structure >>>>>>>> because of #2 >>>>>>>> 4. It is very easy to break the build by changing already released >>>>>>>> code. The room for human error is high. >>>>>>>> 5. Bamboo builds are eternally broken because the build fails at >>>>>>>> some point & Bamboo cannot continue any further >>>>>>>> 6. When we branch, the trunk quickly becomes obsolete, and remains >>>>>>>> in that broken state until the next major platform release. >>>>>>>> 7. Everybody has to build all components/features, even if those >>>>>>>> are not related to their products >>>>>>>> 8. Fixed versions in branches instead of using SNAPSHOT versions. >>>>>>>> This makes it impossible to upload build artifacts to Maven/Nexus >>>>>>>> repos. >>>>>>>> This leads to #7. >>>>>>>> 9. Impossible to integrate code quality tools such as EraInsight >>>>>>>> because of #5 >>>>>>>> >>>>>>>> *Proposed solution* >>>>>>>> We have come up with the following solution after much deliberation >>>>>>>> & thought. >>>>>>>> >>>>>>>> Rationale: >>>>>>>> We started looking at other open source projects out there. We took >>>>>>>> Axis2 as an example. Axis2 had many dependencies including Axiom, >>>>>>>> XmlSchema, Woden, WSS4J etc. Those 3rd party dependencies were also >>>>>>>> developed by some Axis2 contributors, but we never branched all of >>>>>>>> those >>>>>>>> together and brought them into the same code branch. We used to start >>>>>>>> what >>>>>>>> we used to call a release train, where the upstream code would have to >>>>>>>> be >>>>>>>> released first before the downstream code such as Axis2 & Synapse >>>>>>>> could be >>>>>>>> released. This way, we never had any of the problems outlined above. >>>>>>>> >>>>>>>> If you look at the WSO2 product releases, again, the scenario is >>>>>>>> not that much different from the Axis2/Synapse releases. Components & >>>>>>>> features are simply dependencies of the products. In order to get >>>>>>>> product >>>>>>>> releases out, we first need releases of those >>>>>>>> dependencies (components/features). So the proposal is to identify the >>>>>>>> top >>>>>>>> level components/features, and first release that code before doing >>>>>>>> product >>>>>>>> releases. Each of those components will have a GitHub repo. All active >>>>>>>> development will be done on the main branch of those components, and >>>>>>>> will >>>>>>>> be branched when they are close to the release. Instead of granting >>>>>>>> commit >>>>>>>> rights to all, we could only allow the primary developers of those >>>>>>>> components to commit, and others can send pull requests, which will be >>>>>>>> merged in by the primary owners after reviewing. These component teams >>>>>>>> could even have an internal Git repo or clone, and commit to that, run >>>>>>>> all >>>>>>>> tests, and when they are satisfied that the code is in a good state to >>>>>>>> be >>>>>>>> merged into the main branch, they can send a pull request & merge the >>>>>>>> changes. We would run Bamboo which would build the components several >>>>>>>> times >>>>>>>> a day & deploy them to the Nexus repo. That way, you would not have to >>>>>>>> build code that is not directly related to what you are working on, >>>>>>>> which >>>>>>>> would save 100s of valuable dev hours. >>>>>>>> >>>>>>>> In the majority of the cases, the P2 features correspond to one or >>>>>>>> more related components. So we would keep the feature close to the >>>>>>>> component. There are some cases where these components & features >>>>>>>> belong in >>>>>>>> the product itself. AppFactory components are a good example. >>>>>>>> >>>>>>>> Products will also have their own GitHub repos. We will have >>>>>>>> separate GitHub repos for platform integration tests. Products such as >>>>>>>> API >>>>>>>> Manager may opt to have the API Management feature in the product >>>>>>>> itself. >>>>>>>> We would have a separate P2-repo GitHub repo which would build & >>>>>>>> deploy the >>>>>>>> compatible set of P2 features. >>>>>>>> >>>>>>>> We will have Bamboo plans at multiple levels; components, products, >>>>>>>> platform, p2-repo and so on. >>>>>>>> >>>>>>>> We will not change any package structures at this time. We would >>>>>>>> defer that to Carbon 5 (if necessary). We will get rid of the chunk >>>>>>>> based >>>>>>>> release model. Continuous delivery is our ultimate aim & for that to >>>>>>>> happen, we have to release a compatible set of features. We are going >>>>>>>> to >>>>>>>> rely heavily on automation, automated builds & deploys to Maven. The >>>>>>>> developers will do the majority of the QA (just like the Apache Stratos >>>>>>>> team is doing now). >>>>>>>> >>>>>>>> We will use the Maven release plugin to create releases & upload >>>>>>>> them to the Maven repo. >>>>>>>> >>>>>>>> *Implications to WSO2 code developers* >>>>>>>> 1. Life becomes easier because you don't have to spend a lot of >>>>>>>> time building unrelated stuff >>>>>>>> 2. General development would happen in the main branch. >>>>>>>> 3. Close to a release, branches would be cut. >>>>>>>> 4. Once release branches are cut, fixes would be done in the main >>>>>>>> branch, and pull requests would be sent to the branch & merged in. This >>>>>>>> will be easy because we no longer have the trunk & branch structure >>>>>>>> being >>>>>>>> different. >>>>>>>> >>>>>>>> >>>>>>>> *Implications to users* >>>>>>>> Users will not see any difference in the components & features >>>>>>>> because we are not changing packages or binary structure. In fact, we >>>>>>>> would >>>>>>>> have a P2 repo with compatible features which all work together. >>>>>>>> >>>>>>>> Senaka, Isuruwan & Harshana have already started working on a PoC. >>>>>>>> >>>>>>>> Thoughts welcome. Those who were in these discussions, please add >>>>>>>> anything I may have missed. >>>>>>>> >>>>>>>> -- >>>>>>>> *Afkham Azeez* >>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>> * <http://www.apache.org/>* >>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>> *twitter: >>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>> >>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Eranda Sooriyabandara*Senior Software Engineer; >>>>>>> Integration Technologies Team; >>>>>>> WSO2 Inc.; http://wso2.com >>>>>>> Lean . Enterprise . Middleware >>>>>>> >>>>>>> E-mail: eranda AT wso2.com >>>>>>> Mobile: +94 716 472 816 >>>>>>> Linked-In: http://www.linkedin.com/in/erandasooriyabandara >>>>>>> Blog: http://emsooriyabandara.blogspot.com/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Afkham Azeez* >>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>> * <http://www.apache.org/>* >>>>>> *email: **[email protected]* <[email protected]> >>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * >>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>> *twitter: >>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>> * linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>> >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Senaka Fernando* >>>>> Senior Technical Lead; WSO2 Inc.; http://wso2.com >>>>> >>>>> >>>>> >>>>> * Member; Apache Software Foundation; http://apache.org >>>>> <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: >>>>> +1 408 754 7388 <%2B1%20408%20754%207388>; ext: 51736*; >>>>> >>>>> >>>>> *M: +94 77 322 1818 <%2B94%2077%20322%201818> Linked-In: >>>>> http://linkedin.com/in/senakafernando >>>>> <http://linkedin.com/in/senakafernando>* >>>>> Lean . Enterprise . Middleware >>>>> >>>> >>>> >>>> >>>> -- >>>> Sameera Jayasoma, >>>> Architect, >>>> >>>> WSO2, Inc. (http://wso2.com) >>>> email: [email protected] >>>> blog: http://sameera.adahas.org >>>> twitter: https://twitter.com/sameerajayasoma >>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >>>> Mobile: 0094776364456 >>>> >>>> >>>> Lean . Enterprise . Middleware >>>> >>> >>> >>> >>> -- >>> S.Uthaiyashankar >>> VP Engineering >>> WSO2 Inc. >>> http://wso2.com/ - "lean . enterprise . middleware" >>> >>> Phone: +94 714897591 >>> >> >> >> >> -- >> Sameera Jayasoma, >> Architect, >> >> WSO2, Inc. (http://wso2.com) >> email: [email protected] >> blog: http://sameera.adahas.org >> twitter: https://twitter.com/sameerajayasoma >> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections >> Mobile: 0094776364456 >> >> Lean . Enterprise . Middleware >> > > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>* > *email: **[email protected]* <[email protected]> > * cell: +94 77 3320919 <%2B94%2077%203320919> blog: * > *http://blog.afkham.org* <http://blog.afkham.org> > *twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> > * linked-in: **http://lk.linkedin.com/in/afkhamazeez > <http://lk.linkedin.com/in/afkhamazeez>* > > *Lean . Enterprise . Middleware* > -- *[image: http://wso2.com] <http://wso2.com> Senaka Fernando* Senior Technical Lead; WSO2 Inc.; http://wso2.com * Member; Apache Software Foundation; http://apache.org <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 Linked-In: http://linkedin.com/in/senakafernando <http://linkedin.com/in/senakafernando>*Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
