Re: More selective building option
I agree with Mike's point. Given the code structure as it stands we should be doing a full build before checking changes is. This doesn't prevent any kind of profile being created for intermediate builds but on check in we need to ensure that everything works. You will of course say but we don't do that sometimes and I agree however the point is that we should do it. Now, could we do things differently? Firstly is there a way of getting maven to build a module and all the modules that depend on it (either directly or transitively)? If not can we generate build files for another system that will do incremental builds (I remember that we used to have an incremental builder at some point in the past but I don't know how good it was) Alternatively we have to take a manual approach. I see the code separated into a core and the extensions that the core supports. We could make some rules/profiles for the types of build you need to do depending on which files you change. For example, core change = build all while extension change = build the extension and its tests and samples. Alternatively, rely more heavily on the contract between core and extensions so that features can be built independently. None of this says anything about how the code is released b.t.w. That is a separate question. Simon
Re: More selective building option
On Tue, Sep 1, 2009 at 12:50 PM, Simon Lawssimonsl...@googlemail.com wrote: Alternatively we have to take a manual approach. I see the code separated into a core and the extensions that the core supports. We could make some rules/profiles for the types of build you need to do depending on which files you change. For example, core change = build all while extension change = build the extension and its tests and samples. Alternatively, rely more heavily on the contract between core and extensions so that features can be built independently. Yep, I'm pretty sure we will need to do that to keep 2.x builds relatively concise. As we do that there will be build breaks because the unit tests for the core contracts aren't complete and instead we rely on the various extension tests to verify them, but we can improve them as things get noticed over time. We're unlikely to agree in the near future on a big bang type change to move over to any new build structure like that so IMHO what Giorgio is doing with a profile that suits himself seems like a fine way to make a gradual start on this. ...ant
Re: More selective building option
structure like that so IMHO what Giorgio is doing with a profile that suits himself seems like a fine way to make a gradual start on this. Absolutely, this isn't about moving code about in the repository it's about understanding which profiles/tests/contracts support more granular builds. Simon
Re: More selective building option
2009/9/1 Simon Laws simonsl...@googlemail.com: structure like that so IMHO what Giorgio is doing with a profile that suits himself seems like a fine way to make a gradual start on this. Absolutely, this isn't about moving code about in the repository it's about understanding which profiles/tests/contracts support more granular builds. Suppose that you don't need all nuts and belt of your SCA distribution. For example you plan to use for your embedded enviroment...a mobile phone or a network equipment. What will you provide to support that scenario? I agree with Mike when he says that we always should build all the stuff, if we want to commit something (where builds means: download a new svn tree and build with maven). However if we compartamentalize dependencies i guess that's is feasible to identify some part of the infrastructure that are loosely or tighly connected. And one could work on with a small set of the infrastructure without breaking things. Just 1c, Giorgio. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda.
Re: More selective building option
Giorgio Zoppi wrote: However if we compartamentalize dependencies i guess that's is feasible to identify some part of the infrastructure that are loosely or tighly connected. And one could work on with a small set of the infrastructure without breaking things. Just 1c, Giorgio. Folks, I am sceptical at the moment about being able to do this in a way that does not cause a lot of waves when committing changes. That is probably an admission that our module structure is not good and that there are too many cross dependencies, but we are where we are - and all I am saying is that my experience in this area recently has not been good. Yours, Mike.
Re: More selective building option
On Fri, Aug 28, 2009 at 9:34 PM, Mike Edwardsmike.edwards.inglen...@gmail.com wrote: ant elder wrote: I see no issue at all with adding a (or several) new profiles that only build a subset of modules that anyone finds a useful combination, and they could be just committed without needing any consensus. The problems would come if we tried to make one of those subsets the build default. Also a worry is that with the testing spread out in lots of places we may not be testing properly by only building subsets, although that is partially overcome by having the Hudson nightly build keep building everything. ...ant Ant, You've hit the nail on the head there Ant. It's OK doing partial builds UNTIL you want to commit something back to SVN. Then, you must do a full build - otherwise you will have no idea what you are screwing up. Seemingly innocuous changes can have significant impacts on other parts of the code - and stop other folk in their tracks. This has happened to me personally over the past few months and it isn't much fun. Nightly builds are not good enough - if you commit stuff without the full build, you've potentially already messed up other folk by not checking that your changes don't cause trouble to others. Yours, Mike. But consider that when the the build gets so big as 1.x the new or infrequent developers may not even get to see a build break as they struggle with just getting the time or dependencies downloaded. The reality is people do commit without running a full build even in 2.x where the build time relatively quick compared to 1.x. As we add more stuff to 2.x the build time will increase and that will happen more and more unless we find new ways to structure things. ...ant
Re: More selective building option
On Fri, Aug 28, 2009 at 12:56 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: Hi guys, Is feasible (i'm not a great maven expert) a more selective building options? Because Tuscany is getting bigger, and as user i would expect to choose what compile or not without editing any pom. AFAIK, what maven provides are profiles, which can configure which modules would be build... then, as a user, you would do mvn -Pprofile... is this something that would help you ? We have couple profiles defined today in the sca pom. -- Luciano Resende http://people.apache.org/~lresende http://lresende.blogspot.com/
Re: More selective building option
2009/8/28 Luciano Resende luckbr1...@gmail.com: On Fri, Aug 28, 2009 at 12:56 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: Hi guys, Is feasible (i'm not a great maven expert) a more selective building options? Because Tuscany is getting bigger, and as user i would expect to choose what compile or not without editing any pom. AFAIK, what maven provides are profiles, which can configure which modules would be build... then, as a user, you would do mvn -Pprofile... is this something that would help you ? We have couple profiles defined today in the sca pom. That's ok :). -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda.
Re: More selective building option
2009/8/28 ant elder ant.el...@gmail.com: On Fri, Aug 28, 2009 at 8:56 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: Hi guys, Is feasible (i'm not a great maven expert) a more selective building options? Because Tuscany is getting bigger, and as user i would expect to choose what compile or not without editing any pom. Cheers, Giorgio. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda. Other than the Maven profiles or building from some point down inside the project hierarchy there isn't anything, but i wonder if we should do something. In 1.x we always build/test everything and that means the build takes ages especially the first time when all the dependencies need to be downloaded, and this is off putting for new developers. 2.x isn't so bad yet but it is starting to take longer as we're add more extensions. Early on in 2.x we did talk about doing something about this, perhaps splitting things so some of the extensions are optional and not built and tested as part of the main core build. The hard bit would be coming up with a list of what is optional that we all agree with, should we try to do that? I'm preparing a minimal maven profile...that you could use as minimal tuscany compilation. For example i don't need spring integration, rmi, jaxws (as i'd prefer use axis), the binding-json runtime and all the osgi stuff. However we could agree before raising a JIRA we could agree on what we intend as minimal tuscany. Cheers, Giorgio. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda.
Re: More selective building option
On Fri, Aug 28, 2009 at 9:30 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: 2009/8/28 ant elder ant.el...@gmail.com: On Fri, Aug 28, 2009 at 8:56 AM, Giorgio Zoppigiorgio.zo...@gmail.com wrote: Hi guys, Is feasible (i'm not a great maven expert) a more selective building options? Because Tuscany is getting bigger, and as user i would expect to choose what compile or not without editing any pom. Cheers, Giorgio. -- Quiero ser el rayo de sol que cada día te despierta para hacerte respirar y vivir en me. Favola -Moda. Other than the Maven profiles or building from some point down inside the project hierarchy there isn't anything, but i wonder if we should do something. In 1.x we always build/test everything and that means the build takes ages especially the first time when all the dependencies need to be downloaded, and this is off putting for new developers. 2.x isn't so bad yet but it is starting to take longer as we're add more extensions. Early on in 2.x we did talk about doing something about this, perhaps splitting things so some of the extensions are optional and not built and tested as part of the main core build. The hard bit would be coming up with a list of what is optional that we all agree with, should we try to do that? I'm preparing a minimal maven profile...that you could use as minimal tuscany compilation. For example i don't need spring integration, rmi, jaxws (as i'd prefer use axis), the binding-json runtime and all the osgi stuff. However we could agree before raising a JIRA we could agree on what we intend as minimal tuscany. Cheers, Giorgio. I see no issue at all with adding a (or several) new profiles that only build a subset of modules that anyone finds a useful combination, and they could be just committed without needing any consensus. The problems would come if we tried to make one of those subsets the build default. Also a worry is that with the testing spread out in lots of places we may not be testing properly by only building subsets, although that is partially overcome by having the Hudson nightly build keep building everything. ...ant
Re: More selective building option
ant elder wrote: I see no issue at all with adding a (or several) new profiles that only build a subset of modules that anyone finds a useful combination, and they could be just committed without needing any consensus. The problems would come if we tried to make one of those subsets the build default. Also a worry is that with the testing spread out in lots of places we may not be testing properly by only building subsets, although that is partially overcome by having the Hudson nightly build keep building everything. ...ant Ant, You've hit the nail on the head there Ant. It's OK doing partial builds UNTIL you want to commit something back to SVN. Then, you must do a full build - otherwise you will have no idea what you are screwing up. Seemingly innocuous changes can have significant impacts on other parts of the code - and stop other folk in their tracks. This has happened to me personally over the past few months and it isn't much fun. Nightly builds are not good enough - if you commit stuff without the full build, you've potentially already messed up other folk by not checking that your changes don't cause trouble to others. Yours, Mike.
Re: More selective building option
On Fri, Aug 28, 2009 at 1:34 PM, Mike Edwardsmike.edwards.inglen...@gmail.com wrote: ant elder wrote: I see no issue at all with adding a (or several) new profiles that only build a subset of modules that anyone finds a useful combination, and they could be just committed without needing any consensus. The problems would come if we tried to make one of those subsets the build default. Also a worry is that with the testing spread out in lots of places we may not be testing properly by only building subsets, although that is partially overcome by having the Hudson nightly build keep building everything. ...ant Ant, You've hit the nail on the head there Ant. It's OK doing partial builds UNTIL you want to commit something back to SVN. Then, you must do a full build - otherwise you will have no idea what you are screwing up. Seemingly innocuous changes can have significant impacts on other parts of the code - and stop other folk in their tracks. This has happened to me personally over the past few months and it isn't much fun. Nightly builds are not good enough - if you commit stuff without the full build, you've potentially already messed up other folk by not checking that your changes don't cause trouble to others. Yours, Mike. +1, totally agree with Mike. Having said that, although the simple profiles would work perfectly in 1.x, they won't really work for 2.x, as we need to generate the 3rd party dependency bundles during the distribution processing in order to things to work. A better solution would probably be to align these profiles to a distribution which might solve build speed, but will increase the build size on disk when building the full build before a commit. -- Luciano Resende http://people.apache.org/~lresende http://lresende.blogspot.com/
