Hi Rob, Thanks for these thoughts... sounds like we are somewhat in the same boat in that we want to push out new releases, but don't know enough about the (state of the) other modules to know whether it is safe to do so.
And so, yes, breaking up the codebase into separate submodules is the obvious next step, and something that you and I have discussed several times in the past off-list. I don't have any issue with doing it myself, it's an obvious evolution from my proposal. The concept of an edition, then, I think equates to an archetype: it's a project template to get a would-be used up-and-running quickly with a precanned set of modules. What I therefore suggest is that we: 1. wait for Infra to move our code from svn to git (soon, I hope...) 2. flatten out the modules as I outlined in the wiki page 3. raise another ticket for infra to create new repos for each of the modules that we want to release separately... seems to me that these could start out as clones of the original 4. when those new submodule repos exist, we can zap remove all the foreign stuff; conversely, in the original repo, we zap the code for which there are now submodule repos The new repos that I see us needing are: - viewer-wicket - viewer-scimpi - viewer-restful - viewer-html - objectstore-jdo - objectstore-nosql - objectstore-sql - editions // for the various archetypes we might want to create. For the other modules that are not core (eg progmodel-groovy, or viewer-junit), I'd rather leave them in the "default" repo, but not release them until there is a need. For these we can use a Maven profile to exclude them from the release process. Although overall this introduces a little bit more formality into the proceedings, I do think that the time is now right to do this. And, I think it properly answers all the objections that Giovanni raised earlier in this thread. What say we all? Dan On 29 November 2012 20:00, Robert Matthews <[email protected]>wrote: > First and foremost, I think we should release things more granularly so > individual components (most of our Maven modules) can be worked on more > independently. I would find it great to be able to work on improving the > NOSQL store or the Scimpi viewer without having to consider that every > other module has to be got into a releasable state just so one ready module > can be released. Most of the modules, outside of the core, are fairly > independent, typically relying only on the core and will work other > versions of other components, eg a viewer is not typically dependent on an > object store. Basically, we have the core modules, which need to be > maintained and released together, then we have a whole pile of major > component, mostly that can be developed independently. > > The "editions", as I think someone referred to them, are a combination of > the components that are useful together, but are just that. They are > lightweight, just specifying how to pull those components together and > providing supporting resources like examples, configurations and extra > documentation. Think of these like Linux distributions, they pull together > a host of programs together to create a, sometimes unique, set up. The > distribution is just a collation of all the programs along with the > installer, some specific graphics and preconfigured settings. The programs > themselves are developed, and also available, separately. > > Instead of keeping it all monolithic and using profiles to control what is > released (and what isn't) I think we should break it up into:- > > A "Core" > Individual "Components" (viewer, object stores etc) > Specially targetted "Editions" (SOA, Web Framework, Modeller's toolbox > etc) > > This would obviously affect both the organisation of the code and be > reflected in the site as a series of subprojects (like a number of Apache > project do, like James, Maven, Velocity and Xerces). > > (Doesn't Eclipse actually something just like this - they have a core, > there are are series of components that work with it and they provide > different "editions" that target different types of user.) > > Another benefit of working this way is that a (new) developer can > concentrate on a single component, i.e., check out a small amount of code, > use Maven to sort of the dependencies to other framework classes, which > will download the jars and then focus on just that one module. > > Just a few thoughts. > > Regards > > Rob > > > > > On 11/29/12 14:36, Dan Haywood wrote: > >> This discussion post is mostly to the contributors, but cc:ing to user >> list >> for opinions also... >> >> ~~~ >> >> At the moment it's a daunting task to fully verify all the components of >> the framework are working correctly prior to pushing out a release. I >> suppose that wouldn't be so much of a concern if we had better automated >> tests, but, hey, that's how things are. >> >> Moreover, frankly, some of the modules are best considered spikes and >> probably don't constitute something that we would consider >> production-ready. Or, at least, they haven't been used in a real-world >> context, so it's not possible to say whether they are really >> fit-for-purpose. I include here quite a lot of the stuff that I have >> worked >> on over the last few years, eg the wrapper-progmodel, the groovy >> progmodel, >> probably also the JUnit viewer and BDD viewer. Now that we are a top-level >> project, I'd like it that the code we put out is thought of as production >> ready. >> >> In order to make the framework easier to release, I propose that we use >> Maven profiles to be able to release subsets of modules, that a given >> contributor can stand behind and say: "yes, those modules are working and >> are fit for general consumption". Each release would consist of the >> certain >> core modules, along with selected additional viewers and object stores. >> >> For example, over time we might end up releasing: >> >> * v0.3.1 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects >> // a "wicket/jdo release" - eg tested and released by Dan >> >> * v0.4.0 core + objectstore-nosql + viewer-scimpi // a "scimpi/nosql >> release" - eg tested and released by Rob >> >> * v0.5.0 core + objectstore-dflt + objectstore-xml + viewer-dnd // a >> "dev env release" - eg tested and released by Rob >> >> * v0.6.0 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects >> // another "wicket/jdo release" >> >> * v0.7.0 core + objectstore-sql + viewer-html // a "html/sql release" >> - >> eg tested and released by Kevin >> >> * v0.8.0 core + objectstore-jdo + viewer-wicket + viewer-restfulobjects >> // another "wicket/jdo release" >> >> Whenever a release goes out, we would update our website to indicate the >> most recent version of the components. >> >> Now, when I say "release", what I actually mean here is the deployment of >> binary artifacts up to the Maven central repo. This, after all, is what >> most users/would-be users in the community would use and consider a >> release. However, Apache's own formal definition of "release" is actually >> the source code release ... this is what the [VOTE] mechanism is for. I >> don't see this changing... the vote basically says that the software >> complies with all the legal license stuff etc. The whole codebase is >> tagged >> with that release number, and the generated src.zip is a zip of this >> release. >> >> One benefit of this is that it allows the deployment of other modules to >> be >> performed "after the fact". For example, suppose that I (Dan) puts out the >> v0.3.1 release as above, and then Rob later on tests the scimpi viewer in >> that tag and finds it works well. He can (I think) push the release of the >> scimpi viewer up to Maven central repo. >> >> As I see things there are several benefits to this scheme: >> >> a) (as already noted) the user community will only see binary releases >> that >> are known to be of production ready. This should mitigate the "is it safe >> to use this component" worry >> b) (as hinted at) it should be possible for individual contributors to put >> out releases of their modules more rapidly >> c) we get visibility of which modules aren't being released; for example, >> we might say that if a module hasn't been released by anyone for 18 or 24 >> months, say, then it's probably time to remove it from the codebase. >> >> ~~~ >> >> In order to support the above scheme, I propose that we rearrange some of >> the modules so that they can be more easily included within Maven >> modules. I've >> created a page on our wiki [1] which shows my proposed rearrangement. It >> also repeats the above discussion text by way of context. >> >> >> Opinions welcome! >> >> Thx >> >> Dan >> >> >> [1] >> https://cwiki.apache.org/**confluence/display/ISIS/Make+** >> releases+easier+and+more+**frequent<https://cwiki.apache.org/confluence/display/ISIS/Make+releases+easier+and+more+frequent> >> >> >
