On 6/4/2013 9:41 AM, Peter Klügl wrote: > On 04.06.2013 15:34, Marshall Schor wrote: >> I'd like to adjust the build / release process for Eclipse update sites >> slightly >> to achieve several goals: >> 1) speedier builds >> 2) more automation, reliability >> 3) insuring minimal use of SVN space in dist.apache.org svn. >> >> The speedier builds would come from avoiding regenerating the pack.gz files >> on >> older versions of plugins. >> The more automation would come from having smaller update-site poms, not >> having >> to list every older version of plugins. >> The insuring minimal svn use would come from automating the checkout - >> update of >> the eclipse update site in a manner to preserve old plugins, unmodified. >> >> The update site building requires running some eclipse tools to generate >> metadata from a set of existing features and plugins. This process requires >> access to all the features and plugins (both the new ones being released, and >> all of the older versions). Today, this is accomplished by using the maven >> dependency plugin to copy the release feature / plugin artifacts from maven >> central to the target working area. >> >> Today, the build process is regenerating 7 more file for each old plugin - a >> "pack.gz" format of the jar, plus 3 checksums/signings for the plain Jar and >> pack.gz form. >> >> In the new process, rather than pull the existing plugin jar from maven >> central, >> I propose using svn checkout from the distribution svn (release) to get all >> the >> old features, plugins, their pack.gz forms, and their signings/checksums. >> The >> update pom site would only need to call out specifically the new release >> being >> added to the set. >> >> The existing build steps would be reduced by no longer generating pack.gz >> forms >> for older versions, or the signings/checksums for those (I haven't yet >> investigated how to do this). >> >> When the build is complete, because the update site was previously >> checked-out >> of the dist svn, to release, you would just commit the changes (which would >> be >> only the new versions of features / plugins and the top level metadata, thus >> saving svn space, and insuring older versions retain their original >> signings. >> Although I haven't used it before, there's a Maven plugin for svn checkout >> etc: >> http://maven.apache.org/scm/maven-scm-plugin/ >> >> What do you think? Any ideas for improvement? >> >> -Marshall > +1, but ... > > You probably know what I gonna say... ;-) > > Why do we not create a new update site for each release? Then, we only > have to handle the new subfolder and update metadata of the composite > repository. For me, it's kind of a matter of trying to be "conventional" :-). The Eclipse community by convention has lots of tools and approaches optimized to support multiple versions of plugins/features. The update site "combining" is something that by convention is targeted at combining a (few) update sites, typically for different things, and is not (typically) used for multiple versions of the same thing.
The feature and bundles publishing tool, for instance (I recently discovered) supports the Eclipse "incremental" model for adding new feature versions. I'm setting up a general script extension for the uima-wide pom that any update site could use, that makes use of this. This allows the metadata generation to run just on the "new" versions being added, and that info is then "appended" to the existing metadata. Another reason for having the more conventional structure is to reduce observed "clutter" by gathering multiple related files under a containing folder that represents an abstraction. You can see UIMA slowly progressing toward that model when you look in our distribution spot. For UIMA 2.3.1 sdk, we put the individual files at the top level. But for later releases, we put the files for one release under individual folders: uima-as-2.4.0, ruta-2.0.1, etc. I suspect, as time marches on, we might find it organizationally a good abstraction to put releases, eventually, under yet another level of hierarchy - e.g. all the uima-as releases under uima-as, all the uimaj releases under uimaj, etc. With the new and improved method of generating eclipse update sites, which makes use of the existing data from the previously released update sites, the pom for the update site now looks very generic, and probably won't need to be updated (unless you are adding *different* (not just updated versions) features / plugins) for new releases, except to change the version of the pom in the standard (conventional :-) ) way releases are managed. This is because it no longer needs to list older versions of features / plugins - those just come from the existing released update site. (Note: I've got it working and hope to commit changes today or tomorrow...) With this approach, normal updates to the Eclipse update site won't require any update to the "composite" site. The updates will be confined just to the project being released. Finally, following conventional approaches makes things easier for other developers - there's less to "explain" :-) -Marshall > > Best, > > Peter > > > > >
