Jim Fulton wrote:
I discussed this a bit this afternoon with Stephan and we came up with an idea that we think might help. Stephan is going to try to prototype it. I'll try to explain it.
snip
Hopefully, this will give us much greater stability than we've had up to now.
For what it is worth, what follows is the report I send yesterday to the Grok mailinglist that is the result of trying to "pin down" a known working set to rely on for Grok.
I personally like the simplicity of the described solutions and the fact the we really did not need to *invent* anything new, but were able to use the tools (buildout specifically) that are there already.
I hope this is of any use, and feedback from this list is appreciated. kind regards, jw -------- Original Message -------- Subject: Pinning Down Versions Date: Fri, 05 Oct 2007 19:14:16 +0200 From: Jan-Wijbrand Kolman <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Newsgroups: gmane.comp.web.zope.grok.devel =============== Pinned versions =============== During the Neanderthal Grok Sprint, Luciano and I have been working on "pinning down versions" to protect Grok development and deployment from broken, incompatible and conflicting egg releases of all the dependencies. The first thing we did is to come up with and test (on Linux, OS X and Windows) a list of known working egg versions. You can see the current list here: http://svn.zope.org/*checkout*/grok/trunk/versions.cfg With this list of versions, we updated Grok's `buildout.cfg` to: 1) use the `versions.cfg` by way of the `extends` directive. 2) point the `versions` directive to the versions section that is defined in the `versions.cfg` file. What this now means for the development *of* Grok and the development *with* Grok is described below. **NOTE:** Now that we have a set of versions that actually work, we can evolve this to something with as much final releases in at as possible. For developers *of* Grok ========================= The `buildout.cfg` of Grok now use the `versions` and `extends` directive like so:: [buildout] .... extends = versions.cfg versions = versions # being defined in the versions.cfg file .... Now, when running `./bin/buildout` it will make sure exactly those versions of the egg dependencies are downloaded and installed for this Grok buildout. During Grok's lifecycle newer versions of the egg dependencies might be released. If you want to test such a newer version, you could temporarily modify the `buildout.cfg` and override the `versions` section like so:: [buildout] extends = versions.cfg versions = versions # being defined in the versions.cfg file .... [versions] # whatever specific egg version is defined in this section, # will override the version in the versions.cfg zope.i18nmessageid = 3.5.0 One other way to do this is to create a temmporary config file, say `dev.cfg` and experiment with newer eggs versions like so:: [buildout] extends = buildout.cfg [versions] # whatever specific egg version is defined in this section, # will override the version in the versions.cfg zope.i18nmessageid = 3.5.0 Make sure you use this config file when running the buildout command like so:: $ ./bin/buildout -c dev.cfg If the newer egg version proves to be compatible, the newer version number can be put in the `versions.cfg` and checked in. We think it is very important to make sure the newer egg version *and its dependencies* are to be available on at least Windows too. Whenever a Grok release tag is made, we obviously have a tagged fixed `versions.cfg` file as well. Besides the regular steps needed to make an actual Grok release, extra steps will be required to support grokproject (the reason is explained in the following section): 1) the tagged `versions.cfg` needs to be uploaded to http://grok.zope.org/releaseinfo/grok-[versionnumber].cfg 2) the version of the Grok egg needs to be added to the `grok-[versionnumber].cfg` file, e.g. `grok = 0.11`. 3) the contents of http://grok.zope.org/releaseinfo/current needs to be updated to contain the name of the current 'released' Grok versions info file. E.g. `grok-0.11.cfg`. For developers *with* Grok ========================== We think it is reasonable to assume that users of Grok, in other words application developers that use Grok as a basis, are using grokproject for setting up the initial project structure they'll use. At least they should be strongly encouraged to do so. We should also support upgrading an already existing project that has been created with grokproject, i.e. when a new Grok has been released. Initial project creation ------------------------ For the grokproject user, nothing will change really. He will issue the grokproject command and this will setup an project structure for him. What the new grokproject will do by default is this: 1) fetch the name of the current versions file from http://grok.zope.org/releaseinfo/current 2) add to the `buildout.cfg` an `extends` directive pointing to the URL of the named versions file. E.g. http://grok.zope.org/releaseinfo/grok-0.11.cfg 3) add the `versions` directive pointing to the `[versions]` section in the named versions file Now when the buildout command is run, exactly those versions of the eggs will get installed. When running grokproject this behaviour can be overridden. The application developer can choose to specify the URL pointing to custom versions file like so:: $ grokproject \ --version-info-url=http://hostname/some/path/versionsfile.cfg The `extends` directive will then point to that URL. **NOTE:** the implementation for the "new" grokproject lives on the "jw-luciano-use-versions-file" branch until it is merged, probably somewhere next week. Overriding versions of particular eggs -------------------------------------- It is likely that the application developer introduces more dependencies on other software. He should of course add this dependecy in his `setup.py file`. If he wants to make sure that a specific version of the dependency is used when deploying the project, he can add this specification to his `buildout.cfg` like so:: [buildout] extends = http://grok.zope.org/releaseinfo/grok-0.11.cfg versions = versions .... [versions] spam = 1.2.3 Likewise, if the application developers wants to override the version of one of the eggs grok depends on he can adjust his `buildout.cfg` to look like this:: [buildout] extends = http://grok.zope.org/releaseinfo/grok-0.11.cfg versions = versions .... [versions] zope.i18nmessageid = 3.5.0 # will override the original spec By the way, users of grokproject should be encouraged to let the Grok developers know about wanting to use a newer version of a current Grok dependency. With this information, the Grok developers can consider this newer version to be specified - "blessed" - for the upcomming release of Grok. Upgrading to a new Grok release ------------------------------- Whenever an application developer is ready to upgrade his project to a new version of Grok, he only needs to change the URL that points to the versions file to the newest versions file:: [buildout] extends = http://grok.zope.org/releaseinfo/grok-0.99.cfg versions = versions .... Then, he needs to run his buildout command again to make it install the eggs that are now specified for this latest release of Grok. "Fixing up" current grokproject-made projects --------------------------------------------- Projects that have been created with the current grokproject command can be "fixed up" by: 1) removing the `newest = false` line from the project's `buildout.cfg`. 2) adding the following two lines to the `[buildout]` section:: [buildout] .... extends = http://grok.zope.org/releaseinfo/grok-0.10.1.cfg versions = versions and then running the buildout command again. **NOTE**: this "fix up method" will work when Grok 0.10.1 has been released. If you want to try anyway, you could try using this URL: http://grok.zope.org/releaseinfo/grok-0.10.1b1.cfg Next steps: * Martijn and I have been working on a versions file with as much final releases (instead of alphas and betas) in it as possible. This needs to be finalized and tested on Windows. We can probably do this on monday. * Release 0.10.1 and people using Grok-0.10 should then be able to use 0.10.1's versions list to "fix up" their projects. * Review the jw-luciano-use-versions-file branch, and release grokproject. * Tweak the versions file for the grok trunk and release Grok-0.11. And of course, we're curious for you feedback! kind regards, jw ps. I'd like to thank Luciano for pairing-up during the sprint! It has been a pleasure to work together on this! _______________________________________________ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
