Problem
-------

We recently had our first Grok release (0.10). Due to various versions of certain packages, we had to pin down the versions of those packages to ensure you'd get the version we had tested the release with, and to prevent buildout from downloading the versions of packages that had no Windows binaries. The pinning had to be done in setup.py to ensure all Grok installations out there would be on the same page.


Pinning versions in setup.py has major disadvantages, though:

* It's easy to get a version conflict (e.g. you depend on X==1.0 and Y, but Y depends on X==1.0.1)

* It's pretty much impossible to opt for a newer version of a certain package (e.g. in order to fix a security problem).


Fortunately, zc.buildout allows us to keep setup.py free of versions and pin the versions in buildout.cfg instead (see [1] for an extensive example). This works for us experienced developers. This doesn't work for the majority of the consumers of our packages, though. For instance, we can't expect everybody out there to copy and paste 100 lines of stuff into their buildout.cfg when we release a new Grok version. And I'm not even talking about a major or minor release. It would just as well need to happen on every bugfix release.

Grok is just one example where an efficient way of defining, managing and distributing the information on a "known working set" is needed. As we're phasing out the Zope 3 tarball, I'm sure Zope 3 developers who don't keep a close watch on the individual eggs would appreciate if they had a known working set to depend on. The Zope 2 project, for example, is a large consumer of the Zope 3 packages. Other projects such as Plone have the same need for defining known working sets as they're embracing eggs more and more.


Goals
-----

Here's what I'd expect from a known working set:

* It pins packages to certain versions. This information is made available in a way that buildout and/or setuptools can work with it. This has to be easy. Copy and pasting large sections of buildout.cfg is not what I'd consider easy.

* If nothing else is provided, buildout and/or setuptools will install the versions specified in the known working set. It should be possible, however, to deviate from this locally and install different versions of certain packages.

* However the known working set may be defined, it needs to be discoverable and manageable in subversion. It should be possible to give it a version tag.


Proposal
--------

I think zc.buildout's version mechanism in buildout.cfg is a good technical basis. It's simple, light-weight and easy to version-control. For example, the Grok project could create a grok-0.11.cfg defining the known working set for the Grok 0.11 release, with the following contents::

  [grok-0.11]
  grok = 0.11
  ZODB = 3.8.0
  zope.component = 3.4.0
  ...

Then, the Grok folks could distribute this file and everybody who wanted to use Grok 0.11 would simply have to put the following two lines into their buildout.cfg::

  [buildout]
  ...
  extends = grok-0.11.cfg
  versions = grok-0.11

With this, it'll be easily possible to switch to a newer version of e.g. ZODB3 by overriding the setting in buildout.cfg::

  [buildout]
  ...
  extends = grok-0.11.cfg
  versions = grok-0.11

  [grok-0.11]
  ZODB = 3.9.0  # this overrides whatever is set in grok-0.11.cfg


The only problem is that distributing grok-0.11.cfg is a bit tedious. How about if buildout could get it from the web?

  [buildout]
  ...
  extends = http://grok.zope.org/releases/grok-0.11.cfg
  versions = grok-0.11

(alternatively, you could point it to http://svn.zope.org/*checkout*/grok/releases/grok-0.11.cfg to get it directly from SVN.)

So now if you wanted to switch to a newer version of Grok, all you would have to do is point to another known working set in your buildout.cfg (no tedious download and redistribution of a new file involved), e.g.::

  [buildout]
  ...
  extends = http://grok.zope.org/releases/grok-0.12.cfg
  versions = grok-0.12


Deliverables
------------

* zc.buildout's extends mechanism would have to be enhanced to be able to load files from HTTP locations.

* A short and light-weight process would have to be written down to document how we define known good sets, where they're put in subversion and how they're managed.


Thoughts?


[1] http://svn.zope.org/*checkout*/zope.testbrowser/trunk/buildout.cfg

--
http://worldcookery.com -- Professional Zope documentation and training

_______________________________________________
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com

Reply via email to