[Zope-dev] Re: pinning, nailing and kgs'ing

2008-02-05 Thread Martijn Faassen

Hey,

Kevin Teague wrote:
And it would also be really nice 
if the Grok terminology and release methods lined up with the Zope 3 
terminology and release methods :)


Grok got there first, then KGS came along and did a lot of stuff. The 
reason Grok's story is not the same as the one in KGS is that we haven't 
caught up with the new stuff yet. We of course reserve the right to 
simplify it for our purposes. :)  What seems most useful about KGS is 
that it tests versions of eggs together and publishes a list of those 
versions that are known to work together.


The Grok process is fairly simple:

* when we release grok, we also release a versions.cfg on 
grok.zope.org/releaseinfo. We name it after the new release of Grok, and 
don't ever change it anymore.


* we also update a special URL that grokproject makes use of to 
determine the last release of Grok. This way a newly created grokproject 
automatically gets configured to use the right releaseinfo URL


* grokproject then pins those versions in buildout.cfg using the 
'versions = versions' and 'extends = 
http://grok.zope.org/releaseinfo/some-release-versions.cfg' lines in 
[buildout].


We don't have any special index servers or anything. Right now we point 
to PyPI and download.zope.org/distribution; the latter will likely 
eventually go away as all eggs are properly on the cheeseshop.


We do have some issues concerning development, which I think you ran 
into by looking at the viewlet branch. We can probably simplify the 
branch by now as we did release a new version of Martian that has the 
required changes.


We intend to try to go with KGS's list of packages. We need to test that 
and determine the best way to reuse that list. I don't think we'll do 
much else for now - Grok's process works fairly well for us.


One thing that we should eventually improve is to allow grokproject and 
Grok buildouts to work offline. Right now buildout.cfg refers to a URL 
in the 'extends' line which breaks things. Another thing I'd like us to 
experiment with is to offer a one-tarball download of Grok, possibly in 
combination with a frozen repository which contains all Grok 
dependencies for a particular release. On the longer term I am still 
curious about experimenting with getting the these versions of my 
dependencies are tested to work information into the eggs themselves 
somehow.


Regards,

Martijn

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )


[Zope-dev] Re: pinning, nailing and kgs'ing

2008-02-03 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Kevin Teague wrote:
 I wanted to try using the snowsprint-viewlets2 branch of grok in my  
 project the other day. It took me a little time to figure out how to  
 do this, so I thought it'd be nice if there was a bit of documentation  
 on how-to pull in a development version of Grok into a grok project,  
 so I wrote this:
 
 http://grok.zope.org/documentation/how-to/trail-blazing
 
 (I've configured the Grok site to allow OpenId for authentication, and  
 granted the View and Reply to Item permissions to documentation that  
 is in the Waiting for Review state, which means you need to log-in  
 to see the document, but that anyone can log-in, which seems like a  
 decent compromise for unreviewed documentation on the Grok site which  
 is potentially incorrect.)
 
 The part where I was fuzzy in the documentation process was the  
 terminology for Known Good Sets. There are Grok releases such as Grok  
 0.11 which has been called pinned versions (or nailed versions)  
 and when writting my help doc I've been calling this a Known Good  
 Set. I would propose this definition for Known Good Set:
 
Known Good Set : A frozen set of Python egg names and version  
 numbers that have been tested to work together. This set of eggs is  
 also available from an archive maintained by the publisher of the  
 known good set.
 
 For Grok this would be:
 
 http://grok.zope.org/releaseinfo/grok-0.11.cfg
 
 and the eggs are archived at:
 
 http://download.zope.org/distribution/
 
 (at least that's my understanding of the Grok releases, maybe I don't  
 have that quite right ... )

You have one of the tow (disputed) definitions right (and I agree with you).

 However, the Zope 3.4 release announcement varies a bit from my  
 understanding of Grok-centric understanding of Known Good Set:
 
 The known good set -- or in short KGS -- is a package index that  
 derives from the official Python Package Index (PyPI) and thus  
 contains all available packages in the Python world. But for a  
 controlled set of packages, only certain versions that are known to  
 work together are available. 
 
 This is the part that seems a bit confusing to me, since it's not  
 clear if the set also includes the super-set of packages from PyPI.  
 It's also not clear if the super-set of all PyPI packages is also a  
 continually updated mirror, or a frozen index. Or is the known good  
 set just the controlled set of packages?

Agreed:  the'package index he refers to is a mirror of PyPI, with the
exception that packages tracked by the KGS have those versions, rather
than whatever is in PyPI.  The rationale (which I disagree with) is that
people want to use an index, but aren't willing to switch indexes when
installing a package not tracked by the KGS.

I have argued fairly strongly in the past that this is a bad idea, since
there is no possiblity that the indexed packages are known good.

 It's also not entirely clear  
 on how maintenance releases happen with Zope 3.4 from this, e.g. one  
 could interpret the contents of the versions.cfg URL as either  
 frozen (Zope 3.4.0-final) or latest. For maintenance releases  
 would there be:

 http://download.zope.org/zope3.4/versions.cfg

I would argue that this page shouldn't exist yet, because 3.4 final has
not yet been released.  It shod be something like:

 http://download.zope.org/zope3.4.0c1/versions.cfg

 http://download.zope.org/zope3.4.1/versions.cfg
 http://download.zope.org/zope3.4.2/versions.cfg

Those pages don't exist yet, because there are no releases later than 3.4.0.

 Or if this URL will always contain the latest stable packages in the  
 3.4 series then perhaps this should be more explicit with an URL such  
 as:
 
 http://download.zope.org/zope3.4/current-versions.cfg

+1:  if they represent a set which is changing over time, then they
should be named in a way that represents that:  otherwise, people may
expect repeatable results when using the page.

 Shouldn't there also be URLs such as these?
 
 http://download.zope.org/zope3.4/3.4.0c1-versions.cfg
 http://download.zope.org/zope3.4/3.4.0-versions.cfg
 http://download.zope.org/zope3.4/3.4.1-versions.cfg

Something like that. ;)

 It should also be more explicit what controlled means. It seems like  
 this is the same set of packages and versions and versions.cfg except  
 that it also contains versions of previous packages? Is this a  
 continually updated set of the most current versions of all packages  
 that make up the Zope 3.4 series after all test suites pass?
 
 http://download.zope.org/zope3.4/controlled-packages.cfg

I don't nnow what that page means, exaclty.

 Too me it seems easier to understand if there are two URLs in  
 buildout's find-links setting. e.g. for Plone there is a link to a  
 controlled archive of Plone produced packages, then there is a link to  
 PyPI (well the PPIX mirror actually), which makes it more obvious that  
 there is a Zope managed