AW: [Zope-dev] KGS Site Updated
Hi Dieter > Betreff: Re: [Zope-dev] KGS Site Updated > > Chris McDonough wrote at 2007-11-18 16:50 -0500: > > ... > >Note that if the KGS really wants to be a KGS (literally > "known good", > >it's a matter of semantics, not of technology): > > > >- An invariant must be met that only one version of each > package should > >be present in the index. > > > >- The set is frozen (no packages will be added to or removed from or > >changed). > > I do not think this is correct: > > The assumption that something is "known good" can prove to > be an error. > > As soon as this happens, the index is no longer "known good". > > As a consequence, it can (and should) be changed. I agree, But this also means, changes must be 100% compatible with everything which was working with the KGS before the changes. (whatever this means) Otherwise it's not "known good" anymore. Regards Roger Ineichen > -- > Dieter > ___ > Zope-Dev maillist - Zope-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) > ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Monday 19 November 2007, Chris McDonough wrote: > Now we just need to define what a "release" > is ;-) Yes, I thought about this a little bit this weekend and I would love to see some discussion. For example, I do not think it will be necessary to create a new release for every change in the KGS, but rather collect them. So for example, once "3.4.0" is out, I set the version in controlled-packages.cfg to "3.4.1dev". I leave it like that until we are ready for "3.4.1" and change it then. Maybe just calling it "3.4dev" would be better and whenever we are ready for a release, we create a controlled-packages.cfg file with that final version number, such as "3.4.1" or "3.4.2". All of this, of course, does not answer: What is a release? I simply do not know, but I have a hunch that it will be defined by a timespan. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Nov 19, 2007, at 1:26 PM, Dieter Maurer wrote: Chris McDonough wrote at 2007-11-18 16:50 -0500: ... Note that if the KGS really wants to be a KGS (literally "known good", it's a matter of semantics, not of technology): - An invariant must be met that only one version of each package should be present in the index. - The set is frozen (no packages will be added to or removed from or changed). I do not think this is correct: The assumption that something is "known good" can prove to be an error. As soon as this happens, the index is no longer "known good". As a consequence, it can (and should) be changed. I agree this is a really super-useful kind of index and maybe even deserves the term "KGS". If it makes more sense than trying to redefine "KGS", I'm arguing that there's an additional use case of a completely frozen index to get a completely repeatable set of packages, every time, "forever". For example: - You create an automated developer "sandbox" build using the non- minimal KGS to "get zope". You want this build to work repeatably, every time, without fail. - While developing, you find a bug in it that isn't yet fixed upstream. - You modify your automated build process to patch the installed KGS files using diffs to fix the bug locally. The diffs change files installed from the KGS. Those patches may begin to fail for new runs of the automated build if someone uploads a bugfix release to a package you were patching. This really isn't a problem, because Stephan has provided the capability and has signaled the intent to maintain frozen "minimal" sets for each "release". Now we just need to define what a "release" is ;-) - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
Chris McDonough wrote at 2007-11-18 16:50 -0500: > ... >Note that if the KGS really wants to be a KGS (literally "known good", >it's a matter of semantics, not of technology): > >- An invariant must be met that only one version of each package >should be present in the index. > >- The set is frozen (no packages will be added to or removed from or >changed). I do not think this is correct: The assumption that something is "known good" can prove to be an error. As soon as this happens, the index is no longer "known good". As a consequence, it can (and should) be changed. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Nov 19, 2007, at 2:17 AM, Stephan Richter wrote: On Sunday 18 November 2007, Chris McDonough wrote: I disagree. This is not what this means to me. I think a KGS can receive bug fix releases, which the Zope 3.4 KGS does. However, no new feature releases are allowed. In the Linux world, these purposes are served by separate indexes (e.g. Debian security releases are in their own index). There's no problem with what you're saying, it's just not a source for repeatable installations, which is something that's required for many builds. Yes, I agree. I accommodated those needs by having versioned links.html, versions.cfg, controlled-packages.cfg, and now minimal/ files and directories. Those will *never* change once released. (Of course there will be versions like "3.4.1dev" that change until "3.4.1" is released, simply so that I do not have to version every single little change to the KGS while testing. But maybe that is even desirable at some point. We will see.) Sorry, I don't think I understand the last sentence there but I get the sense from the rest of your message that the minimal index will not change once released, and new versions of it will be made for each "release" (however those are defined in a post-big-release world) and that will be beneficial for lots of people. Right. Note that versions-*.cfg and links-*cfg are frozen. I am probably going to freeze minimal/ as have minimal-*/ too. Will they will change for bugfixes? Nope. That's the reason they are frozen. They are like tags in SVN. OK, that's good to hear. I have tried buildout, of course. Massaging the index to meet some set of expectations developers have of the client they use to install the software is fine, you did a lot of work to do this, which is appreciated, and it's your index. But I suggest that we don't discount the *really KGS* requirement, which is *absolutely repeatable builds* (today, tomorrow, two years from now), and we let people know that the KGS is not that. Why? I have listened to the community very early on, reacting to the need having certain frozen output. A few weeks back I sent a mail to the zope-dev _[1] list outlining how I think the index can be used in buildout. Since then, functionality has only expanded and other combinations are possible now. .. [1] http://mail.zope.org/pipermail/zope-dev/2007-November/030210.html So if you go to http://download.zope.org/zope3.4/intro.html into the sub-section "Version 3.4.0b2" you see a bunch of links. With the exception of the "Index" link, all other links point to versioned file and directories, which will not change. Also, any of the versioned files and directories do not contain references to packages that are not controlled by the KGS. Great. I'm happy to be wrong. A mention on the intro page about how this versioning scheme is meant to work going forward would be nice. Nice job, - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Sunday 18 November 2007, Chris McDonough wrote: > > I disagree. This is not what this means to me. I think a KGS can > > receive bug > > fix releases, which the Zope 3.4 KGS does. However, no new feature > > releases > > are allowed. > > In the Linux world, these purposes are served by separate indexes > (e.g. Debian security releases are in their own index). There's no > problem with what you're saying, it's just not a source for repeatable > installations, which is something that's required for many builds. Yes, I agree. I accommodated those needs by having versioned links.html, versions.cfg, controlled-packages.cfg, and now minimal/ files and directories. Those will *never* change once released. (Of course there will be versions like "3.4.1dev" that change until "3.4.1" is released, simply so that I do not have to version every single little change to the KGS while testing. But maybe that is even desirable at some point. We will see.) > Even if there are bugfixes available, you may not want them, or you > may have fixed the bugs your own way already (though patches or what- > have you). Maybe the KGS isn't it, but there needs to be a way to get > the very same packages, unfailingly, every time, no exceptions for > bugfixes, etc. I wish the KGS was this but it still isn't it. Well, the versioned versions.cfg file will not change. In the buildout world, I think most people will end up using versions-*.cfg when pinning down versions plus specify their own version block in their buildout.cfg for packages that we do not control. > >> The first is true for what we have now neither of the "minimal" set > >> or > >> the set you're calling the "KGS". > > > > Right. Note that versions-*.cfg and links-*cfg are frozen. I am > > probably going > > to freeze minimal/ as have minimal-*/ too. > > Will they will change for bugfixes? Nope. That's the reason they are frozen. They are like tags in SVN. > I have tried buildout, of course. Massaging the index to meet some > set of expectations developers have of the client they use to install > the software is fine, you did a lot of work to do this, which is > appreciated, and it's your index. But I suggest that we don't > discount the *really KGS* requirement, which is *absolutely repeatable > builds* (today, tomorrow, two years from now), and we let people know > that the KGS is not that. Why? I have listened to the community very early on, reacting to the need having certain frozen output. A few weeks back I sent a mail to the zope-dev _[1] list outlining how I think the index can be used in buildout. Since then, functionality has only expanded and other combinations are possible now. .. [1] http://mail.zope.org/pipermail/zope-dev/2007-November/030210.html So if you go to http://download.zope.org/zope3.4/intro.html into the sub-section "Version 3.4.0b2" you see a bunch of links. With the exception of the "Index" link, all other links point to versioned file and directories, which will not change. Also, any of the versioned files and directories do not contain references to packages that are not controlled by the KGS. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Nov 18, 2007, at 7:52 PM, Stephan Richter wrote: On Sunday 18 November 2007, Chris McDonough wrote: 4. A new "minimal/" folder now contains an index just of the controlled packages. This minimal index can be used by compoze as one contributing index. (I have not tested this yet, can anyone try this and report how it can be done?) Very cool, thank you! I'll try this. Great. I would love to get a little write-up on how to use it, so I can put it on the intro page. Compoze is not ready for prime time yet, I'm afraid. You can use it to download all source packages from a working set and create an index from these. But currently I think only Tres knows how to make it do it. ;-) Note that if the KGS really wants to be a KGS (literally "known good", it's a matter of semantics, not of technology): I agree. - An invariant must be met that only one version of each package should be present in the index. I disagree. This is not what this means to me. I think a KGS can receive bug fix releases, which the Zope 3.4 KGS does. However, no new feature releases are allowed. In the Linux world, these purposes are served by separate indexes (e.g. Debian security releases are in their own index). There's no problem with what you're saying, it's just not a source for repeatable installations, which is something that's required for many builds. Even if there are bugfixes available, you may not want them, or you may have fixed the bugs your own way already (though patches or what- have you). Maybe the KGS isn't it, but there needs to be a way to get the very same packages, unfailingly, every time, no exceptions for bugfixes, etc. I wish the KGS was this but it still isn't it. The first is true for what we have now neither of the "minimal" set or the set you're calling the "KGS". Right. Note that versions-*.cfg and links-*cfg are frozen. I am probably going to freeze minimal/ as have minimal-*/ too. Will they will change for bugfixes? The second I'm pretty sure is not true of the thing you're calling the KGS (it still mirrors the cheeseshop regularly), I'm not sure about the intent for the "minimal" set. Well, this is a limitation of the current setuptools system. In buildout I can only specify one index, since setuptools can have only one index. So it is necessary that I make all packages available in that index and this is what I produce as download.zope.org/zope3.4. But I have explained this multiple times before, so I am not going to repeat the argument any more. Please try buildout and see for yourself. I have tried buildout, of course. Massaging the index to meet some set of expectations developers have of the client they use to install the software is fine, you did a lot of work to do this, which is appreciated, and it's your index. But I suggest that we don't discount the *really KGS* requirement, which is *absolutely repeatable builds* (today, tomorrow, two years from now), and we let people know that the KGS is not that. In any case, thanks a lot for the work! - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Sunday 18 November 2007, Stephan Richter wrote: > > The first is true for what we have now neither of the "minimal" set or > > the set you're calling the "KGS". > > Right. Note that versions-*.cfg and links-*cfg are frozen. I am probably > going to freeze minimal/ as have minimal-*/ too. I have now changed the code to version the minimal index as well. It still contains multiple versions of packages that we know work together; however, for your purposes setuptools will only pick up the latest one anyways. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Sunday 18 November 2007, Chris McDonough wrote: > > 4. A new "minimal/" folder now contains an index just of the > > controlled > > packages. This minimal index can be used by compoze as one > > contributing > > index. (I have not tested this yet, can anyone try this and report > > how it can > > be done?) > > Very cool, thank you! I'll try this. Great. I would love to get a little write-up on how to use it, so I can put it on the intro page. > Note that if the KGS really wants to be a KGS (literally "known good", > it's a matter of semantics, not of technology): I agree. > - An invariant must be met that only one version of each package > should be present in the index. I disagree. This is not what this means to me. I think a KGS can receive bug fix releases, which the Zope 3.4 KGS does. However, no new feature releases are allowed. > - The set is frozen (no packages will be added to or removed from or > changed). Yep, I agree. No packages will be added or removed for the Zope 3.4 version of the KGS. I will soon add a new Zope-Dev KGS, which is open to new feature releases. > The first is true for what we have now neither of the "minimal" set or > the set you're calling the "KGS". Right. Note that versions-*.cfg and links-*cfg are frozen. I am probably going to freeze minimal/ as have minimal-*/ too. > The second I'm pretty sure is not > true of the thing you're calling the KGS (it still mirrors the > cheeseshop regularly), I'm not sure about the intent for the "minimal" > set. Well, this is a limitation of the current setuptools system. In buildout I can only specify one index, since setuptools can have only one index. So it is necessary that I make all packages available in that index and this is what I produce as download.zope.org/zope3.4. But I have explained this multiple times before, so I am not going to repeat the argument any more. Please try buildout and see for yourself. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS Site Updated
On Nov 17, 2007, at 10:00 PM, Stephan Richter wrote: Hi everyone, I just updated the Zope 3.4 KGS with some new features. 1. You only need to upload the "controlled-packages.cfg" file now; all other files are generated. 2. The "controlled-packages.cfg" file now supports a name and version option. Thus, all generated files are versioned now. The unversioned version of the files refers to the latest one. 3. A new "links.html" file now lists all packages in one URL, so that it is suitable for the "find-links" option. (I have not tested this yet, anyone?) Nice. 4. A new "minimal/" folder now contains an index just of the controlled packages. This minimal index can be used by compoze as one contributing index. (I have not tested this yet, can anyone try this and report how it can be done?) Very cool, thank you! I'll try this. 5. There is now an introduction page called "intro.html" that lists all versions and files. It also explains how to use the files. See: http://download.zope.org/zope3.4/intro.html 6. The KGS now supports extensions, meaning one KGS can build on another. You simply specify the "extend" option in the "[KGS]" section of the controlled packages configuration file. This should allow frameworks built on top of Zoep 3, like grok, to build their own KGS without much effort. That said, I would really like for people to test all the features and see whether they work for them. But most importantly, I would like for people to contribute to the "Usage" section of the introduction page! Thanks to all for listening! Note that if the KGS really wants to be a KGS (literally "known good", it's a matter of semantics, not of technology): - An invariant must be met that only one version of each package should be present in the index. - The set is frozen (no packages will be added to or removed from or changed). The first is true for what we have now neither of the "minimal" set or the set you're calling the "KGS". The second I'm pretty sure is not true of the thing you're calling the KGS (it still mirrors the cheeseshop regularly), I'm not sure about the intent for the "minimal" set. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] KGS Site Updated
Hi everyone, I just updated the Zope 3.4 KGS with some new features. 1. You only need to upload the "controlled-packages.cfg" file now; all other files are generated. 2. The "controlled-packages.cfg" file now supports a name and version option. Thus, all generated files are versioned now. The unversioned version of the files refers to the latest one. 3. A new "links.html" file now lists all packages in one URL, so that it is suitable for the "find-links" option. (I have not tested this yet, anyone?) 4. A new "minimal/" folder now contains an index just of the controlled packages. This minimal index can be used by compoze as one contributing index. (I have not tested this yet, can anyone try this and report how it can be done?) 5. There is now an introduction page called "intro.html" that lists all versions and files. It also explains how to use the files. See: http://download.zope.org/zope3.4/intro.html 6. The KGS now supports extensions, meaning one KGS can build on another. You simply specify the "extend" option in the "[KGS]" section of the controlled packages configuration file. This should allow frameworks built on top of Zoep 3, like grok, to build their own KGS without much effort. That said, I would really like for people to test all the features and see whether they work for them. But most importantly, I would like for people to contribute to the "Usage" section of the introduction page! Thanks to all for listening! Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
