Re: [Zope3-dev] zc.buildout and develop-eggs
Hey, Am Mittwoch, den 02.05.2007, 09:36 -0400 schrieb Jim Fulton: On May 2, 2007, at 9:31 AM, Michael Howitz wrote: Am 02.05.2007 um 15:12 schrieb Jim Fulton: Assuming that you have a checkout of the trunk zc.form, then you are mistaken. The version number in the setup.py file: http://svn.zope.org/zc.form/trunk/setup.py?view=auto is 0.1dev. This is before 0.1dev-r74753 and thus doesn't satisfy the requirement zc.form=0.1dev-r74753. If you aren't using the trunk of zc.form, you should share what version you are using. I'm using the trunk of zc.form. So I thought the trunk should be counted as newer as each snapshot release. You thought wrong. Well. It definitely should be. Reading the setuptools documentation again it becomes obvious how the version numbering scheme has to be used to make that happen: Note: the project version number you specify in setup.py should always be the next version of your software, not the last released version. So, the trunk's version in setup.py should be 0.1 in this case. Making a snapshot release would be done using:: $ python setup.py egg_info -rb.dev sdist This results in the egg being labeled: project-0.1.dev-r1234 Now, a develop egg would be version 0.1 which is represented by the trunk, just not released yet. And any snapshot will have the pre-release tag dev and the post-release tags given by the subversion revision. IMHO we should take care arranging our setup.py files in this way. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Hey, I'm in front of the same wall that Fred was going up. ;) I talked to Zagy about the problems we're facing with the current egg releases of projects that live in Zope3/src/zope/ and Zope3/src/zope/app/ and wrote this up a bit: Problems When working with eggs, we need to carefully use version numbers for the releases and be able to do continuous releases. This is currently a bit icky: - Coding in the tree doesn't force you to get the individual dependencies right, especially when dependencies evolve over time and require constraints based on the version number. - Releasing and tagging doesn't go together very well in this schema, although continuous releases shouldn't require tags anyway. The normal continuous releases from setuptools get somewhat awkward because we have the indirection of the external. I was trying to think about the constraints of handling the large tree when moving the code of all projects into satellite projects. IIRC the requirement is that all projects that we move out now will get a common version number. Proposal Here is what we found would be workable and would like to do later today before doing the beta release. I call this approach synchronized satellites picking up on the terminology Fred came up with. ;) - Move the code of the remaining projects into the satellite projects. - Replace the satellite's external with the actual code on its trunk and point the Zope 3 trunk back to the package in the satellite project's trunk. - Create a directory version in the Zope 3 tree that holds a version.txt file. This will be the common version number that Zope 3 and the satellite projects share. - Create an external in the satellite projects trunk that points to the version directory in Zope 3's trunk. - Set the version.txt file to read 3.4.0b1. This will always read the next version that is going to be released. - Change the setup.py for the satellite projects to read their version number from version/version.txt In this setting we can develop the satellite projects on their own and make releases that match the code. Future releases When releasing Zope 3 the large project as beta 1 in this case, following steps would be involved to keep the projects in sync: - Create a release tag on the satellites (tags/3.4.0b1) - Create a release tag on the Zope 3 trunk (tags/3.4.0b1) - Update the `version` external on the tagged satellites to the tag of the Zope 3 trunk - Update the Zope 3 tag to refer to the release tags on the satellites. - Increase the version number on the Zope 3 trunk to `3.4.0b2` Note: When deciding that we don't target b2 anymore but c1, we can just update the trunk at any point in time. I don't expect any hassles from that change. Most important thing is that the version.txt on the trunk is monotonically increasing. Support scripts --- As we are dealing with more than 90 eggs here, we'll also need script support to keep the handling of the synchronized satellites bearable. I'll expect to need following scripts: - Transform the current source tree into the synchronized satellites approach (one time thing) - Do a release tag of the Zope 3 tree that includes the mechanics of updating the externals as described in the section 'Future releases' (needed in the future) For those scripts I'll need to maintain a list of those synchronized satellite projects that needs to live somewhere. Not all zope.* and zope.app.* projects are satellites, just because they are referenced as externals from the Zope 3 tree (e.g. zope.testing is not) so I'm afraid I'll need to put an explicit list somewhere. Conclusion -- I hope the proposed approach sounds reasonable to you as well and would like to get feedback. I'd like to keep the beta on hold until we solved the packaging issues that I (and others) are facing currently and am willing to put in work today to keep the schedule if we can agree on a schema how to handle this. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: In a small side note: I still don't know how we recommend using the eggs in a way that says I want the egg-version of Zope 3.4.0 and the users gets those eggs (and only those) that are synchronised satellites and have 3.4.0 as their version number. I don't think we've defined what the egg-version of Zope 3.4.0 means (nor do I think we need to as part of a classic zpkg-based Zope 3.4.0). The 'and have 3.4.0 as their version number' I consider a non-goal. Maybe a meta-egg that would introduce the dependency of zope.xx==3.4.0 as their dependency would allow that to happen because if this gets involved in a working set then the synchronized satellites would be restricted to the exact version. Are you suggesting the satellites would have this dependency? If so, -1. If not, and this is just used as a way of collecting the eggs, I think this isn't valuable, and hence is a decoy. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Hey, Am Donnerstag, den 03.05.2007, 08:59 -0400 schrieb Fred Drake: On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: In a small side note: I still don't know how we recommend using the eggs in a way that says I want the egg-version of Zope 3.4.0 and the users gets those eggs (and only those) that are synchronised satellites and have 3.4.0 as their version number. I don't think we've defined what the egg-version of Zope 3.4.0 means (nor do I think we need to as part of a classic zpkg-based Zope 3.4.0). The 'and have 3.4.0 as their version number' I consider a non-goal. Maybe a meta-egg that would introduce the dependency of zope.xx==3.4.0 as their dependency would allow that to happen because if this gets involved in a working set then the synchronized satellites would be restricted to the exact version. Are you suggesting the satellites would have this dependency? If so, -1. If not, and this is just used as a way of collecting the eggs, I think this isn't valuable, and hence is a decoy. I might have stated my goals the wrong way. I find it valuable to be able to predict which exact versions of things get pulled in from a buildout. The current way that dependencies are declared is that when I run buildout in a year I'll get the zope.app.publication-3.5dev-r122 egg on a stable application. I don't want that. I want to again and again and again get the 3.4 egg of zope.app.publication because those eggs where tested together. OTOH. I'm just remembering that Jim talked about some `freeze` feature for buildout ... is that what I seem to want to tackle this issue? ;) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
Am Donnerstag, den 03.05.2007, 08:55 -0400 schrieb Fred Drake: On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: I'm in front of the same wall that Fred was going up. ;) And I ran up against a related issue again on Tuesday, which I've not had the time to write up. - Move the code of the remaining projects into the satellite projects. - Replace the satellite's external with the actual code on its trunk and point the Zope 3 trunk back to the package in the satellite project's trunk. +1 -- The sooner, the better. - Create a directory version in the Zope 3 tree that holds a version.txt file. This will be the common version number that Zope 3 and the satellite projects share. Once the code is in the satellites, there's no reason for them to share version numbers. Ever. They can all start off with 3.4b1, and go from there. The goal is to separate the release cycles. I'm fine with that being complete for 3.4 final, but I certainly don't want the satellite projects to be tied to the Zope 3 trunk at all. - Create an external in the satellite projects trunk that points to the version directory in Zope 3's trunk. -1 -- The dependency relationship should be one-way. - Set the version.txt file to read 3.4.0b1. This will always read the next version that is going to be released. - Change the setup.py for the satellite projects to read their version number from version/version.txt -1 -- Just the version number for the satellite, please! In this setting we can develop the satellite projects on their own and make releases that match the code. Tying their version numbers to Zope 3 doesn't affect this. I'm building up on a comment by Jim who wanted to keep those synchronized. If we don't want that then the effort for keeping them synchronized would go away of course. Many things in this proposal are an effect of that requirement. Future releases When releasing Zope 3 the large project as beta 1 in this case, following steps would be involved to keep the projects in sync: - Create a release tag on the satellites (tags/3.4.0b1) -1 The branched/tagged Zope 3 should refer to specific tags/branches/revisions of the satellites, but the satellite projects should not be affected by Zope 3 releases. I'll expect to need following scripts: ... - Do a release tag of the Zope 3 tree that includes the mechanics of updating the externals as described in the section 'Future releases' (needed in the future) This should not be needed, because the satellite projects should not be affected by subsequent Zope 3 releases. Same as above. ;) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] [URGENT] Organising the satellite projects, eggs and version numbers before beta today
On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: I might have stated my goals the wrong way. I find it valuable to be able to predict which exact versions of things get pulled in from a buildout. Me too; that's very, very, very important to me. The current way that dependencies are declared is that when I run buildout in a year I'll get the zope.app.publication-3.5dev-r122 egg on a stable application. I don't want that. I want to again and again and again get the 3.4 egg of zope.app.publication because those eggs where tested together. OTOH. I'm just remembering that Jim talked about some `freeze` feature for buildout ... is that what I seem to want to tackle this issue? ;) Jim's already implemented a versions feature in zc.buildout, and I find it immensely helpful. You can set buildout:versions to the name of a section that contains version requirements. Each of those requirements is for an exact version. For example: -- [buildout] versions = versions [versions] feedparser = 4.1 -- When I build this, I get feedparser 4.1 exactly; no other version will show up. I generally nail down every version in my buildouts (including buildout recipes), except for software that I'm specifically choosing to track development for (usually application-related), and for deployments, I nail those down as well. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] zope.app.twisted.main and zope multiservice
Hi. I'm really wanting to do more with twisted in zope. One thing that would make this much easier is to have a means of getting hold of the twisted multiservice following startup as opposed to using a different main.py (as I have been) to allow the other services to be added, started or stopped at, or any time following startup. There are use cases for starting and stopping particular services (a short interval) also. An api should be flexible enough to add other clients and servers to the mix. Right now there is no way of getting the multiservice from the current main script. What is needed is a class that could be given this object so that there is a means to access it directly in an app. The multiservice is an important object from the standpoint of being able to start, stop, or add other services to the multiservice (from other packages) even after the main zope startup. A small api for determining what services are part of the multiservice and methods to add or remove services from the running reactor could be added (since these methods are available in twisted). I am hoping by bringing this up that there may be some discussion on how best to accomplish this. I know that Jim has communicated recent interest in a twisted version of ZEO. I share the goal of having a secure transport between clients and servers, however there are many more opportunities for twisted in zope. Being able to access the multiservice is important to this. Many thanks. Regards, David ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] buildbot failure in Zope3 trunk 2.4 Linux zc-buildbot
The Buildbot has detected a failed build of Zope3 trunk 2.4 Linux zc-buildbot. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 937 Blamelist: hdima BUILD FAILED: failed test_2 sincerely, -The Buildbot ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.app.twisted.main and zope multiservice
On 5/3/07, David Pratt [EMAIL PROTECTED] wrote: Hi. I'm really wanting to do more with twisted in zope. One thing that would make this much easier is to have a means of getting hold of the twisted multiservice following startup as opposed to using a different main.py (as I have been) to allow the other services to be added, started or stopped at, or any time following startup. I am not sure what you are looking for, but I have no trouble starting additional services after Zope startup. See my Wing IDE integration for Zope3, for example; http://trac.zopatista.com/zopatista/browser/z3wingdbg/trunk/ It starts additional services, such as a single-threaded HTTP debug server on a separate port, either at Zope start or later as the user requests. -- Martijn Pieters ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.app.twisted.main and zope multiservice
Hi Martijn. Many thanks for your reply. I was not aware of your project so it is quite nice to see. I am hoping to try it out. :-) At zope's startup, a multiservice is started that currently adds the servers setup in the zope.conf. I have created configuration for additional clients and servers to use the existing thread pool and currently adding them to the multiservice using a modified main.py By having access to the multiservice object, you have complete control of each service in the app as well as the ability to incorporate other services to an already running reactor. Interaction within the app using multiple reactors is not safe - so access to this object allows you to add, remove, start or stop services. I also want the configuration to be explicit - so to using zconfig and recipes. Overall, I am integrating apps into buildouts as well. It is useful to be explicit here too since your configuration will otherwise be in a single egg somewhere. This could make it awkward to run multiple servers performing the same task on a machine, which is the pattern I am working with. Regards, David Martijn Pieters wrote: On 5/3/07, David Pratt [EMAIL PROTECTED] wrote: Hi. I'm really wanting to do more with twisted in zope. One thing that would make this much easier is to have a means of getting hold of the twisted multiservice following startup as opposed to using a different main.py (as I have been) to allow the other services to be added, started or stopped at, or any time following startup. I am not sure what you are looking for, but I have no trouble starting additional services after Zope startup. See my Wing IDE integration for Zope3, for example; http://trac.zopatista.com/zopatista/browser/z3wingdbg/trunk/ It starts additional services, such as a single-threaded HTTP debug server on a separate port, either at Zope start or later as the user requests. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: tracking satellite project's trunks
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Christian Theune wrote: Hi, Am Donnerstag, den 03.05.2007, 13:18 -0400 schrieb Tres Seaver: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Fred Drake wrote: On 5/3/07, Christian Theune [EMAIL PROTECTED] wrote: I started moving packages to their own projects (manually right now, preparing for doing this scripted) and noticed that zope.index is tracked with a specific revision number. My understanding is that the trunk of the Zope 3 tree should be tracking the trunk of the satellite project's tree without pinning it to a revision. Probably my fault; I've become really wary of externals that aren't specific. Frankly, I don't *really* care how the Zope 3 tree refers to satellite projects. My hope is that I can very quickly get away from ever looking at the Zope 3 conglomeration (checkout or release), and only use the satellite projects. Conversely, I care very much about the satellite projects not referring to the Zope 3 tree, and look forward to what you're working on today. When a truly egg-based Zope3 ships, it should be a meta-egg with explicitly-versioned dependencies. Approximating that in tree would be to have the Zope3 tree point at *tagged releases* of the satellite projects; revision-stamped versions are a poor (but acceptable in the near-term) substitute. Same here. I'm feeling like I'm not yet understanding what the trunk will be used like. Tracking the trunk of any dependency is not acceptable: it undoes the reason for moving the projects out-of-tree in the first place. If we aren't going to do release management on the pieces, then for sanity's sake keep them in-tree. Plone's own trunk-based bundles are living proof of the hell that is caused by having svn:externals point at non-frozen dependencies. Ah. I don't have any experience with those. What would the trunk of the Zope 3 tree look like then during development cycles? When do updates on the externals in the Zope 3 trunk happen? Typically whenever somebody who knows both decides that the satellite has changes which need pulling into Z3; at the latest, they get updated when doing a Z3 release, I would guess. This is like what happens now with ZODB, etc., or with Zope3 as an external in Zope2. I think we are headed to a place where much less trunk-like development, except for things which have not yet (or maybe ever) been spun off into satellite projects. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGOmDp+gerLs4ltQ4RAtYzAJ9E7W9Ate0Ntr1rSxP28wcuk+gROACgrR+g 2HDuD9SkZkEWYpPvU5sRqmI= =UCYx -END PGP SIGNATURE- ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: tracking satellite project's trunks
Hi, Am Donnerstag, den 03.05.2007, 18:23 -0400 schrieb Tres Seaver: Typically whenever somebody who knows both decides that the satellite has changes which need pulling into Z3; at the latest, they get updated when doing a Z3 release, I would guess. This is like what happens now with ZODB, etc., or with Zope3 as an external in Zope2. Hmm. Right. So those would be less often updated. Typically only after waiting for (a number of) the external packages to have stabilized after new features were added and are gathered together as the classical release. I think we are headed to a place where much less trunk-like development, except for things which have not yet (or maybe ever) been spun off into satellite projects. I'm currently working on getting the whole zope.* and zope.app.* namespace turned into satellite projects. A script is helping me so that should be done pretty soon. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
