Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hey, On Thu, Feb 28, 2008 at 7:41 AM, Christian Theune [EMAIL PROTECTED] wrote: [snip] but I don't see it flying given the sentiments against that idea so far. Perhaps I'm wrong. Humm. Maybe there's just a misunderstanding. I didn't get that you wanted to only trump version pinning, not constraints in general. So there might be the chance that others had that understanding too (might be only my mistake anyway). Possibly. I think Jim was clear on the idea, though. :) Perhaps you can write a new proposal that summarizes the points and making a proposal? I don't think I'm in the position to push this topic. :) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Christian Theune wrote: [snip] Here's an idea: Let `develop` trump version pinning, but not any other constraints. As far as I can see this would allow both of our scenarios to work or continue to work. I'd be happy with that too, and was really what I was aiming at, and I think it doesn't conflict with anything else, but I don't see it flying given the sentiments against that idea so far. Perhaps I'm wrong. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: [snip] Here's an idea: Let `develop` trump version pinning, but not any other constraints. As far as I can see this would allow both of our scenarios to work or continue to work. I'd be happy with that too, and was really what I was aiming at, and I think it doesn't conflict with anything else, Right, that's why I proposed it. but I don't see it flying given the sentiments against that idea so far. Perhaps I'm wrong. Humm. Maybe there's just a misunderstanding. I didn't get that you wanted to only trump version pinning, not constraints in general. So there might be the chance that others had that understanding too (might be only my mistake anyway). Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Tue, Feb 26, 2008 at 3:38 PM, Christophe Combelles [EMAIL PROTECTED] wrote:
But probably the feature has been created before the name 'develop' was
chosen,
and it should have an other name ('egg_path'? 'local_egg'?).
source_egg?
--
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Christian Theune wrote: Martijn Faassen schrieb: [snip] I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. I don't understand what you're getting at. A development package will *never* be supported by a package listing in a particular buildout, as it should always have a 'dev' marker in its version number in setup.py. If it's listed in any list of fixed versions, it's always going to be wrong. And yes, when you release a package you've been developing you do need to update the versions listing of the buildout that depends on it. It's clear enough from what's listed in 'develop' what you need to be updating. Did really nobody run into this surprise before? I am really surprised at that. I mean, you were working with a buildout with a fixed version list, added a package to 'develop' and then (after some head-scratching) noticed your changes weren't being picked up at all, and then you *weren't* surprised about it and wondering what to do about it? Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: Martijn Faassen schrieb: [snip] I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. I don't understand what you're getting at. A development package will *never* be supported by a package listing in a particular buildout, as it should always have a 'dev' marker in its version number in setup.py. If it's listed in any list of fixed versions, it's always going to be wrong. And yes, when you release a package you've been developing you do need to update the versions listing of the buildout that depends on it. It's clear enough from what's listed in 'develop' what you need to be updating. Did really nobody run into this surprise before? I am really surprised at that. I mean, you were working with a buildout with a fixed version list, added a package to 'develop' and then (after some head-scratching) noticed your changes weren't being picked up at all, and then you *weren't* surprised about it and wondering what to do about it? Nope. I'm not always working against a fixed version list. E.g. when I developt z3c.zalchemy then this is a library package, not an application, so I don't fix the versions but let anything that satisfies the the requirements in setup.py come in. In this situation I might use a checkout of SQLAlchemy that doesn't match the version requirement so it doesn't get picked. (I.e. there are two branches of zalchemy, one for SA 0.3 and one for 0.4) I don't want to start developing on the 0.3 branch using SA 0.4 accidentally. buildout can be and is used in a much broader way than just applications that provide fixed versions. Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
David Pratt wrote: Hi Martijn. I respect the points you make, but disagree with your comments. Wichert's reply accurately articulates what we are asking buildout to do. I share this view. It's not very useful to talk about a we asking buildout to do things when there is clearly a debate and no we. On a personal note, I tend to rely on my own version lists but refer to the online lists (for support in creating them). On explicit vs implicit, it is debatable any time you consider incorporating implicit behaviour. We're not considering incorporating implicit behavior. We're debating two competing explicit behaviors. When you make the point about versions duplication, you may not be considering the utility of buildout. In fact, a buildout does not require a setup.py at all. setup.py is only a requirement for packaging in python. Buildout is already being used together with other packaging solutions and in other ways as I have previously mentioned. I don't understand your point. Explain to me how you can use the 'develop' line *without* it pointing to a package with a setup.py? Explain to me how 'versions' can be applied to version numbers of things that are *not* eggs? There is a genuine debate possible over what 'develop' actually does. If you ignore version pinning, you can have two descriptions: * if you want to develop a package, put it under 'develop'. * listing a package under 'develop' adds it to the package search path for packages *and* this package will trump any other versions available elsewhere on the path. These descriptions are equivalent, though one is task-oriented and the other one talks about the details of what's going on. I had internalized the former model, and was surprised when it was broken when version pinning was included. Evidently the rule for 'develop' is now: * listing a package under 'develop' adds it to the package search path for packages *and* this package will trump any other versions available elsewhere on the path *unless* there is an explicit version pinning indicator that says you want to use that version of the package. The task oriented description to develop a package becomes: * if you want to develop a package, put it under 'develop', and just to be sure it isn't pinned in some version list somewhere, add a 'versions' entry to your buildout if it doesn't exist already, and then list package_name = in it to make sure it is being picked up. (alternatively you can make the user check the versions lists for the presence of the package and only if it there have the user list it. Or yet another alternative is to run the buildout in verbose mode and look at what versions pass by) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. When applying DRY we still need to beware that we don't lock out combinations that are currently possible/helpful. Why would I want to list a package under 'develop' in a buildout and not have it be picked up? A combination only needs to be possible if it is helpful, so could you please give me an example a use case where this doesn't happen that is helpful? I'm really struggling to come up with a use case where you want to add a development package to the search path and then have it never being picked up. I experimented with recipes that change other recipes' configuration at run-time, but had a bad experience because of the part-ordering that prevents this, otherwise I'd say you could use a recipe for simpler declaration of develop eggs. That would make you type more in each buildout, though. I just want to add the behavior to buildout. Then I can tell people, if you want to develop a package, use really_develop. I started trying to add a test to buildout that demonstrates the current behavior (not really_develop). Strangely enough, it fails! I asked on the distutils list what I'm doing wrong. The branch is 'faassen-develop'. (there are some unrelated test failures too, but they exist on the trunk too for some reason) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Previously Martijn Faassen wrote: Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. That highly depends on how you work. I normally do not pin to exact revisions but to ranges, which work just fun with dev eggs. If you don't want to pin something - remove the pin. Don't expect something to magically make your pin go away. Wichert. -- Wichert Akkerman [EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
[Originally sent to Martin only; meant to send to the list.] On Mon, Feb 25, 2008 at 5:18 PM, Martijn Faassen [EMAIL PROTECTED] wrote: Meanwhile, you're satisfied already with actually looking at setup.py of the develop package and then repeating the version number *too*. It's fascinating how people look differently at these things, isn't it? Heh. We have buildouts where multiple versions of a package are used by different parts of what's built. There's a real value in separating the meanings of develop and version requirements. We also typically work with the following settings, just to make sure we don't leave any information out of our build configuration: [buildout] allow-picked-versions = false use-dependency-links = false Having to repeat on ocaission isn't wonderful, but it's actually pretty unusual, and far better than leaving something out that can't be recovered later. -Fred -- Fred L. Drake, Jr.fdrake at gmail.com Chaos is the score upon which reality is written. --Henry Miller ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Wichert Akkerman wrote: Previously Martijn Faassen wrote: Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. That highly depends on how you work. I normally do not pin to exact revisions but to ranges, which work just fun with dev eggs. I'm confused. How do you work with ranges in a versions section? I'm talking about this construction: [buildout] develop = my_package versions = some_release [some_release] my_package = 3 I didn't realize you could work with ranges there? If you don't want to pin something - remove the pin. Don't expect something to magically make your pin go away. I do want to pin everything but the packages I'm presently developing. Do you want your 'develop' packages to be picked up if there's a released version of the package available somewhere, and your ranges allow both to be picked up equally, by the way? Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Previously Martijn Faassen wrote: Wichert Akkerman wrote: Previously Martijn Faassen wrote: Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. That highly depends on how you work. I normally do not pin to exact revisions but to ranges, which work just fun with dev eggs. I'm confused. How do you work with ranges in a versions section? I'm talking about this construction: [buildout] develop = my_package versions = some_release [some_release] my_package = 3 I didn't realize you could work with ranges there? I don't, but I can put ranges in setup.py. If you don't want to pin something - remove the pin. Don't expect something to magically make your pin go away. I do want to pin everything but the packages I'm presently developing. Do you want your 'develop' packages to be picked up if there's a released version of the package available somewhere, and your ranges allow both to be picked up equally, by the way? Only if it has a newer version number or I somehow enforce that. Wichert. -- Wichert Akkerman [EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Martijn Faassen a écrit :
(...)
I think the term 'develop' is badly chosen. You are right if you argue while
having the meaning of 'develop' in mind. You are explaining what you think a
'develop' option should be. A 'develop' option means: I want to 'develop' on
this package, so I want it active, otherwise I don't put 'develop'.
But probably the feature has been created before the name 'develop' was chosen,
and it should have an other name ('egg_path'? 'local_egg'?).
However something has to be done because some people find it not logical, some
other people will probably have headaches with it.
The two easiest choices are 1) issue a clear warning in stderr, or 2) rename
'develop' to something else.
Christophe
Regards,
Martijn
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev Aaron Lehmann___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Christian Theune wrote: [snip] Nope. I'm not always working against a fixed version list. E.g. when I developt z3c.zalchemy then this is a library package, not an application, so I don't fix the versions but let anything that satisfies the the requirements in setup.py come in. This thread is called 'buildout versions and develop conflict, and I was talking about that topic. I can see how this is related to other topics from the perspective of consistency arguments, but perhaps you should be more clear. :) In this situation I might use a checkout of SQLAlchemy that doesn't match the version requirement so it doesn't get picked. (I.e. there are two branches of zalchemy, one for SA 0.3 and one for 0.4) And you're listing both of them in 'develop'? Fred pointed out to me that you might want to rely on the same package but different versions multiple times in a buildout. I can see how this can be useful, but I don't see how one can use the 'versions' construction of the [buildout] section and do this. I don't want to start developing on the 0.3 branch using SA 0.4 accidentally. I think that this information should be codified in the setup.py requirements of your package. If it relies on 0.3, it should say so with a dependency range. I don't think pinned versions are very useful to stop you from making accidents here. buildout can be and is used in a much broader way than just applications that provide fixed versions. My summary: The consistency argument indicates to me that version pinning should have the same effect as listing version constraints in setup.py. A reasonable argument. Since version constraints are the most specific constraints possible, if there is no constraint conflict they will be the ones picked. There is a use case argument that allows the listing of multiple versions of a development package in 'develop' at the same time, and then have different parts of the buildout pick up on it. Sounds like a valid use case. I wonder though what will be picked up if you *don't* specify the version explicitly later on in your buildout. Unspecified behavior? Having multiple versions of the same package listed in the 'develop' section can never work, I take it, in the face of version pinning. If you have two versions listed in 'develop' and want to use different ones in different sections of your buildout, you can't do this in the face of a global version list, right? I'd say the consistency argument implies buildout should fail with an error if you try to use two different versions of a package while the version is pinned down globally. The task-driven and most-common use case argument (I want to develop!) have made the 'develop' section trump any other listed packages if the version constraints allow both. This is the current behavior. The task I am concerned about entails wanting to hack on a package that is pinned down. This is in my experience very common in the face of shared lists of pinned versions (from the KGS or Grok). In the face of such lists, it's not very pleasant to have to explain to people who want to hack on a package that they should also add the package to the 'versions' section of a buildout to override any entry of them in the list. The goal of this feature is to allow us to tell people the following: * if you want to hack on a package, check it out into your buildout and add it to 'really_develop', and then re-run bin/buildout. and that this will work even in the face of explicit version pinning. My proposal is to add a new feature to buildout, temporarily called 'really_develop', which places a package on the develop list and trumps version pinning. We can debate whether it should also be trumping any version constraints of other packages as well - I don't know whether this is a good idea or not. I think perhaps it should. It *should* still obey the constraints listed in the setup.py of the package under development however. In constraint terms, there would be the ability to override existing constraints by placing a package in 'really_develop'. You could see this as an 'override' version: a package listed in 'really_develop' gets a version 'override' that is guaranteed to fulfill any constraints on version *and* is the most-preferred version. Listing a package in 'develop' doesn't change the version but makes that version the most preferred version if multiple versions of that package would fulfill the set of constraints. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Aaron Lehmann a écrit : On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, You would get a warning *only* if you have specified a 'develop' option and that option has no effect (because of a version). I suppose nobody work in such a situation because it makes no sense. Either you want to specify a version to let your develop-egg be used, or you just remove the develop option. Nobody wants to specify an option while knowing it won't be used. The warning is just to tell you have specified 'develop=' but your develop-egg is not used and the egg version x.x is used instead. Christophe or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev Aaron Lehmann ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Aaron Lehmann wrote: On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev It's longer and shorter: * longer: create a 'versions' section if it isn't there yet, and point to it from [buildout] with 'versions'. (the versions section may not be there yet if this buildout.cfg extends another one with such a section) * shorter: [versions] mynewpin = Removes the pin. In the absence of a pin, it will pick the newest version. Or perhaps I'm wrong about this somehow, as I couldn't get the test case to work when I added it to buildout. :) There are two problems with this solution, both usability related: * People tend to be surprised when they first run into this. The solution is non-obvious in the face of external version lists. It can be expected people will be surprised in the future. * You make people type more, repeating a version name, in a common case (in the face of KGS) So, we're looking at a way to increase the usability of a common task: hack on a package without having to understand its place in a potentially very complicated dependency structure. The simplest solution I see to these problems is to add another option to buildout that does both at the same time (and perhaps override all other constraints), and direct people to that should they wish to develop a package. The name 'develop' was picked which is understandable historically, and it can't be changed. It's consistent with the behavior of easy_install. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hey, Thanks everybody for this discussion. I'm going to bail out now, and I want to share some of my conclusions: * We're going to have to live with the current 'versions/develop' story for a while. I've started try to document the existing behavior in buildout's doctests (faassen-develop) branch. I'll also document it for Grok. * It would be nice if there was some way to override dependency constraints: point at a (development) package and give it a special version number that *always* satisfies all dependency constraints and is *always* the most specific package to do so. * All this makes me think again about the topic of dependency constraint resolution with special override nodes (also in relation to the addition of 'or' constraints to setuptools). * Doing a good dependency constraint resolution needs extensions to the cheeseshop: as I understand it, no dependency information is online right now and the package needs to be downloaded in order to inspect it. Basically it'd be nice if the egg_info directory was published for each release. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Feb 26, 2008, at 10:29 AM, Martijn Faassen wrote: Aaron Lehmann wrote: On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev It's longer and shorter: * longer: create a 'versions' section if it isn't there yet, and point to it from [buildout] with 'versions'. (the versions section may not be there yet if this buildout.cfg extends another one with such a section) If you're extending a buildout with a versions section, you already have one. That's what extending the buildout did for you. * shorter: [versions] mynewpin = This will burn you if your develop egg is not the highest available version (for example, you are working on a bugfix for an en egg that is not bleeding edge). Also, some folks require everything be pinned. So, we're looking at a way to increase the usability of a common task: hack on a package without having to understand its place in a potentially very complicated dependency structure. If you are hacking on an egg, you should know what the current version is, and how your version is different. If you don't, your confusion is deserved. Aaron Lehmann___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: [snip] Nope. I'm not always working against a fixed version list. E.g. when I developt z3c.zalchemy then this is a library package, not an application, so I don't fix the versions but let anything that satisfies the the requirements in setup.py come in. This thread is called 'buildout versions and develop conflict, and I was talking about that topic. I can see how this is related to other topics from the perspective of consistency arguments, but perhaps you should be more clear. :) Sorry, I thought I was. In this situation I might use a checkout of SQLAlchemy that doesn't match the version requirement so it doesn't get picked. (I.e. there are two branches of zalchemy, one for SA 0.3 and one for 0.4) And you're listing both of them in 'develop'? No, I usually only eneter one. Fred pointed out to me that you might want to rely on the same package but different versions multiple times in a buildout. I can see how this can be useful, but I don't see how one can use the 'versions' construction of the [buildout] section and do this. That's a valid use case too, it's not mine though (right now). I don't want to start developing on the 0.3 branch using SA 0.4 accidentally. I think that this information should be codified in the setup.py requirements of your package. If it relies on 0.3, it should say so with a dependency range. I don't think pinned versions are very useful to stop you from making accidents here. That's what I do. That allows me to notice when I include the wrong develop package because it will not be picked up. (In spite of other people that posted to this thread who seem to prefer not to care about this problem during development.) buildout can be and is used in a much broader way than just applications that provide fixed versions. My summary: The consistency argument indicates to me that version pinning should have the same effect as listing version constraints in setup.py. A reasonable argument. Since version constraints are the most specific constraints possible, if there is no constraint conflict they will be the ones picked. Right, that's my understanding as well. There is a use case argument that allows the listing of multiple versions of a development package in 'develop' at the same time, and then have different parts of the buildout pick up on it. Sounds like a valid use case. I wonder though what will be picked up if you *don't* specify the version explicitly later on in your buildout. Unspecified behavior? Good question. Having multiple versions of the same package listed in the 'develop' section can never work, I take it, in the face of version pinning. If you have two versions listed in 'develop' and want to use different ones in different sections of your buildout, you can't do this in the face of a global version list, right? I'd say the consistency argument implies buildout should fail with an error if you try to use two different versions of a package while the version is pinned down globally. I think you're right. I haven't given thought whether making buildout fail is a good idea, it might be though. The task-driven and most-common use case argument (I want to develop!) have made the 'develop' section trump any other listed packages if the version constraints allow both. This is the current behavior. The task I am concerned about entails wanting to hack on a package that is pinned down. This is in my experience very common in the face of shared lists of pinned versions (from the KGS or Grok). In the face of such lists, it's not very pleasant to have to explain to people who want to hack on a package that they should also add the package to the 'versions' section of a buildout to override any entry of them in the list. The goal of this feature is to allow us to tell people the following: * if you want to hack on a package, check it out into your buildout and add it to 'really_develop', and then re-run bin/buildout. and that this will work even in the face of explicit version pinning. I really hate the term `really_develop` and I don't like having two develop statements. Here's an idea: Let `develop` trump version pinning, but not any other constraints. As far as I can see this would allow both of our scenarios to work or continue to work. [snip] Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
David Pratt wrote: Hi. I agree with Jim. Buildout is doing the right thing. This is not a conflict since you have explicitly identified the software with a version already. I think the right thing to do under the circumstances would be to append a custom versions.cfg to nail the versions you want. KGS versions is a point in time list and it does not apply to the full scope of what buildout is being used for. I believe this should be kept in mind since it serves more than z3. Changes to buildout to have it automatically do the 'right' thing opens the implicit versus explicit argument. A developer would then need to be aware of the implicit cases that would cause a different software selection. Much like zcml configuration in zope, I want to tell buildout what to do and have it do it without surprise (or for that matter fighting any implicit nature folks may be inclined to give it). While I understand the concern about the development egg for your build, I would see any move in this direction as corrupting the nature of buildout to 'do what you have told it to do' I want to tell buildout what to do have it do it without surprise as well. I was surprised when it didn't do what I expected: give priority to the develop package. Why else would I choose to put it on the develop line? I take it you have run into this and weren't surprised at all, then? I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. I realize that it has a reason: it does what you tell it do. But lists of locked versions are things that are frequently maintained offline - even sitting off on some URL, and maintained by someone else. Yes, indirectly you are telling buildout about versions, but you may not be very aware of it. These are long lists, after all. It'd be nice if these lists could be treated as mostly opaque (encapsulation) and that you can simply look at what's in setup.py instead. That is not possible now. You need to *know* that it lists the package you are trying to hack on, and you need to know that you need to add it to [versions]. The workaround I find myself using frequently now is this: [versions] the_package = I don't see the point when I already say this in 'develop'. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: buildout 'versions' and 'develop' conflict
Christian Theune wrote: Stephan Richter schrieb: On Saturday 23 February 2008, Jim Fulton wrote: The additional version specification should be merged into the extends version section. The version 1.3.1dev is the version the develop egg specifies. Yes. That's how it works now. Cool, then I think the behavior is correct. I think it is merely a matter of communicating that things work this way. I agree. Shame on you, Christian, are you a Grok developer? :) It's a clear DRY violation, the name of the package (and even the version number) repeats here. This means we should not be satisfied with the response of communication if there's a way to fix this. There are two ways to fix this: * change 'develop'. It's clear this one is out. People like its current behavior. I'm surprised not more people were surprised than me, but I guess I'm just as smart as most people. :) * add a new option, 'really_develop', that adds the thing to [versions] and to the develop path at the same time. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: David Pratt wrote: Hi. I agree with Jim. Buildout is doing the right thing. This is not a conflict since you have explicitly identified the software with a version already. I think the right thing to do under the circumstances would be to append a custom versions.cfg to nail the versions you want. KGS versions is a point in time list and it does not apply to the full scope of what buildout is being used for. I believe this should be kept in mind since it serves more than z3. Changes to buildout to have it automatically do the 'right' thing opens the implicit versus explicit argument. A developer would then need to be aware of the implicit cases that would cause a different software selection. Much like zcml configuration in zope, I want to tell buildout what to do and have it do it without surprise (or for that matter fighting any implicit nature folks may be inclined to give it). While I understand the concern about the development egg for your build, I would see any move in this direction as corrupting the nature of buildout to 'do what you have told it to do' I want to tell buildout what to do have it do it without surprise as well. I was surprised when it didn't do what I expected: give priority to the develop package. Why else would I choose to put it on the develop line? I take it you have run into this and weren't surprised at all, then? I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. I realize that it has a reason: it does what you tell it do. But lists of locked versions are things that are frequently maintained offline - even sitting off on some URL, and maintained by someone else. Yes, indirectly you are telling buildout about versions, but you may not be very aware of it. These are long lists, after all. It'd be nice if these lists could be treated as mostly opaque (encapsulation) and that you can simply look at what's in setup.py instead. That is not possible now. You need to *know* that it lists the package you are trying to hack on, and you need to know that you need to add it to [versions]. The workaround I find myself using frequently now is this: [versions] the_package = I don't see the point when I already say this in 'develop'. See above. However, I agree that buildout should make it more obvious what's happening. Maybe, as a supportive action, buildout could tell that or why a development package was not picked (whenever an egg with a similar name is required) and give a warning (like when variables are unused) Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: Stephan Richter schrieb: On Saturday 23 February 2008, Jim Fulton wrote: The additional version specification should be merged into the extends version section. The version 1.3.1dev is the version the develop egg specifies. Yes. That's how it works now. Cool, then I think the behavior is correct. I think it is merely a matter of communicating that things work this way. I agree. Shame on you, Christian, are you a Grok developer? :) Too little in the last time, I suppose. ;) It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). When applying DRY we still need to beware that we don't lock out combinations that are currently possible/helpful. This means we should not be satisfied with the response of communication if there's a way to fix this. There are two ways to fix this: * change 'develop'. It's clear this one is out. People like its current behavior. I'm surprised not more people were surprised than me, but I guess I'm just as smart as most people. :) Again, I was surprised by this, but thinking about it and understanding what's going on I think I like it the way it is. I agree that we need to make it easier to understand and let people be less surprised. * add a new option, 'really_develop', that adds the thing to [versions] and to the develop path at the same time. I really (pun semi-intended) don't like that name, but the idea seems attractive. I experimented with recipes that change other recipes' configuration at run-time, but had a bad experience because of the part-ordering that prevents this, otherwise I'd say you could use a recipe for simpler declaration of develop eggs. That would make you type more in each buildout, though. Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi Martijn. I respect the points you make, but disagree with your comments. Wichert's reply accurately articulates what we are asking buildout to do. I share this view. On a personal note, I tend to rely on my own version lists but refer to the online lists (for support in creating them). On explicit vs implicit, it is debatable any time you consider incorporating implicit behaviour. When you make the point about versions duplication, you may not be considering the utility of buildout. In fact, a buildout does not require a setup.py at all. setup.py is only a requirement for packaging in python. Buildout is already being used together with other packaging solutions and in other ways as I have previously mentioned. Overall, buildout cfg files are an abstraction. Its most attractive features are that it is simple, readable, fairly well documented and without a great deal of obfuscation or magic. You may consider a recipe and utility script that uses versions to help build a setup.py. It would seem more in line with the character of the software. Regards, David Martijn Faassen wrote: David Pratt wrote: Hi. I agree with Jim. Buildout is doing the right thing. This is not a conflict since you have explicitly identified the software with a version already. I think the right thing to do under the circumstances would be to append a custom versions.cfg to nail the versions you want. KGS versions is a point in time list and it does not apply to the full scope of what buildout is being used for. I believe this should be kept in mind since it serves more than z3. Changes to buildout to have it automatically do the 'right' thing opens the implicit versus explicit argument. A developer would then need to be aware of the implicit cases that would cause a different software selection. Much like zcml configuration in zope, I want to tell buildout what to do and have it do it without surprise (or for that matter fighting any implicit nature folks may be inclined to give it). While I understand the concern about the development egg for your build, I would see any move in this direction as corrupting the nature of buildout to 'do what you have told it to do' I want to tell buildout what to do have it do it without surprise as well. I was surprised when it didn't do what I expected: give priority to the develop package. Why else would I choose to put it on the develop line? I take it you have run into this and weren't surprised at all, then? I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. I realize that it has a reason: it does what you tell it do. But lists of locked versions are things that are frequently maintained offline - even sitting off on some URL, and maintained by someone else. Yes, indirectly you are telling buildout about versions, but you may not be very aware of it. These are long lists, after all. It'd be nice if these lists could be treated as mostly opaque (encapsulation) and that you can simply look at what's in setup.py instead. That is not possible now. You need to *know* that it lists the package you are trying to hack on, and you need to know that you need to add it to [versions]. The workaround I find myself using frequently now is this: [versions] the_package = I don't see the point when I already say this in 'develop'. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
AW: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi Christian Betreff: Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict [...] I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. This is what really confuses me. Probably I don't understand what all others are thinking about develop externals. I can't see a use case for develop externals which do not need to add an explicit version at the same time. Is it really possible to use develop externals without to adjust versions? If so, then you are probably all right. But if not then it's just a double definition which makes no sense at all. btw, what happens to packages which I add as develop externals which are not listed in an index. I guess they behave very different, right? For me it's still very simple like for Martijn. If you develop it's an explicit task and later if you are ready, you have to think about release and deployment. Then comes the right time to think about versions and compatibility to other versions. Regards Roger Ineichen ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
