Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hey, On Thu, Feb 28, 2008 at 7:41 AM, Christian Theune [EMAIL PROTECTED] wrote: [snip] but I don't see it flying given the sentiments against that idea so far. Perhaps I'm wrong. Humm. Maybe there's just a misunderstanding. I didn't get that you wanted to only trump version pinning, not constraints in general. So there might be the chance that others had that understanding too (might be only my mistake anyway). Possibly. I think Jim was clear on the idea, though. :) Perhaps you can write a new proposal that summarizes the points and making a proposal? I don't think I'm in the position to push this topic. :) Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: [snip] Here's an idea: Let `develop` trump version pinning, but not any other constraints. As far as I can see this would allow both of our scenarios to work or continue to work. I'd be happy with that too, and was really what I was aiming at, and I think it doesn't conflict with anything else, Right, that's why I proposed it. but I don't see it flying given the sentiments against that idea so far. Perhaps I'm wrong. Humm. Maybe there's just a misunderstanding. I didn't get that you wanted to only trump version pinning, not constraints in general. So there might be the chance that others had that understanding too (might be only my mistake anyway). Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Tue, Feb 26, 2008 at 3:38 PM, Christophe Combelles [EMAIL PROTECTED] wrote:
But probably the feature has been created before the name 'develop' was
chosen,
and it should have an other name ('egg_path'? 'local_egg'?).
source_egg?
--
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: Martijn Faassen schrieb: [snip] I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. I don't understand what you're getting at. A development package will *never* be supported by a package listing in a particular buildout, as it should always have a 'dev' marker in its version number in setup.py. If it's listed in any list of fixed versions, it's always going to be wrong. And yes, when you release a package you've been developing you do need to update the versions listing of the buildout that depends on it. It's clear enough from what's listed in 'develop' what you need to be updating. Did really nobody run into this surprise before? I am really surprised at that. I mean, you were working with a buildout with a fixed version list, added a package to 'develop' and then (after some head-scratching) noticed your changes weren't being picked up at all, and then you *weren't* surprised about it and wondering what to do about it? Nope. I'm not always working against a fixed version list. E.g. when I developt z3c.zalchemy then this is a library package, not an application, so I don't fix the versions but let anything that satisfies the the requirements in setup.py come in. In this situation I might use a checkout of SQLAlchemy that doesn't match the version requirement so it doesn't get picked. (I.e. there are two branches of zalchemy, one for SA 0.3 and one for 0.4) I don't want to start developing on the 0.3 branch using SA 0.4 accidentally. buildout can be and is used in a much broader way than just applications that provide fixed versions. Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Previously Martijn Faassen wrote: Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. That highly depends on how you work. I normally do not pin to exact revisions but to ranges, which work just fun with dev eggs. If you don't want to pin something - remove the pin. Don't expect something to magically make your pin go away. Wichert. -- Wichert Akkerman [EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Previously Martijn Faassen wrote: Wichert Akkerman wrote: Previously Martijn Faassen wrote: Christian Theune wrote: Martijn Faassen schrieb: [snip] It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). The rule for the most common use case is now: If you want to develop a package, add it to 'develop' *and* create a versions section if it doesn't exist, let the 'versions' line in [buildout] point to it, and add the package_name = to it. I see a repetition in the package name here. You list it in two places. I think that's a DRY violation given that wanting a package listed in develop to be picked up is the most common use case. That highly depends on how you work. I normally do not pin to exact revisions but to ranges, which work just fun with dev eggs. I'm confused. How do you work with ranges in a versions section? I'm talking about this construction: [buildout] develop = my_package versions = some_release [some_release] my_package = 3 I didn't realize you could work with ranges there? I don't, but I can put ranges in setup.py. If you don't want to pin something - remove the pin. Don't expect something to magically make your pin go away. I do want to pin everything but the packages I'm presently developing. Do you want your 'develop' packages to be picked up if there's a released version of the package available somewhere, and your ranges allow both to be picked up equally, by the way? Only if it has a newer version number or I somehow enforce that. Wichert. -- Wichert Akkerman [EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Martijn Faassen a écrit :
(...)
I think the term 'develop' is badly chosen. You are right if you argue while
having the meaning of 'develop' in mind. You are explaining what you think a
'develop' option should be. A 'develop' option means: I want to 'develop' on
this package, so I want it active, otherwise I don't put 'develop'.
But probably the feature has been created before the name 'develop' was chosen,
and it should have an other name ('egg_path'? 'local_egg'?).
However something has to be done because some people find it not logical, some
other people will probably have headaches with it.
The two easiest choices are 1) issue a clear warning in stderr, or 2) rename
'develop' to something else.
Christophe
Regards,
Martijn
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev Aaron Lehmann___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Aaron Lehmann a écrit : On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, You would get a warning *only* if you have specified a 'develop' option and that option has no effect (because of a version). I suppose nobody work in such a situation because it makes no sense. Either you want to specify a version to let your develop-egg be used, or you just remove the develop option. Nobody wants to specify an option while knowing it won't be used. The warning is just to tell you have specified 'develop=' but your develop-egg is not used and the egg version x.x is used instead. Christophe or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev Aaron Lehmann ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
On Feb 26, 2008, at 10:29 AM, Martijn Faassen wrote: Aaron Lehmann wrote: On Feb 26, 2008, at 9:38 AM, Christophe Combelles wrote: Martijn Faassen a écrit : (...) The two easiest choices are 1) issue a clear warning in stderr, or 2) rename 'develop' to something else. So, the people that understand either get spammed with warning messages every build, or have to change all their builds in order to accommodate your thinking? We all know what the solution is, it was supplied at the beginning of the thread: [versions] mynewpin = somenewversion_dev It's longer and shorter: * longer: create a 'versions' section if it isn't there yet, and point to it from [buildout] with 'versions'. (the versions section may not be there yet if this buildout.cfg extends another one with such a section) If you're extending a buildout with a versions section, you already have one. That's what extending the buildout did for you. * shorter: [versions] mynewpin = This will burn you if your develop egg is not the highest available version (for example, you are working on a bugfix for an en egg that is not bleeding edge). Also, some folks require everything be pinned. So, we're looking at a way to increase the usability of a common task: hack on a package without having to understand its place in a potentially very complicated dependency structure. If you are hacking on an egg, you should know what the current version is, and how your version is different. If you don't, your confusion is deserved. Aaron Lehmann___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: [snip] Nope. I'm not always working against a fixed version list. E.g. when I developt z3c.zalchemy then this is a library package, not an application, so I don't fix the versions but let anything that satisfies the the requirements in setup.py come in. This thread is called 'buildout versions and develop conflict, and I was talking about that topic. I can see how this is related to other topics from the perspective of consistency arguments, but perhaps you should be more clear. :) Sorry, I thought I was. In this situation I might use a checkout of SQLAlchemy that doesn't match the version requirement so it doesn't get picked. (I.e. there are two branches of zalchemy, one for SA 0.3 and one for 0.4) And you're listing both of them in 'develop'? No, I usually only eneter one. Fred pointed out to me that you might want to rely on the same package but different versions multiple times in a buildout. I can see how this can be useful, but I don't see how one can use the 'versions' construction of the [buildout] section and do this. That's a valid use case too, it's not mine though (right now). I don't want to start developing on the 0.3 branch using SA 0.4 accidentally. I think that this information should be codified in the setup.py requirements of your package. If it relies on 0.3, it should say so with a dependency range. I don't think pinned versions are very useful to stop you from making accidents here. That's what I do. That allows me to notice when I include the wrong develop package because it will not be picked up. (In spite of other people that posted to this thread who seem to prefer not to care about this problem during development.) buildout can be and is used in a much broader way than just applications that provide fixed versions. My summary: The consistency argument indicates to me that version pinning should have the same effect as listing version constraints in setup.py. A reasonable argument. Since version constraints are the most specific constraints possible, if there is no constraint conflict they will be the ones picked. Right, that's my understanding as well. There is a use case argument that allows the listing of multiple versions of a development package in 'develop' at the same time, and then have different parts of the buildout pick up on it. Sounds like a valid use case. I wonder though what will be picked up if you *don't* specify the version explicitly later on in your buildout. Unspecified behavior? Good question. Having multiple versions of the same package listed in the 'develop' section can never work, I take it, in the face of version pinning. If you have two versions listed in 'develop' and want to use different ones in different sections of your buildout, you can't do this in the face of a global version list, right? I'd say the consistency argument implies buildout should fail with an error if you try to use two different versions of a package while the version is pinned down globally. I think you're right. I haven't given thought whether making buildout fail is a good idea, it might be though. The task-driven and most-common use case argument (I want to develop!) have made the 'develop' section trump any other listed packages if the version constraints allow both. This is the current behavior. The task I am concerned about entails wanting to hack on a package that is pinned down. This is in my experience very common in the face of shared lists of pinned versions (from the KGS or Grok). In the face of such lists, it's not very pleasant to have to explain to people who want to hack on a package that they should also add the package to the 'versions' section of a buildout to override any entry of them in the list. The goal of this feature is to allow us to tell people the following: * if you want to hack on a package, check it out into your buildout and add it to 'really_develop', and then re-run bin/buildout. and that this will work even in the face of explicit version pinning. I really hate the term `really_develop` and I don't like having two develop statements. Here's an idea: Let `develop` trump version pinning, but not any other constraints. As far as I can see this would allow both of our scenarios to work or continue to work. [snip] Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: David Pratt wrote: Hi. I agree with Jim. Buildout is doing the right thing. This is not a conflict since you have explicitly identified the software with a version already. I think the right thing to do under the circumstances would be to append a custom versions.cfg to nail the versions you want. KGS versions is a point in time list and it does not apply to the full scope of what buildout is being used for. I believe this should be kept in mind since it serves more than z3. Changes to buildout to have it automatically do the 'right' thing opens the implicit versus explicit argument. A developer would then need to be aware of the implicit cases that would cause a different software selection. Much like zcml configuration in zope, I want to tell buildout what to do and have it do it without surprise (or for that matter fighting any implicit nature folks may be inclined to give it). While I understand the concern about the development egg for your build, I would see any move in this direction as corrupting the nature of buildout to 'do what you have told it to do' I want to tell buildout what to do have it do it without surprise as well. I was surprised when it didn't do what I expected: give priority to the develop package. Why else would I choose to put it on the develop line? I take it you have run into this and weren't surprised at all, then? I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. OTOH it makes you aware about potential version mismatches very early, because you try to develop on a package that doesn't seem to be supported by that particular buildout. So maybe you, for example actually wanted/should increase the version number. I realize that it has a reason: it does what you tell it do. But lists of locked versions are things that are frequently maintained offline - even sitting off on some URL, and maintained by someone else. Yes, indirectly you are telling buildout about versions, but you may not be very aware of it. These are long lists, after all. It'd be nice if these lists could be treated as mostly opaque (encapsulation) and that you can simply look at what's in setup.py instead. That is not possible now. You need to *know* that it lists the package you are trying to hack on, and you need to know that you need to add it to [versions]. The workaround I find myself using frequently now is this: [versions] the_package = I don't see the point when I already say this in 'develop'. See above. However, I agree that buildout should make it more obvious what's happening. Maybe, as a supportive action, buildout could tell that or why a development package was not picked (whenever an egg with a similar name is required) and give a warning (like when variables are unused) Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi, Martijn Faassen schrieb: Christian Theune wrote: Stephan Richter schrieb: On Saturday 23 February 2008, Jim Fulton wrote: The additional version specification should be merged into the extends version section. The version 1.3.1dev is the version the develop egg specifies. Yes. That's how it works now. Cool, then I think the behavior is correct. I think it is merely a matter of communicating that things work this way. I agree. Shame on you, Christian, are you a Grok developer? :) Too little in the last time, I suppose. ;) It's a clear DRY violation, the name of the package (and even the version number) repeats here. It's not clear to me that it's a DRY violation (see my argument that those functions are actually orthogonal). When applying DRY we still need to beware that we don't lock out combinations that are currently possible/helpful. This means we should not be satisfied with the response of communication if there's a way to fix this. There are two ways to fix this: * change 'develop'. It's clear this one is out. People like its current behavior. I'm surprised not more people were surprised than me, but I guess I'm just as smart as most people. :) Again, I was surprised by this, but thinking about it and understanding what's going on I think I like it the way it is. I agree that we need to make it easier to understand and let people be less surprised. * add a new option, 'really_develop', that adds the thing to [versions] and to the develop path at the same time. I really (pun semi-intended) don't like that name, but the idea seems attractive. I experimented with recipes that change other recipes' configuration at run-time, but had a bad experience because of the part-ordering that prevents this, otherwise I'd say you could use a recipe for simpler declaration of develop eggs. That would make you type more in each buildout, though. Christian -- gocept gmbh co. kg - forsterstrasse 29 - 06112 halle (saale) - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildout 'versions' and 'develop' conflict
Hi Martijn. I respect the points you make, but disagree with your comments. Wichert's reply accurately articulates what we are asking buildout to do. I share this view. On a personal note, I tend to rely on my own version lists but refer to the online lists (for support in creating them). On explicit vs implicit, it is debatable any time you consider incorporating implicit behaviour. When you make the point about versions duplication, you may not be considering the utility of buildout. In fact, a buildout does not require a setup.py at all. setup.py is only a requirement for packaging in python. Buildout is already being used together with other packaging solutions and in other ways as I have previously mentioned. Overall, buildout cfg files are an abstraction. Its most attractive features are that it is simple, readable, fairly well documented and without a great deal of obfuscation or magic. You may consider a recipe and utility script that uses versions to help build a setup.py. It would seem more in line with the character of the software. Regards, David Martijn Faassen wrote: David Pratt wrote: Hi. I agree with Jim. Buildout is doing the right thing. This is not a conflict since you have explicitly identified the software with a version already. I think the right thing to do under the circumstances would be to append a custom versions.cfg to nail the versions you want. KGS versions is a point in time list and it does not apply to the full scope of what buildout is being used for. I believe this should be kept in mind since it serves more than z3. Changes to buildout to have it automatically do the 'right' thing opens the implicit versus explicit argument. A developer would then need to be aware of the implicit cases that would cause a different software selection. Much like zcml configuration in zope, I want to tell buildout what to do and have it do it without surprise (or for that matter fighting any implicit nature folks may be inclined to give it). While I understand the concern about the development egg for your build, I would see any move in this direction as corrupting the nature of buildout to 'do what you have told it to do' I want to tell buildout what to do have it do it without surprise as well. I was surprised when it didn't do what I expected: give priority to the develop package. Why else would I choose to put it on the develop line? I take it you have run into this and weren't surprised at all, then? I think the explicit versus implicit discussion has no place here. Placing a package on the 'develop' line is a very explicit action, and you place it on that line because you want to *develop on it*. Having another package being picked up is surprising. I realize that it has a reason: it does what you tell it do. But lists of locked versions are things that are frequently maintained offline - even sitting off on some URL, and maintained by someone else. Yes, indirectly you are telling buildout about versions, but you may not be very aware of it. These are long lists, after all. It'd be nice if these lists could be treated as mostly opaque (encapsulation) and that you can simply look at what's in setup.py instead. That is not possible now. You need to *know* that it lists the package you are trying to hack on, and you need to know that you need to add it to [versions]. The workaround I find myself using frequently now is this: [versions] the_package = I don't see the point when I already say this in 'develop'. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
