Re: What should we do now?
* Anthony Towns aj@azure.humbug.org.au [011023 09:07]: On Tue, Oct 23, 2001 at 01:31:50AM -0400, David M. Cooke wrote: At some point, Anthony Towns aj@azure.humbug.org.au wrote: Again, _why_ does this matter? Who does this? Is it even remotely common? That people would even consider installing another version of python in /usr/local surely just points to a problem with the Debian packaging, no? Well, I do it for one. And is there some reason you can't call them /usr/local/python2.1-opt, say? Or is there some reason you can't just install all the modules you otherwise have installed? Just to make the discussion a little bit more focussed: I think several issues were mixed up in my original mail: (1) For once, #!/usr/bin/env has it's problems. Scripts that use #!/usr/bin/env python are more fragile than scripts without. (2) Another issue is python vs. python1.5. It's a separate question whether scripts need to be strictly coupled to a specific Python version. Regarding (1): If you ask me how common the situation is that people install local Python versions in /usr/local, then I will ask you how common it is that it's reasonable that a script provided by a Debian package will benefit from using #!/usr/bin/env ? Regarding (2): Making the dependency explicit (by using /usr/bin/python1.5) is just playing safe. As far as I can see, if we hadn't the legacy of the existing packages and installations, and if versioned dependencies would work on all systems in all situations, we could come up with a quite clean setup: python-base et al. could be empty packages (virtual packages wouldn't work!), and other packages could depend on them (e.g. Depends: python-base (= 2.0), python-gtk for a package that need at least Python 2.0 or better. Still in this case, what could you do if Python 2.3 broke that script ?) But we haven't an ideal situation, so we have to fight with that kind of legacy. But maybe it brings us to the core of the problem: As far as I can tell our principal problem is that there's a large set of existing packages (from slink, potato) that have wrong dependencies (like unversioned Depends: python-base for packages that depend on Python 1.5.2) that would be (erronously) satisfied if we just continued to use python-base and so on. One way to resolve this is to include a huge list of conflicts in python-base et al. Another way to resolve this is to drop these package names, perhaps in favour of some other package names. IMHO, in the current, messed up situation, scripts and packages that play safe in explicitely stating a Python version have the big advantage that they will lead to these kinds of problems with underspecified dependencies. It's much the same reasoning why libraries have a number in them that will be increased for incompatible API changes. AFAICS, with the setup you are suggesting you can never warranty that kind of robustness. Gregor
Re: What should we do now?
On Tue, Oct 23, 2001 at 01:27:22PM +1000, Anthony Towns wrote: On Mon, Oct 22, 2001 at 08:32:33AM -0700, Neil Schemenauer wrote: Anthony Towns wrote: On Mon, Oct 22, 2001 at 10:13:17AM +0200, Gregor Hoffleit wrote: Say, you would install 2.1.2 in /usr/local. How about we just say Don't install other versions of python in /usr/local ? Please no. Making this work properly is not hard. Again, _why_ does this matter? Who does this? Is it even remotely common? That people would even consider installing another version of python in /usr/local surely just points to a problem with the Debian packaging, no? The most common reason for installing python in /usr/local is to play with the latest and greatest bleeding edge. Typicaly you want to play without everything else installed on you system suddenly using and relying on your experiment. I know that you can rename the executable to something else, but what does a basic make install do? The problems with using #!/usr/bin/python1.5 is threefold: first, it makes dependencies that much more complicated: *all* python scripts have to depend on versioned modules in every way, ie Depends: python1.5-base, python1.5-glade, python1.5-gtk, python1.5-numeric, second it means *all* How else are you going to ensure that a script that relies on version 1.5 of python runs using that version, when you also have python2.1-base installed, and python-base has made the default /usr/bin/python point at /usr/bin/python2.1? If it genuinely needs python1.5 and related modules, there is no other way to specify the dependancies either. I think you are mistaking this as a suggestion to always refer to a particular version of python. This is not what is being proposed... the #!/usr/bin/pythonX.Y is only for scripts that explicitly rely on a particular version of Python. For version independant or default version dependant scripts, #!/usr/bin/python is recomended. The real debate here is #!/usr/bin/python[X.Y] vs #!/usr/bin/env python[X.Y] (note the X.Y is optional) for Debian scripts. From hearing the arguments for and against, I agree with Gregor; #!/usr/bin/python[X.Y] is safer and should be recommended, but packagers might have legitimate reasons for using #!/usr/bin/env python[X.Y]. ie, leave it up to the package mantainers, but make them aware of the pro's and con's. python executables need to be modified at the source level for every python upgrade, and finally it makes Debian veer away from upstream standards for python scripts. This is the strongest argument for /usr/bin/env... If you install new versions of standard tools in /usr/local you have to be careful. This applies to a hypothetical /usr/local/bin/dpkg, or [...] but remember why /usr/bin/env was made the upstream standard; it was because Python is not a standard tool, and hence could be located in /usr/local/bin, ~/bin, or anywhere. The env trick meant scripts could run without modification on any system, regardless of where Python was installed. In a Debian package environment, we have the benefit of a garenteed location for python, and a dependancy system to ensure it's actualy there. -- -- ABO: finger [EMAIL PROTECTED] for more info, including pgp key --
Re: What should we do now?
On Tue, Oct 23, 2001 at 02:42:42PM +0200, Gregor Hoffleit wrote: * Anthony Towns aj@azure.humbug.org.au [011023 09:07]: On Tue, Oct 23, 2001 at 01:31:50AM -0400, David M. Cooke wrote: At some point, Anthony Towns aj@azure.humbug.org.au wrote: [...] Just to make the discussion a little bit more focussed: I think several issues were mixed up in my original mail: [...] Noticed :-) As far as I can see, if we hadn't the legacy of the existing packages and installations, and if versioned dependencies would work on all systems in all situations, we could come up with a quite clean setup: python-base et al. could be empty packages (virtual packages wouldn't work!), and other packages could depend on them (e.g. Depends: python-base (= 2.0), python-gtk for a package that need at least Python 2.0 or better. Still in this case, what could you do if Python 2.3 broke that script ?) This is what we should be (are?) aiming for. As you point out below, the safe way to avoid the python 2.3 broke that script problem is to always use Depends: pthon-base (=2.0), python-base (2.3), and take a punt on it breaking when 2.3 comes out. But we haven't an ideal situation, so we have to fight with that kind of legacy. But maybe it brings us to the core of the problem: As far as I can tell our principal problem is that there's a large set of existing packages (from slink, potato) that have wrong dependencies (like unversioned Depends: python-base for packages that depend on Python 1.5.2) that would be (erronously) satisfied if we just continued to use python-base and so on. One way to resolve this is to include a huge list of conflicts in python-base et al. Matthias has done this with his packages. I think this is the way to go. Sure, we might need to maintain the conflicts for as long as we support back to potato perhaps... but are we required to support mixed potato/woody systems anyway? A dist-upgrade will surely upgrade the old stuff away, provided there are new packages in woody, won't it? Another way to resolve this is to drop these package names, perhaps in favour of some other package names. I feel this is ugly... are we going to keep inventing new names every time we hit this kind of problem? IMHO, in the current, messed up situation, scripts and packages that play safe in explicitely stating a Python version have the big advantage that they will lead to these kinds of problems with underspecified ^ not? dependencies. [...] -- -- ABO: finger [EMAIL PROTECTED] for more info, including pgp key --
Re: What should we do now?
Neil Schemenauer wrote: ___ [wf:] good morning This is my first mail to this mailing list. I use woody and potato. I use python for my coding. I am waiting to python-pyqt under woody :-) The problems with using #!/usr/bin/python1.5 is threefold: first, it makes dependencies that much more complicated: *all* python scripts have to depend on versioned modules in every way This has nothing to do with using /usr/bin/python(X.Y)? vs. /usr/bin/env python(X.Y)?. If you want to use the latest packaged version of Python you should use /usr/bin/python __ [wf:] This problem looks like the source.list with apt-get. if you whant to use the stable version actualy you avec 2 choices. stable or potato if you choose potato you are sure not to upgrade to woody when it(he) will become stable. if you choose potato you are sur not to upgrade. some peaple prefere to upgrade other not to. but theire are one thing sur when woody will become stable the simbolic lync in the ftp will change. so why not to do the same thing with python. python package is the last one (2.2 for exemple) with name python and python2_2 . If i need python1.5: apt-get install python1_5 now last python is python 3.0 (only for exemple the number is false ) if you have choose the /usr/bin/python you will use python30 if /usr/bin/python(X.Y) you will use pythonx_y. now if we change the actual policy to have the last python in python pacquage this may change script using python1.5 witch do not work unde thon2.2 why not changing the header #!/usr/bin/python to #!/usr/bin/python1_5 with a script before upgrading to the new policy? I hope you can understand me because of my poor english. William If you install new versions of standard tools in /usr/local you have to be careful. This applies to a hypothetical /usr/local/bin/dpkg, or a /usr/local/bin/sed, or whatever. Going out of your way to make sure it doesn't apply to a /usr/local/bin/python seems to this observer a complete waste of time. The effort involved is small. Use /usr/bin/python instead of /usr/bin/env python. Use /usr/bin/pythonX.Y instead of /usr/bin/env pythonX.Y. Neil -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- \\\|/// \\ - - // ( @ @ ) +---oOOo-(_)-oOOo+ |Famy William 36 avenue des barattes 74000 Annecy France| |email:[EMAIL PROTECTED] [EMAIL PROTECTED]| +Oooo+ oooO ( ) ( )) / \ ((_/ \_)
Re: What should we do now?
* Anthony Towns aj@azure.humbug.org.au [011023 17:22]: On Tue, Oct 23, 2001 at 02:42:42PM +0200, Gregor Hoffleit wrote: Just to make the discussion a little bit more focussed: I think several issues were mixed up in my original mail: Sounds like a plan. (1) For once, #!/usr/bin/env has it's problems. Scripts that use #!/usr/bin/env python are more fragile than scripts without. (2) Another issue is python vs. python1.5. It's a separate question whether scripts need to be strictly coupled to a specific Python version. Regarding (1): If you ask me how common the situation is that people install local Python versions in /usr/local, then I will ask you how common it is that it's reasonable that a script provided by a Debian package will benefit from using #!/usr/bin/env ? Well, how about you answer the question instead? It's reasonable for Debian scripts to use #!/usr/bin/env because any harm that could be caused is both rare and easily avoidable, and because that's what many upstream scripts use. AFAICS. Certainly that's not a fair statistic, but if I grep through my /usr/bin, there are 12 scripts using some kind of #!/usr/bin/env python, but 79 scripts using something like #!/usr/bin/python (in /usr/local/bin, the ratio is 2 for /usr/bin/env and 4 for /usr/bin/python). That's consistent with my observations: The common idiom in the Python community is #!/usr/local/bin/python, /usr/bin/env is only rarely used. That means most Python scripts have to be modified anyway. It's just the question whether we should replace #!/usr/local/bin/python with #!/usr/bin/env python or with #!/usr/bin/python. (I don't really care if you want to encourage people to not use /usr/bin/env, though) Regarding (2): Making the dependency explicit (by using /usr/bin/python1.5) is just playing safe. It's not merely playing safe though. It's also creating a strict dependency that will bite you later (by making transitions to python X.Y++ require significantly more changes, by having packages break needlessly when those changes aren't made). As far as I can see, if we hadn't the legacy of the existing packages and installations, and if versioned dependencies would work on all systems in all situations, we could come up with a quite clean setup: python-base et al. could be empty packages (virtual packages wouldn't work!), and other packages could depend on them (e.g. Depends: python-base (= 2.0), python-gtk for a package that need at least Python 2.0 or better. Still in this case, what could you do if Python 2.3 broke that script ?) There're two possible solutions, obviously: either you assume in advance that scripts will work with new versions of python and allow them to be installed together, or you assume they don't and forbid them from being installed together. I think that's a correct statement: I can't see any other possibilities being even logically consistent. If you assume that scripts will work with new versions of python, you end up with the result that occassionally your assumption will be wrong for some scripts, and you'll have some possible partial upgrades that'll break. This isn't a cardinal sin. OTOH, if you assume that scripts *won't* work, you're committing yourselves and a bunch of other people to going through whole bunches of Python code checking that it works, and twiddling bits to note that fact. The only question is whether the (presumable) handful of bugs every year or two due to the former is worth the (presumable) whole lot of effort involved in the latter. The easiest way to handle the latter, IMO, is to add a: Depends: python-base ( X.Y+1) to the package, where X.Y is the most recent version of python that it works with. Note that this allows you to use a package on multiple versions of python (so if woody were to release with python 1.5 and 2.0, your package could conceivably Depend: python-base (= 2.0), python-base ( 2.3), and you won't have to make further changes when Debian packages for 2.1 or 2.2 come out), unlike #!pythonX.Y, which is much more restrictive. Just to make sure we have the same thing in mind: At this point, python-base is no virtual package provided by some real pythonX.Y-base package, but it's a real package, right ? This would work indeed, provided a single, canonical Python version was an option (see below). As far as I can tell our principal problem is that there's a large set of existing packages (from slink, potato) that have wrong dependencies (like unversioned Depends: python-base for packages that depend on Python 1.5.2) that would be (erronously) satisfied if we just continued to use python-base and so on. Well there's two questions; one is working out what we want, and the other is working out how to get there. One way to resolve this is to include a huge list of conflicts in python-base et al. Another way to
Re: What should we do now?
On Wed, Oct 24, 2001 at 01:42:12AM +1000, Anthony Towns wrote: On Tue, Oct 23, 2001 at 10:59:45PM +1000, Donovan Baarda wrote: [...] Uh, how many scripts rely on python 1.5? If Debian's main python is 2.1, why should a python 1.5 script remain available? I can't see any reason for this, and I can't see any reason why it would even happen. Perhaps you should read back through the archives at this point... this has all been discussed already... and dare I say we already have a solution... The obvious answer, anyway, is that you simply don't get to install python 2.1 as /usr/bin/python while you have that script on your system, due to its dependency on python-base 2.1. [...] There are many _big_ applications which are tied to particular versions of Python and upgrade at different rates to Python. Zope, mailman, etc. These big apps would either end up including their own Python, holding back the Debian Python, or would just not exist as debs under a single Python scheme. One of the main objectives of the slowly gelling policy has been to support multiple versions of Python installed at once, while still allowing applications that are not tied to a particular version to use more a general Depends: python-base (=1.5), python-base (2.2) and #!/usr/bin/python (or #!/usr/bin/env python) without tying themselves up. To step back a bit, the idea is that for any Debian release we should have a single main version of python. For woody, that'll probably be, well, let's assume 2.0, and that for the next release it'll probably be 2.1 or 2.2. And so on. All packages that use python will be expected to use whichever version that is. Most people (including myself) shouldn't have to give a damn beyond this. [...] Yep, that's the plan. We implement it with multiple pythonX.Y-base packages, and then the python-base (X.Y-Z) package makes one of these the default by providing a symlink /usr/bin/python to /usr/bin/pythonX.Y. What I'm trying to say is that the important case to consider, the one to make as easy and efficient, and effective and reliable as possible is the first one, not the second. Admins who need an old version of python can cope with changing the top line in the few scripts where it matters from /usr/bin/env python to /usr/bin/pythonX.Y. Python hackers trying to keep up with the van Rossums ought to be able to manage to use the altinstall target without breaking a sweat. There is more to this than just #!/usr/bin/pythonX.Y. Where do packages install their python modules? Do they use /usr/lib/python2.1 because 2.1 is the default? What happens when 2.2 becomes the default? Read the archives for a solution :-) -- -- ABO: finger [EMAIL PROTECTED] for more info, including pgp key --
Re: What should we do now?
Anthony Towns writes: On Tue, Oct 23, 2001 at 06:13:31PM +0200, Gregor Hoffleit wrote: Regarding (1): If you ask me how common the situation is that people install local Python versions in /usr/local, then I will ask you how common it is that it's reasonable that a script provided by a Debian package will benefit from using #!/usr/bin/env ? Well, how about you answer the question instead? Certainly that's not a fair statistic, Uh, I meant the question I asked. :) But anyway... Just to make sure we have the same thing in mind: At this point, python-base is no virtual package provided by some real pythonX.Y-base package, but it's a real package, right ? Right. This could be done as either: python-base 1.5.1 Depends: python1.5-base 1.5.1 (a dummy package just containing some docs and the /usr/bin/python link) or python-base 1.5.1 Provides: python1.5-base depending on how you wanted to maintain it. (The former would make it easy to keep 1.5 packages around when you move to 2.x, the latter would have less dummy packages). I would prefer the former. It provides a basic python for packages that cannot be upgraded. Sorry, but maintaining a single canonical version of Python in the distribution won't work. All this means is that on all woody systems, /usr/bin/python is the same version of python, and that any packages that don't use that version need a damn good reason for it. I agree on this interpretation of canonical version. That's not my opinion, that's what the Python developers crew at python-dev told me. Until recently, Mailman didn't work with Python 1.5.2. Zope 2.2 didn't work with Python 1.5.2. Zope 2.4 doesn't work with Python 2.1. How did mailman and zope manage to end up like this? From the python.org pages, the only backwards incompatibilities were a couple of things about changes in arguments, that seem trivially fixable? I really don't care why they ended there. It's legitimate for a package maintainer not to diverge from the upstream version and stay with an old version. In any event, I *think* we're in violent agreement here: there ought to be old versions in the archive that you can declare a dependency on, get other modules for, and set as your interpretor; but there also ought to be a single /usr/bin/python in any release that's used by every normal package. we agree. (and there could be newer versions, which cannot be made the normal version soon enough). So if you go apt-get install python you end up with python 2.0 (say), say 2.1 and we agree. and if you follow that up by apt-get install zope, you still have python 2.0 for everything normal, but you also get python 1.5 (and any necessary python 1.5 modules) in some out of the way place for zope's use. Which would be something like: [well explaining example removed] That's how I thought Matthias' last proposal would work anyway (well, with python-base not python anyway). So you did understand me correctly, perhaps you could reword the policy proposal, where I am unclear (hint, hint, ... ;-) And even the python-base - python substitution sounds ok, now that we know that 100 conflicts on a line are a bad idea. Matthias Btw, where can I find python-apt ;-)
Re: What should we do now?
Anthony Towns wrote: On Mon, Oct 22, 2001 at 10:13:17AM +0200, Gregor Hoffleit wrote: Say, you would install 2.1.2 in /usr/local. How about we just say Don't install other versions of python in /usr/local ? Please no. Making this work properly is not hard. Besides, if the magic is #!/usr/bin/env python then we would have to say don't install other versions of python in your path. Neil
Re: What should we do now?
On Mon, Oct 22, 2001 at 08:32:33AM -0700, Neil Schemenauer wrote: Anthony Towns wrote: On Mon, Oct 22, 2001 at 10:13:17AM +0200, Gregor Hoffleit wrote: Say, you would install 2.1.2 in /usr/local. How about we just say Don't install other versions of python in /usr/local ? Please no. Making this work properly is not hard. Again, _why_ does this matter? Who does this? Is it even remotely common? That people would even consider installing another version of python in /usr/local surely just points to a problem with the Debian packaging, no? The problems with using #!/usr/bin/python1.5 is threefold: first, it makes dependencies that much more complicated: *all* python scripts have to depend on versioned modules in every way, ie Depends: python1.5-base, python1.5-glade, python1.5-gtk, python1.5-numeric, second it means *all* python executables need to be modified at the source level for every python upgrade, and finally it makes Debian veer away from upstream standards for python scripts. And all this for what, precisely? If you install new versions of standard tools in /usr/local you have to be careful. This applies to a hypothetical /usr/local/bin/dpkg, or a /usr/local/bin/sed, or whatever. Going out of your way to make sure it doesn't apply to a /usr/local/bin/python seems to this observer a complete waste of time. Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. Security here. Yes, maam. Yes. Groucho glasses. Yes, we're on it. C'mon, guys. Somebody gave an aardvark a nose-cut: somebody who can't deal with deconstructionist humor. Code Blue. -- Mike Hoye, see http://azure.humbug.org.au/~aj/armadillos.txt
