Re: Status report on python2 transition (possible solution)
D-Man [EMAIL PROTECTED] writes: Yes. Maybe each extension should just depend on a single version of python and need to be rebuilt for each new python release. It makes things considerably simpler, from my point of view. Then, of course, we need unique package names for each package. Something like python-imaging-python1.5, python-imaging-python2.0 and python-imaging-2.1? Aagh. (Picking on python-imaging because it contains binary modules, so it's version specific anyway.) | Are there any other reasons to provide all the modules for Python | 1.5.2 (now more than two years old) in Debian 3.0? Who knows what people might be using that isn't packaged for Debian. True. I feel that we can't keep everyone happy forever, and Python 1.5 has to go away someday; OTOH, I'm running quite up to date unstable, so maybe I don't have the same perspective as many users. ;) -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition (possible solution)
On Sat, Jul 14, 2001 at 04:05:56PM +1200, Carey Evans wrote: | D-Man [EMAIL PROTECTED] writes: | | Yes. Maybe each extension should just depend on a single version of | python and need to be rebuilt for each new python release. | | It makes things considerably simpler, from my point of view. | | Then, of course, we need unique package names for each package. Yeah | Something like python-imaging-python1.5, python-imaging-python2.0 | and python-imaging-2.1? Aagh. Since we have python1.5 and python2.1, how about python1.5-imaging and python2.1-imaging. Then if the maintainer wants to include a version on the package it could be (I don't know PIL's version) python2.1-imaging2.3. I don't think that is too ugly. | (Picking on python-imaging because it contains binary modules, so it's | version specific anyway.) | | | Are there any other reasons to provide all the modules for Python | | 1.5.2 (now more than two years old) in Debian 3.0? | | Who knows what people might be using that isn't packaged for Debian. | | True. I feel that we can't keep everyone happy forever, and Python | 1.5 has to go away someday; OTOH, I'm running quite up to date | unstable, so maybe I don't have the same perspective as many users. ;) I agree with both of those, though I think that at least one old version should be kept. I don't think 1.5.2 needs to disappear quite yet because it is the most common version (in a lot of diff. environments). It could probably go in the next (after woody) release because by then 2.x will have filled it place as the most commonly used. -D
Re: Status report on python2 transition
On Wed, Jul 11, 2001 at 05:30:03PM +0200, Gregor Hoffleit wrote: On Wed, Jul 04, 2001 at 09:00:06PM -0500, Chris Lawrence wrote: My semi-well-thought-out solution: - python-* should provide a Standard Python for each Debian release. 2.1.1 seems a sensible target for woody. These packages should be invoked by /usr/bin/python and /usr/bin/pythonx.y - python-* should (virtual) provide python-x.y-*. - python-v.w-* (where v.w != x.y) can exist at the discretion of the Python maintainer. Provides /usr/bin/pythonv.w I recommend at least having 1.5.2 in woody. 2.0.1 may also be useful. I don't see any good reason for us to ship 1.6.1. After banging my head a few times, and realizing time constraints, this currently looks to me like the most practical solution for woody: Simple rule of thumb: Each package that installs stuff in /usr/lib/pythonX.Y/site-packages has to depend on python-X.Y-base. The main thing is to get something that works within Debian. You don't _have_ to support people putting weird versions of python in their path; you can just say if you want to use a different version of python, build it as a .deb and install it, and resolve the dependencies yourself or put it in /usr/local/pythonX.Y, but don't add it to your path, or whatever. Supporting all sorts of weird things that users might want to do is a nice quality of implementation issue, but it's a long way from release critical. Which is to say, don't introduce a whole lot of RC bugs (this package doesn't depend on that! this other one uses the wrong #! line!) if you don't need to and can't fix them all yourselves in a single block, as well as providing forward and backwards compatibility. A solution that's ugly (that's five lines of Conflicts! Ewww!), but that's timely and works is better than one that reeks of elegance but takes a few months to get worked out. Note also that python needs to be in basically its final form (no new packages etc) and free of RC bugs by the end of next month. (It's in standard now, thanks to reportbug) Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) pgpsgoQg7hhcF.pgp Description: PGP signature
Re: Status report on python2 transition
On Thu, Jul 12, 2001 at 04:55:59PM -0600, Bruce Sass wrote: ... I'm a little confused... A solution to the problem has already been proposed in previous discussions on this list -- and I though it had already (at least) started to be implemented... Sorry, I just recently subscribed to this list, and thought I'd gone over the archive, must have been dreaming:) So, off doing some reading. -- groetjes, carel
Re: Status report on python2 transition (possible solution)
On Fri, Jul 13, 2001 at 03:25:20PM -0400, D-Man wrote: ... o The #! line should look something like #!/usr/bin/deb_py_ver 1.5.2 - I can't get things like this to work. It seems one parameter atmost is allowed there. That's why I proposed this symlink trick the other day. Then the first line would look like: #!/usr/bin/python1.5.2- #!/usr/bin/python1.5.2-2.0 and all would work. Very debian specific and all, but it would allow precise description of the versions the script works with. -- groetjes, carel
Re: Status report on python2 transition (possible solution)
On Fri, 13 Jul 2001, D-Man wrote: On Fri, Jul 13, 2001 at 11:30:28PM +0200, Carel Fellinger wrote: | On Fri, Jul 13, 2001 at 03:25:20PM -0400, D-Man wrote: | ... | o The #! line should look something like | #!/usr/bin/deb_py_ver 1.5.2 - | | I can't get things like this to work. It seems one parameter at | most is allowed there. Huh? The idea was that there would be a script/program named deb_py_ver. It would take 2 command-line arguments and determine which python binary should be used. It would then exec that python on the script (which, IIRC, is on stdin of deb_py_ver). This would allow having a single program, rather than an exponentially growing number of symlinks. Otherwise the symlink idea is the simplest implementation, except for handling which package to put each link in. It has something to do with the #! thing. e.g., SYNOPSIS env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...] yet if you do #!/usr/bin/env command arg you get a message about command arg not being found. - Bruce
Re: Status report on python2 transition (possible solution)
On Fri, Jul 13, 2001 at 06:27:20PM -0400, David M. Cooke wrote: | At some point, D-Man [EMAIL PROTECTED] wrote: | On Fri, Jul 13, 2001 at 11:30:28PM +0200, Carel Fellinger wrote: | | On Fri, Jul 13, 2001 at 03:25:20PM -0400, D-Man wrote: | | ... | | o The #! line should look something like | | #!/usr/bin/deb_py_ver 1.5.2 - | | | | I can't get things like this to work. It seems one parameter at | | most is allowed there. | | Huh? The idea was that there would be a script/program named | deb_py_ver. It would take 2 command-line arguments and determine | which python binary should be used. It would then exec that python on | the script (which, IIRC, is on stdin of deb_py_ver). This would allow | having a single program, rather than an exponentially growing number | of symlinks. Otherwise the symlink idea is the simplest | implementation, except for handling which package to put each link in. | | That's not the problem -- the problem is that the command specified in | the #! is passed the rest of the line as _one_ argument (at least on | Linux). That's why you can't do, for instance, | | #!/usr/bin/env python -O | | as the first line. I thought that would work, which is why I was confused :-). | I suppose you could split the argument on spaces -- although I'm not sure if | other Unices (the Hurd, anyone?) pass the rest of the line or just the first | argument. | | Another alternative would be | | #!/usr/bin/deb_py_ver 1.5.2- | | and | | #!/usr/bin/deb_py_ver 1.5.2-2.1.1 | | That's one argument. Ok, sure, replace the space separator between the versions with a different ASCII separator. -D
Re: Status report on python2 transition (possible solution)
On Sat, Jul 14, 2001 at 11:27:55AM +1200, Carey Evans wrote: | D-Man [EMAIL PROTECTED] writes: | | o all core (the interpreter) python packages will be versioned | ex | python-1.5.2 | python-2.0 | python-2.1.1 | | Each of these will contain /usr/bin/pythonX.Y.Z and provide | 'python'. | | It might be more convenient to have a separate package called | 'python'. This could depend on the latest Python to make apt-get | dist-upgrade work, and conflict with any old packages that are a | problem (like the current python-base). It might be more convenient, unless a user, for some strange reason, wants to only have an older version of python. | It would also provide the /usr/bin/python symlink, if it doesn't get | managed by alternatives. Since the python command will be for users, | should the sysadmin really get to choose an older version for them? Why shouldn't the sysadmin get to choose? Suppose a new python is released, but the admin is afraid that his scripts (suppose they use 'python' too) might not work with it. He might want the default python to stay the same (the now old version) while installing the new one to start migrating to. | (See the gcc, gcc-2.95 and gcc-3.0 packages, where /usr/bin/gcc is a | symlink to /usr/bin/gcc-2.95; except on hppa which uses gcc 3.0.) Are you saying that gcc 2.95 and 3.0 can peacefully co-exist on a system? That would be good news to me :-). | BTW, I would suggest python-2.1 instead; 2.1.1 is only a bug-fix | release, even if the license is one of the bugs, and 2.1.2 will be the | same if it's ever released. Ok, if that is the naming convention that is preferred. | o all other packages will have a versioned depends on the lowest | version it runs with, also a max version if it exists | | I wouldn't like to try to be clairvoyant if I was packaging a Python Sure, but also consider older packages. For example, we are now moving to 2.0 (or 2.1) for the default python. We still want to provide 1.5.2 versions of all the other packages, so they should (now) specify that they don't work with = 2.0 because we know that now. | script; how about just depending on the latest version in the archive | that it works with. One of the problems at the moment is that almost | all packages using Python optimistically depend on = 1.5.2. Either it is optimistic, or there was no known conflict with a newer version (because it didn't exist ;-)) when it was built. | Just have them depend on python-1.5.2 or python-2.1. Ok. So we have a version of each extension package to match with each version of python currently supported. I have no problem with that. | o /usr/bin/deb_py_ver will be a script/program that takes 2 | arguments, a min version and a max version of python that can be | used to run this script. | | Can I suggest that with only a few weeks until Python freezes, | anything like this is left until the following Debian release? That | will give us several months to iron out any bugs in the script, and in | the packages using it, before time runs out. Well, I have no fancy title (like Debian Maintainer) so I really have no authority on the matter. The idea just came to me and it seemed pretty good so I thought I'd share it :-). You can do what you like with the idea. -D
Re: Status report on python2 transition (possible solution)
D-Man [EMAIL PROTECTED] writes: It might be more convenient, unless a user, for some strange reason, wants to only have an older version of python. In which case they won't have anything else installed that depends on python, and they'll just install python-2.1, for example, and never see python-2.2. Given this, /usr/bin/python better be managed by alternatives. [...] Are you saying that gcc 2.95 and 3.0 can peacefully co-exist on a system? That would be good news to me :-). Yep. There's still the familiar problem of differing C++ ABIs, so everything in woody should still be compiled with gcc 2.95. [...] Sure, but also consider older packages. For example, we are now moving to 2.0 (or 2.1) for the default python. We still want to provide 1.5.2 versions of all the other packages, so they should (now) specify that they don't work with = 2.0 because we know that now. However, that can lead to packages breaking when a new version of Python is installed, without pulling in the newer extension modules and packages that use the new scope rules properly, and have variables named yield and div renamed to something else, etc. dpkg and apt provide very good dependency checking, so we should try to use it. Anyway, *do* we actually need all the extension modules for Python 1.5.2? For Debian itself, there's Zope and Mailman, but they don't depend on any other Python packages. reportbug uses python-newt, but it should be changed once python-2.1 is available, so that Python 1.5.2 isn't installed by default on new Debian 3.0 installations. Are there any other reasons to provide all the modules for Python 1.5.2 (now more than two years old) in Debian 3.0? Either it is optimistic, or there was no known conflict with a newer version (because it didn't exist ;-)) when it was built. This may be true, especially when Python 2, which would break lots of stuff, was in the distant future. Python 3K may be far away now, but Python 2.x is breaking stuff with every release. Well, I have no fancy title (like Debian Maintainer) so I really have no authority on the matter. The idea just came to me and it seemed pretty good so I thought I'd share it :-). You can do what you like with the idea. I probably shouldn't be using my @debian.org address for this discussion anyway; I've done one package upload in the last six months, which hardly makes me an active Debian maintainer or any kind of authority. I'm not really talking just to you, but generally; by all means keep coming up with ideas to improve the Debian Python packages, but please don't keep Python 2.1 out of Debian 3.0! -- Carey Evans http://home.clear.net.nz/pages/c.evans/ I'm not the only one who disagrees with everything. - Fur Patrol, _Man In A Box_
Re: Status report on python2 transition (possible solution)
D-Man [EMAIL PROTECTED] writes: Say, ..., is there a way for a python script to find out where the python binary executing it is? If so, then the real script could be run via os.system by the Debian script that uses #! deb_py_ver. sys.executable, which comes from argv[0] in Py_Main(), looking in $PATH if necessary. os.system isn't the best in terms of signal handling and return values; you'd probably want os.execv. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
Bruce Sass [EMAIL PROTECTED] writes: Surely adopting a convention of... #!/usr/bin/env python[major.minor] ...is preferable. It does the same thing without additional runtime overhead and without creating a class of Debian-specific Python programs. I object to this. It make scripts not portable to other systems as it currently does with current python 2.0 scripts in Debian. We shall end up with this trick as soon as possible. -- Jérôme Marant
Re: Status report on python2 transition
More random thoughts... How many versions of Python do we really need to support at one time? Zope and Mailman both need Python 1.5.2 (and depend on python-base = 1.5.2-x, BTW), but don't need any extra packages like mxDateTime. If there are any other packages that don't work with Python 2.1, do they need extra extensions themselves? It seems to me that each stable release of Debian will only need to support a maximum of two different versions of (C)Python, both of which will be quite different. All extra Python modules just have to be compiled against the latest package, and the version of Python from the last Debian release can be removed. For example, woody will have Python 1.5.2 and 2.1.1 when it's released. By the following release of Debian, 2.2 or even 2.3 will be current. Will there be any packages that need 2.1.1 by then? I think I was trying to make a point here: Especially while the freeze is beginning for woody, there's no time to design, implement, test and deploy any scheme for sharing module source, etc., or for doing really fancy dependencies. Instead we should ensure that there is a smooth upgrade path from potato to woody, and that future improvements for woody+0.1 are not ruled out. So... Python 1.5.2, in package python-1.5, installs modules in /usr/lib/python1.5/, installs the binary in /usr/bin/python1.5 and uses alternatives for /usr/bin/python. It conflicts with python-base. Zope and Mailman are recompiled with dependencies on python-1.5 (no versioned depends). Python 2.1.1, in package python-2.1, installs modules in /usr/lib/python2.1/, installs the binary in /usr/bin/python2.1, and uses alternatives, with a higher priority than python-1.5. python-2.1 conflicts with python-base, too. All other packages providing Python modules or using libpython, are recompiled against 2.1.1, with an unversioned dependency on python-2.1. python-base is removed from the archive. Virtual package python turns into a real but empty package, that depends on python-2.1. Scripts written in Python may depend on python, but may expect bug reports later when they turn out to be incompatible with a new version of Python. (Or an old version via alternatives. They should probably just depend on python-2.1 themselves.) Debian packages that have scripts written in Python should use one of /usr/bin/pythonx.y or /usr/bin/env pythonx.y if they depend on python-x.y. They should use /usr/bin/python or /usr/bin/env python if they depend on python. I have no idea whether /usr/bin/env python or /usr/bin/python is better. /usr/bin/env means that another version of Python can be installed in /usr/local, and scripts using env will start using it. Whether this is good or bad for a Debian package is another question. ... Wow. That got a bit longer than I anticipated. If anyone got down to here, do you have any comments? I can expand on my reasoning if it would help. Thanks. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
On Thu, Jul 12, 2001 at 09:03:07AM +0200, Jérôme Marant wrote: Bruce Sass [EMAIL PROTECTED] writes: Surely adopting a convention of... #!/usr/bin/env python[major.minor] ... I object to this. It make scripts not portable to other systems as it currently does with current python 2.0 scripts in Debian. True as this might be, the above scema still doesn't address all needs. The simple case were your script only runs with e.g. 2.1, is covered, but what happens when a newer Python is released? Is your script then still running with 2.1? Or do we need to update all scripts in order for them to use newer Pyton releases? The following scema will address such matters: #!/usr/bin/env python[[major[.minor][-][major[.minor]]] The interpretation is like, the major.minor before the - gives the lowest Python release for which the script works. If no lower bound is specified, any release will do. Similar for the major.minor with respect to the upper bound. It is an error to specify a lower and a upper bound without the -, and if there's no - it's ment to only run with this specific version. For this to work one has to make a series of symlinks for all of the above specified pythonlower-upper actually in use at a specific site, either in /usr/bin or in ~/bin, such that pythonlower-upper links to the highest available python release allowed by the upper limmit. Further more we need symlinks linking pythonmajor to the highest available minor release for that major release. The above seems complicated, but versioning a language is complicated. But take heart, Python manages to be fairly downwards compatible, so the upper bound might not be needed that often, though it is there for power users trying out stuff tied into specific releases. -- groetjes, carel
Re: Status report on python2 transition
On 12 Jul 2001, Jérôme Marant wrote: Bruce Sass [EMAIL PROTECTED] writes: Surely adopting a convention of... #!/usr/bin/env python[major.minor] ...is preferable. It does the same thing without additional runtime overhead and without creating a class of Debian-specific Python programs. I object to this. It make scripts not portable to other systems as it currently does with current python 2.0 scripts in Debian. We shall end up with this trick as soon as possible. How not portable? Any build of Python I have done installs both: bin/pythonmajor.minor and bin/python as hardlinks... ...calling python-wrapper to execute the program is definately not portable to other systems. - Bruce
Re: Status report on python2 transition
On Jul 12, Bruce Sass wrote: On 12 Jul 2001, Jérôme Marant wrote: Bruce Sass [EMAIL PROTECTED] writes: Surely adopting a convention of... #!/usr/bin/env python[major.minor] ...is preferable. It does the same thing without additional runtime overhead and without creating a class of Debian-specific Python programs. I object to this. It make scripts not portable to other systems as it currently does with current python 2.0 scripts in Debian. We shall end up with this trick as soon as possible. How not portable? Any build of Python I have done installs both: bin/pythonmajor.minor and bin/python as hardlinks... ...calling python-wrapper to execute the program is definately not portable to other systems. True... but it would only be done by Python scripts in Debian packages that depend on external modules. The problem with a hardcoded version in the #! path is that it locks the script into running with that version. Let me give you a for instance: - I create a package that uses and depends on python-newt. Python-newt is only supplied for Python 1.5 at the moment, so I hardcode a call to /usr/bin/python1.5 - python-newt's maintainer releases a new python-newt that works with python 2.1, and drops python 1.5 support. - I install the new python-newt (and probably a lot of the python-2.1-*) - My package is now broken and I have to reupload it, even though all I need to change is the #!. With python-wrapper: - python-newt's maintainer does the right black magic with the python-wrapper business. - I create a package that uses and depends on python-newt. I call /usr/bin/python-wrapper. - python-newt's maintainer releases a new python-newt that works with python 2.1, and drops python 1.5 support. - I install the new python-newt (and probably a lot of the python-2.1-*) - My package still works fine, since python-wrapper knows that this version of python-newt needs python 2.1 to run. Now, two questions arise from all this: - Are we willing to have One and Only One Python in each stable release? If so, screw this python-wrapper business. It really doesn't solve any problems with locally-installed Pythons, except by making sure Debian packaged python programs don't use local pythons. But we will need to figure out some way to cleanly deal with not having python 1.5 in woody (a long Conflicts line looks likely). - If we aren't willing to do this, then we either live with a lot of messy dependencies that have to be updated every time a new x.y python comes out, or go with something like python-wrapper that leaves dealing with new Pythons mostly painless. The only other solution is to call python-wrapper /usr/bin/python, and make it fallback to calling a Python interpreter (defined by alternatives or whatever) if no suitable wrapper is found. That is completely portable, except in the case where /usr/bin/python is expected to be an ELF binary (does freeze need this?), but it adds overhead to every invocation. I have nothing invested one way or the other, mind you. Chris -- Chris Lawrence [EMAIL PROTECTED] - http://www.lordsutch.com/chris/ Instructor and Doctoral Student, Political Science, Univ. of Mississippi 208 Deupree Hall - 662-915-5949
Re: Status report on python2 transition
On Thu, 12 Jul 2001, Carel Fellinger wrote:
On Thu, Jul 12, 2001 at 09:03:07AM +0200, Jérôme Marant wrote:
Bruce Sass [EMAIL PROTECTED] writes:
Surely adopting a convention of...
#!/usr/bin/env python[major.minor]
...
I object to this. It make scripts not portable to other systems as
it currently does with current python 2.0 scripts in Debian.
True as this might be, the above scema still doesn't address all needs.
The simple case were your script only runs with e.g. 2.1, is covered,
but what happens when a newer Python is released? Is your script then
still running with 2.1? Or do we need to update all scripts in order
for them to use newer Pyton releases?
Right, it is not ideal, but it is portable.
The following scema will address such matters:
#!/usr/bin/env python[[major[.minor][-][major[.minor]]]
...but is not necessarily portable because Python installs executables
under bin/python{,major.minor}.
I'm a little confused...
A solution to the problem has already been proposed in previous
discussions on this list -- and I though it had already (at least)
started to be implemented...
bms:~# ls -l /usr/bin/python*
-rwxr-xr-x2 root root 3040 Apr 10 02:09 /usr/bin/python
-rwxr-xr-x2 root root 3040 Apr 10 02:09 /usr/bin/python1.5
-rwxr-xr-x1 root root 3080 Jun 23 15:52 /usr/bin/python2
lrwxrwxrwx1 root root7 Jun 24 18:34 /usr/bin/python2.0 -
python2
...'cause I've never seen a 3k python executable. So, using an
explicit python-wrapper in the #! line is a step backwards to me.
Unless Python grows the smarts to figure out on its own that there is
more than one version of itself available, and which version should be
used (not as straightforward as just looking at the code)... the only
reasonable solution is to replace bin/python with something that has
those smarts, the next best is to be explicit about which Python the
program is targeted for.
...
The above seems complicated, but versioning a language is complicated.
But take heart, Python manages to be fairly downwards compatible, so
the upper bound might not be needed that often, though it is there
for power users trying out stuff tied into specific releases.
Yes.
I think it is perfectly reasonable for Debian to insist that a
specific version of Python be installed, and that any Python programs
distributed by Debian must work with a certain version of Python or
better. You can then use an unversioned python executable and not
worry about breakage when the specific minimum version of Python
required by Debian gets bumped up, because Python is backwards
compatible. The people who are gonna run into trouble then are those
that try to point bin/python at pythonmajor.minor, where major +
minor is less than the minimum version Debian expects.
If Python breaks backwards compatibility... well, you probably have
some editing work to do (as will every other Python developer who
wants old code to run without problems) no matter how the
infrastructure has been set up (either rewriting the code to not use
the obsolete paradigm, or simply specifying a version of python it
will work with).
- Bruce
Re: Status report on python2 transition
On Thu, 12 Jul 2001, Chris Lawrence wrote: On Jul 12, Bruce Sass wrote: bin/pythonmajor.minor and bin/python as hardlinks... ...calling python-wrapper to execute the program is definately not portable to other systems. True... but it would only be done by Python scripts in Debian packages that depend on external modules. Ok The problem with a hardcoded version in the #! path is that it locks the script into running with that version. Let me give you a for instance: ... - My package is now broken and I have to reupload it, even though all I need to change is the #!. ya, bummer. With python-wrapper: - python-newt's maintainer does the right black magic with the python-wrapper business. ... - My package still works fine, since python-wrapper knows that this version of python-newt needs python 2.1 to run. ya, nice. Of course, it is up to you with @debian.org addresses to ultimately decide if the black magic route is best. Now, two questions arise from all this: - Are we willing to have One and Only One Python in each stable release? How about, bin/python must meet certain minimum requirements. Is there any difference between having only one Python installed, and only using one of the Pythons that are installed? ...and is it OK if that effect is achieved by convention, as opposed to explicit dependencies? I must admit, I am thinking of stable... is it likely that the (for instance) version of python-newt is gonna change on you after a release... or is the example confined to unstable? If so, screw this python-wrapper business. It really doesn't solve any problems with locally-installed Pythons, except by making sure Debian packaged python programs don't use local pythons. But we will need to figure out some way to cleanly deal with not having python 1.5 in woody (a long Conflicts line looks likely). Hmmm, have 1.5.2, 2.0.1, and 2.1.1 in Woody, adopt the convention bin/python be at least equivalent to python2.0, have any packages still depending on 1.5.2 explicitly state so and use python1.5 in their executables (everyone else can assume #!/usr/bin/env python gets you 2.0 or better. - If we aren't willing to do this, then we either live with a lot of messy dependencies that have to be updated every time a new x.y python comes out, or go with something like python-wrapper that leaves dealing with new Pythons mostly painless. I'm not really sure how messy the dependencies would get (I tend to only pay attention when something is messed up and I need to edit .../dpkg/status as a work around), but I would only expect it to be so (maybe) for those depending on an old version of Python... good incentive to get it working with a more current Python. ;) The only other solution is to call python-wrapper /usr/bin/python, and make it fallback to calling a Python interpreter (defined by alternatives or whatever) if no suitable wrapper is found. That is completely portable, except in the case where /usr/bin/python is expected to be an ELF binary (does freeze need this?), but it adds overhead to every invocation. The python replacement (a.k.a python-wrapper) could be an ELF executable, it may still require a Debianized freeze module, but the overhead would be less. - Bruce
Re: Status report on python2 transition
Bruce Sass [EMAIL PROTECTED] writes: bms:~# ls -l /usr/bin/python* -rwxr-xr-x2 root root 3040 Apr 10 02:09 /usr/bin/python -rwxr-xr-x2 root root 3040 Apr 10 02:09 /usr/bin/python1.5 -rwxr-xr-x1 root root 3080 Jun 23 15:52 /usr/bin/python2 lrwxrwxrwx1 root root7 Jun 24 18:34 /usr/bin/python2.0 - python2 ...'cause I've never seen a 3k python executable. So, using an explicit python-wrapper in the #! line is a step backwards to me. $ ldd /usr/bin/python1.5 libpython1.5.so.0.0 = /usr/lib/libpython1.5.so.0.0 (0x4001f000) ... $ ldd /usr/bin/python2.0 libpython2.0.so.0.0 = /usr/lib/libpython2.0.so.0.0 (0x4001f000) ... These *are* real executables; it looks like all they do is call Py_Main from whichever libpython they're linked against. [...] I think it is perfectly reasonable for Debian to insist that a specific version of Python be installed, and that any Python programs distributed by Debian must work with a certain version of Python or better. Agreed, although it seems OK to me to only expect them to work with the current version, in any stable release of Debian. As the language changes, adding keywords like yield and div, etc., expecting full forwards compatibility might be a little unreasonable. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
Chris Lawrence [EMAIL PROTECTED] writes: - My package is now broken and I have to reupload it, even though all I need to change is the #!. They'll probably be Debian policy changes, new upstream versions of your program, and/or changes in the Python language that require you to do a new upload between releases anyway. IMO, the minimal effort required here isn't worth the black magic. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
On Wed, 11 Jul 2001, Gregor Hoffleit wrote: ... How to do this: - Executables use #!/usr/bin/python-wrapper (*) - Have a file /var/lib/python/wrapper-config (*) with executable: version1, version2, ... This is written by the postinst. - Have a python-wrapper that reads $1, figures out which executable is involved, and invokes the appropriate python. (It can probably be in Python itself... perhaps provided by python-x.y-base and managed by alternatives.) Surely adopting a convention of... #!/usr/bin/env python[major.minor] ...is preferable. It does the same thing without additional runtime overhead and without creating a class of Debian-specific Python programs. ... Sounds pretty dumb, but the missing key point in my thoughts was the virtual package python-X.Y-base (perhaps python-X.Y is better ?). I just didn't get it, and always thought about ugly solutions involving multiple versioned dependencies. That is a good solution. Thank you for the explanation. - Bruce
Re: Status report on python2 transition
Gregor Hoffleit [EMAIL PROTECTED] writes: s/not well-behaved/buggy/: Any binary Python extension package that doesn't depend on 'python-base = X.Y, python-base X.Y+1' is buggy (a few weeks ago I asked in debian-python for volunteers that filed bug reports against those packages; don't know about the current status, though). There's not much we can do about all the Python packages in stable that just depend on python-base (= 1.5.2-1) though, and I don't see why apt would upgrade to a new version of python-pqueue, for example, just because someone does apt-get install python-base after getting their shiny new Debian 3.0 CDs. The only way I can see around this is to scrap python-base and go for python-base-major.minor, or for a new name like python-dist-base. Some sort of python-base-x.y would be nice anyway, for maintainers and for the packaging system, so that modules can have a simple depends on a (maybe virtual) package, instead of a versioned depends. (Does dpkg support versioned virtual packages yet?) -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
On Wed, Jul 04, 2001 at 09:00:06PM -0500, Chris Lawrence wrote: - python-* should provide a Standard Python for each Debian release. 2.1.1 seems a sensible target for woody. These packages should be invoked by /usr/bin/python and /usr/bin/pythonx.y If we are going to have multiple versions of python in woody, then /usr/bin/python should be handled using update-alternatives, perhaps with python-* (2.1.1) having a higher update-alternatives priority than other versions. -- Harry Henry Gebel West Dover Hundred, Delaware GPG encrypted email gladly accepted. Key ID: B853FFFE Fingerprint: 15A6 F58D AEED 5680 B41A 61FE 5A5F BB51 B853 FFFE pgplaq4OnuoEI.pgp Description: PGP signature
Re: Status report on python2 transition
On Wed, 4 Jul 2001, Gregor Hoffleit wrote: First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. Great ... - If we want to support concurrent Python versions and still don't want to have multiple packages for each Python extension, a setup like the Debian Emacsen system might be a solution: Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. Yes! Conclusions --- I have no final conclusion what that means for our Debian Python policy. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. The way we're doing this currently for python-* and python2-* is certainly no good--it means duplication of work, and it's a mess when it comes to installing a new version. If it's possible to do this before woody, by any means I would go that way. Everything else will be a mess later on. We should look at the perl packages (they support concurrent versions) I don't think I've noticed any more than one perl in the archive at a time, whatever they are doing may not be as well tested as... and a Emacsen (they have a system for registration of .el files that can be byte-compiled at installation time, and they support this for concurent and different Emacsen flavors). ...what the emacsen do. No need to reinvent the wheel if it is not necessary, and it will look'n'feel good from a consistency point of view to Debian users. - Bruce
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 07:13:54AM -0700, Neil Schemenauer wrote: Gregor Hoffleit wrote: Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. This doesn't imply that Debian has to support multiple concurrent versions of Python _packages_. To me, it means Debian should play nicely if you want to install other versions of Python in /usr/local or wherever. Currently it does not. Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. Woody should have one core Python package (python-base_2.1.1-X). Extension modules should have a versioned dependency on the interpreter (ie. python (= X.Y)). Pure Python packages should only have a dependency on python or perhaps python (= X.Y). That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. Gregor
Re: Status report on python2 transition
Gregor Hoffleit wrote: Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? Sometimes /usr/local/bin/python is used instead of /usr/bin/python. For example, dput uses #!/usr/bin/env python. Also, its postinst script it does: python -c 'from compileall import main;main()' -q $DIR which fails if a stock distribution of compileall is used (it doesn't support -q). I've seen other packages do this as well. That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? Neil
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 07:56:57AM -0700, Neil Schemenauer wrote: Gregor Hoffleit wrote: Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? Sometimes /usr/local/bin/python is used instead of /usr/bin/python. For example, dput uses #!/usr/bin/env python. Also, its postinst script it does: python -c 'from compileall import main;main()' -q $DIR which fails if a stock distribution of compileall is used (it doesn't support -q). I've seen other packages do this as well. Ok, I see. The postinst problem is my fault; the scripts certainly should use an explicit path to a well-known python installation that does support this. I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of scripts that use #!/usr/bin/env python. If we consider the possibility that somebody installs non-compatible Python versions in the path, then these are bugs in that packages. Thanks for pointing this out! I guess we really, really need a Debian Python policy that mentions all these things. That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? Binary extension modules depend on the version of Python that they were compiled against (a different micro version should be ok, AFAIK). Pure Python extension modules not necessarily depend on the version of Python they were packaged for. There's a tricky situation wrt. byte-compilation in postinst, but currently, this shouldn't hurt us (since on upgrading python-base, site-packages is re-byte-compiled by an compileall.py call). s/not well-behaved/buggy/: Any binary Python extension package that doesn't depend on 'python-base = X.Y, python-base X.Y+1' is buggy (a few weeks ago I asked in debian-python for volunteers that filed bug reports against those packages; don't know about the current status, though). A pure Python extension package that installs stuff in /usr/lib/pythonX.Y and doesn't have this dependency is also buggy. A pure Python extension package that installs stuff in site-python doesn't need a versioned dependency. Gregor
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 05:33:37PM +0200, Gregor Hoffleit wrote: On Thu, Jul 05, 2001 at 07:56:57AM -0700, Neil Schemenauer wrote: That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? I guess I missed your point here. Binary extensions certainly need to be rebuilt once for every Python feature version. Pure Python packages not necessarily would need to be rebuilt (if the code was cross-version compatible). That was the point of my original inquiry on python-dev. Possible solutions for cross-version compatible code would be installation in a version-neutral directory (e.g. /usr/lib/python/site-packages) and either adding this to sys.path (has been depreciated in python-dev) or somehow arranging symlinks into the actual versioned site-package directories. In this case, we would have to setup something comparable to the Debian emacsen's registration system. Gregor
Re: Status report on python2 transition
Gregor Hoffleit wrote: Pure Python packages not necessarily would need to be rebuilt (if the code was cross-version compatible). It almost always is. Python tries very hard to remain source compatible across releases. I've been using Python for 9 years and can only think of one case were my code broke after an upgrade. Possible solutions for cross-version compatible code would be installation in a version-neutral directory (e.g. /usr/lib/python/site-packages) and either adding this to sys.path (has been depreciated in python-dev) or somehow arranging symlinks into the actual versioned site-package directories. I assume we are talking about cross-version compatible _library_ code now (ie. libraries implemented in Python). In that case, something like /usr/lib/python/site-packages sounds like a reasonable solution to me. Note that Python is smart enough to check the version numbers of bytecompiled code. For example if foo.py and foo.pyc both exist but the .pyc file is an old version then the .py file is used. This slows down the import a bit but causes no other ill effects. I think it may be enough to have Python library code install itself in /usr/lib/python/site-packages and be compiled by the currently installed version of Python. If python-base is upgraded then its postinst recompiles all of the code in site-packages. Could it be that simple? Maybe I've missed something. Neil
Re: Status report on python2 transition
In article [EMAIL PROTECTED] you wrote: I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of scripts that use #!/usr/bin/env python. If we consider the possibility that somebody installs non-compatible Python versions in the path, then these are bugs in that packages. Thanks for pointing this out! I guess we really, really need a Debian Python policy that mentions all these things. Especially since #!/usr/bin/env python is recommended in the Python FAQ (section 4.63 -- http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is it bad to use in general, or just bad to use on Debian systems? Aq. -- There have to be people like me, so people like you can look so clever. -- Max Jaeger, Fatherland
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 06:43:08PM +0100, Aquarius wrote: | In article [EMAIL PROTECTED] you wrote: | I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of | scripts that use #!/usr/bin/env python. If we consider the possibility | that somebody installs non-compatible Python versions in the path, then | these are bugs in that packages. Thanks for pointing this out! | | I guess we really, really need a Debian Python policy that mentions all | these things. | | Especially since #!/usr/bin/env python is recommended in the Python | FAQ (section 4.63 -- | http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is | it bad to use in general, or just bad to use on Debian systems? It is bad only when you wanted to use a different version of python than the one that is first in the PATH (which is what /usr/bin/env will tell you when you ask for 'python'). The whole discussion here is how to simultaneously deal with different, potentially incompatible or known to be incompatible, versions. I don't have any real solutions. Maybe as a reference it would be a good idea to mention what I have done for Java on my cygwin (win2k) box at work : I have jdk 1.1.8, jdk 1.2.2, and jdk 1.3.0 installed into various directories. As is common with windows and java everything for that version lies under that base directory (ie d:\jdk1.1.8). I have some shell scripts in my path called 'java1' 'java2' and 'java3'. I have a symlink from 'java' to the version I want to be the default. Then I can pick which JVM to run the java code by simply switching the command slightly. The shell scripts set the JAVA_HOME environment variable to match the selected JVM. I also added a command line switch, --classpath, that _appends_ the jar files to the existing classpath (which would include commonly used libraries such as swing or 3rd party libs used by our product). Maybe that will help someone devise a plan (policy) for Python on Debian. If you want those shell scripts, I am more than happy to provide them. They are really simplistic hacks, though. -D
Re: Status report on python2 transition
Aquarius wrote: Especially since #!/usr/bin/env python is recommended in the Python FAQ (section 4.63 -- http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is it bad to use in general, or just bad to use on Debian systems? Depends. If your writing a script or program to distribute then the env trick is a good idea since you don't know were Python is installed. For Python programs/scripts that come with Debian we know that Python is /usr/bin/python. Neil
Re: Status report on python2 transition
On Thu, 5 Jul 2001, D-Man wrote: On Thu, Jul 05, 2001 at 06:43:08PM +0100, Aquarius wrote: | In article [EMAIL PROTECTED] you wrote: | I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of | scripts that use #!/usr/bin/env python. If we consider the possibility | that somebody installs non-compatible Python versions in the path, then | these are bugs in that packages. Thanks for pointing this out! | | I guess we really, really need a Debian Python policy that mentions all | these things. | | Especially since #!/usr/bin/env python is recommended in the Python | FAQ (section 4.63 -- | http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is | it bad to use in general, or just bad to use on Debian systems? It is bad only when you wanted to use a different version of python than the one that is first in the PATH (which is what /usr/bin/env will tell you when you ask for 'python'). The problem with both the FAQ and Debian's Python is that they have been assuming nobody will ever have more than one Python on the system, and it will always be as recent as the most recent program... as long as the language is backwards compatible there should be no problem -- too bad that is not always the case. The correct way to do it is: #!/usr/bin/env python[major.minor] using this heuristic: If the program will run with any version of Python currently in use then have python execute it, otherwise use the lowest numbered version you know it will run on, or the version of Python you are using if you do not know when the feature was introduced. The nice thing about the Python documentation is that it usually tells you when a feature is new to that version... which makes it not too difficult to decide between using a plain python and a versioned one in the hatch(she)-bang line, without needing to know the history of Python. Is PyChecker smart enough to determine the lowest numbered version of Python a program will run with... that is the kind of functionality our beloved Python maintainers need to make this necessary transition a little less painful (or just let the bug reports flow in, after all, it is gonna be in unstable ;). The whole discussion here is how to simultaneously deal with different, potentially incompatible or known to be incompatible, versions. I don't have any real solutions. Maybe as a reference it would be a good idea to mention what I have done for Java on my cygwin (win2k) box at work : ... Maybe that will help someone devise a plan (policy) for Python on Debian. If you want those shell scripts, I am more than happy to provide them. They are really simplistic hacks, though. Different versions of Python can co-exist quite well (if you don't mind a duplication of .py's), so there should be no need to run a switcher script or wrapper. I do it this way... - build py-whatever in /usr/local - goto the Tools/scripts dir and modify the fixps.py program to change all hatch-bang lines to use a versioned python executable. - get rid of the python[major.minor] -- python hardlink in /usr/local/bin (it is first in my PATH) - use the convention I outlined at the start of this message You can now choose your default Python with a symlink, and every program you write will work no matter which Python is the default... and if Debian's Python maintainers adopt this scheme you will not need to worry about their programs breaking when you change the default interpreter. - Bruce
Re: Status report on python2 transition
On Thu, 5 Jul 2001, Bruce Sass wrote: The problem with both the FAQ and Debian's Python is that they have been assuming nobody will ever have more than one Python on the system, and it will always be as recent as the most recent program... as long as the language is backwards compatible there should be no problem -- too bad that is not always the case. Not always the case that only one version of Python is on the system that is, Python is backwards compatible in my experience. Sorry 'bout that.
Re: Status report on python2 transition
Gregor Hoffleit wrote: Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. This doesn't imply that Debian has to support multiple concurrent versions of Python _packages_. To me, it means Debian should play nicely if you want to install other versions of Python in /usr/local or wherever. Currently it does not. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. Woody should have one core Python package (python-base_2.1.1-X). Extension modules should have a versioned dependency on the interpreter (ie. python (= X.Y)). Pure Python packages should only have a dependency on python or perhaps python (= X.Y). Neil
Re: Status report on python2 transition
Here's my two cents. It might be a bit rambling, given how late it is here... [...] Barry A. Warsaw [EMAIL PROTECTED]: Yes, definitely as both a Zope and Mailman developer wink I need multiple Python versions. But I suspect even normal users of the system will need multiple versions. Different Python-based apps are requiring their users to upgrade Python on their own schedule, so multiple versions will still be required. For the big applications like Zope and Mailman, how much is required in the way of non-core packages? It would be considerably easier if the only versioned Python packages were those compiled from the distribution tarball, and extra packages like python-gnome were only required to be packaged for the latest Python version in time for each Debian release. [...] Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. This seems like a reasonable idea, for code that works with all available Python versions. There's no need to change the Python interpreter to look in different places, or even to change compileall.py. The registration/update script could be in a python-common package that each of the different Python version packages depend on. [...] Gregor Hoffleit: We should look at the perl packages (they support concurrent versions) and a Emacsen (they have a system for registration of .el files that can be byte-compiled at installation time, and they support this for concurent and different Emacsen flavors). *Does* Perl support concurrent versions? All I seem to have available from the mirrors is 5.6.1. Thinking about the transition, it seems to me that relying on all the Python packages to change their dependencies on python-base is going to be hard to get right. It would still be possible to use apt-get to install a new version of python-base, and existing packages that only depend on = 1.5.2 won't be automatically upgraded. Perhaps python-base should be removed in favour of Python package names that include the first and second version number components, something like the perl-api-x.y packages. (python-base-2.1 or just python-2.1?) The python package could be something like the gcc package - mostly just an indication of the default version from the user's point of view, and something to make sure the latest version is available after a dist-upgrade. Though that leaves the problem of the packages currently depending on python... If it's any consolation while you're trying to work out a plan, just be glad Python isn't Essential: yes like parts of Perl. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
On Jul 04, Gregor Hoffleit wrote: First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. That is good news. Now for the bad news: I don't have any conclusive plan how to manage transitions for future Python feature releases, like 2.2, 2.3 and so on. Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. If we're going to support this in Debian regularly, then it get's even much more complicated than with our current setup of python-* and python2-*. Please have a look at the python-dev archives for the full discussions: http://mail.python.org/pipermail/python-dev/2001-July/015715.html My semi-well-thought-out solution: - python-* should provide a Standard Python for each Debian release. 2.1.1 seems a sensible target for woody. These packages should be invoked by /usr/bin/python and /usr/bin/pythonx.y - python-* should (virtual) provide python-x.y-*. - python-v.w-* (where v.w != x.y) can exist at the discretion of the Python maintainer. Provides /usr/bin/pythonv.w I recommend at least having 1.5.2 in woody. 2.0.1 may also be useful. I don't see any good reason for us to ship 1.6.1. - The complete version number of Python should appear in the version field. I recommend using X.Y(.Z)?[abcfp](N?)-rev; this also complies with Debian's sort order. .Z and N can be omitted if 0. However, f is mandatory for a final release, to maintain sort order. (Packages built from CVS could be 2.2a-2001.) - Packages that don't care what Python is installed can depend on python-*. Most simple Python packages fit here. If any features used are deprecated in the most recent X.Y release, they can depend on python-* ( X.Y+1). They can use #!/usr/bin/python. - Packages that provide Python-language library modules: - Depend on python-v.w-* | python-y.z-* | ... for all versions they can run on. - Provide /usr/lib/pythonX.Y/site-packages/... for all versions they can run on; I'd use symlinks if the code needn't change. - Invoke compileall.py for each version available. - Run python-wrapper-config * in postinst and postrm. - Packages that provide binary-linked library modules: - Depend on python-v.w-* | python-y.z-* | ... for all versions they can run on. - Build-depend on python-v.w-*, python-y.z-*, ... for all versions they can run on. - Provide /usr/lib/pythonX.Y/site-packages/... for all versions they can run on. - If two releases share the same API version, the same binaries may be shared in site-packages. (Not sure if this happens in practice.) - Follow the above policy with regards to any Python-language modules. - Run python-wrapper-config * in postinst and postrm. - Python packages that need particular modules: This is sticky. - Depend on the needed packages to provide the modules. - Depend on python-* (= X.Y) if it uses features unique to X.Y+ - In postinst, figure out the python-x.y packages that are installed and provide all necessary modules and fulfill the dependencies for this package. Write this to a file. (This is handled by python-wrapper-config *) We need to register each package that cares about this in a file (/var/lib/python/needs-wrapper *) for module postinst/rm. We are guaranteed to have at least one Python version that works if one module is involved by dependencies. If multiple modules are involved, we may have zero working versions (which is a bug) and will be detected by this step, causing package install to fail. How to do this: - Executables use #!/usr/bin/python-wrapper (*) - Have a file /var/lib/python/wrapper-config (*) with executable: version1, version2, ... This is written by the postinst. - Have a python-wrapper that reads $1, figures out which executable is involved, and invokes the appropriate python. (It can probably be in Python itself... perhaps provided by python-x.y-base and managed by alternatives.) - This figuring out will have to be done by each python-x.y-* and each modules package on install and remove too. - Remove this cruft in the postrm. (*) Rename this to something better if you like. The main disadvantage is that people compiling binary-linked library modules will need multiple Pythons installed; of course, unless some magic API fix comes along, they would anyway. It also doesn't include any way to accomodate locally-built Pythons, unless they are built and installed as
Status report on python2 transition
First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. Now for the bad news: I don't have any conclusive plan how to manage transitions for future Python feature releases, like 2.2, 2.3 and so on. Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. If we're going to support this in Debian regularly, then it get's even much more complicated than with our current setup of python-* and python2-*. Please have a look at the python-dev archives for the full discussions: http://mail.python.org/pipermail/python-dev/2001-July/015715.html The discussion -- - Python has now a reliable, well-behaved version numbering policy: - feature releases: may introduce new features, may change the bytecode format. The minor (middle) version number changes (e.g. 1.5.2 - 1.6, 2.0 - 2.1). - micro-versions: bug fixes only, bytecode format can't vary. Only the micro component of the version number changes (e.g. 1.5.1 - 1.5.2 or 2.0 - 2.0.1). - site-python is depreciated; Python code should go into a version specific directory (originally I thought I might use site-python for all packages--would make Python updates very much easier): Greg Ward [EMAIL PROTECTED]: Oh yeah, another thing I vaguely recall from the pre-Distutils-0.1 era: Guido doesn't (didn't?) like site-python and wanted to deprecate it. ... BTW, I'm skeptical about keeping .py and .pyc code in a non-Python-version-specific directory (ie. site-python). Debian's bytecode-recompilation at installation time scheme sounds cool, but the desire/need to have multiple Python versions available kind of nixes it. Fred L. Drake, Jr. [EMAIL PROTECTED]: Another reason not to use site-python is that it is actually still hard to write cross-version Python code -- there are enough differences that any substantial volume of code (and in Python, you don't need many KLoC to get substantial code!) is bound to encounter a few, especially if you get used to using only Python 2.0+ -- it's easy to get used to features like string methods, list comprehensions, and augmented assignment! The site-packages directory was introduced to avoid the deficiencies of the site-python directory. Guido van Rossum [EMAIL PROTECTED]: Indeed, /usr/local/lib/pythonmajor.minor/site-packages/ is where site packages should go. - Support for multiple concurrent Python versions on the same system is necessary: Barry A. Warsaw [EMAIL PROTECTED]: GH Seriously, does anybody (besides the Python developers) feel a GH need to have multiple Python versions on the same system ? Yes, definitely as both a Zope and Mailman developer wink I need multiple Python versions. But I suspect even normal users of the system will need multiple versions. Different Python-based apps are requiring their users to upgrade Python on their own schedule, so multiple versions will still be required. Fred L. Drake, Jr. [EMAIL PROTECTED]: Another excellent reason to support multiple versions! As more widely distributed applications are written using Python and don't want to include the interpreter, this becomes a more noticable issue. - If we want to support concurrent Python versions and still don't want to have multiple packages for each Python extension, a setup like the Debian Emacsen system might be a solution: Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. Conclusions --- I have no final conclusion what that means for our Debian Python policy. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. The way we're doing this currently for python-* and python2-* is certainly no good--it means duplication of work, and it's a mess when it comes to installing a new version. If it's possible to do this before woody, by any means I would go that way. Everything else will be a mess later on. We should look at the perl packages (they support
