Re: Status report on python2 transition
On Wed, Jul 04, 2001 at 09:00:06PM -0500, Chris Lawrence wrote: - python-* should provide a Standard Python for each Debian release. 2.1.1 seems a sensible target for woody. These packages should be invoked by /usr/bin/python and /usr/bin/pythonx.y If we are going to have multiple versions of python in woody, then /usr/bin/python should be handled using update-alternatives, perhaps with python-* (2.1.1) having a higher update-alternatives priority than other versions. -- Harry Henry Gebel West Dover Hundred, Delaware GPG encrypted email gladly accepted. Key ID: B853FFFE Fingerprint: 15A6 F58D AEED 5680 B41A 61FE 5A5F BB51 B853 FFFE pgplaq4OnuoEI.pgp Description: PGP signature
Re: Status report on python2 transition
On Wed, 4 Jul 2001, Gregor Hoffleit wrote: First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. Great ... - If we want to support concurrent Python versions and still don't want to have multiple packages for each Python extension, a setup like the Debian Emacsen system might be a solution: Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. Yes! Conclusions --- I have no final conclusion what that means for our Debian Python policy. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. The way we're doing this currently for python-* and python2-* is certainly no good--it means duplication of work, and it's a mess when it comes to installing a new version. If it's possible to do this before woody, by any means I would go that way. Everything else will be a mess later on. We should look at the perl packages (they support concurrent versions) I don't think I've noticed any more than one perl in the archive at a time, whatever they are doing may not be as well tested as... and a Emacsen (they have a system for registration of .el files that can be byte-compiled at installation time, and they support this for concurent and different Emacsen flavors). ...what the emacsen do. No need to reinvent the wheel if it is not necessary, and it will look'n'feel good from a consistency point of view to Debian users. - Bruce
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 07:13:54AM -0700, Neil Schemenauer wrote: Gregor Hoffleit wrote: Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. This doesn't imply that Debian has to support multiple concurrent versions of Python _packages_. To me, it means Debian should play nicely if you want to install other versions of Python in /usr/local or wherever. Currently it does not. Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. Woody should have one core Python package (python-base_2.1.1-X). Extension modules should have a versioned dependency on the interpreter (ie. python (= X.Y)). Pure Python packages should only have a dependency on python or perhaps python (= X.Y). That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. Gregor
Re: Status report on python2 transition
Gregor Hoffleit wrote: Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? Sometimes /usr/local/bin/python is used instead of /usr/bin/python. For example, dput uses #!/usr/bin/env python. Also, its postinst script it does: python -c 'from compileall import main;main()' -q $DIR which fails if a stock distribution of compileall is used (it doesn't support -q). I've seen other packages do this as well. That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? Neil
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 07:56:57AM -0700, Neil Schemenauer wrote: Gregor Hoffleit wrote: Sorry ? What problems do you have installing Python 2.1 in /usr/local on a Debian system ? Sometimes /usr/local/bin/python is used instead of /usr/bin/python. For example, dput uses #!/usr/bin/env python. Also, its postinst script it does: python -c 'from compileall import main;main()' -q $DIR which fails if a stock distribution of compileall is used (it doesn't support -q). I've seen other packages do this as well. Ok, I see. The postinst problem is my fault; the scripts certainly should use an explicit path to a well-known python installation that does support this. I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of scripts that use #!/usr/bin/env python. If we consider the possibility that somebody installs non-compatible Python versions in the path, then these are bugs in that packages. Thanks for pointing this out! I guess we really, really need a Debian Python policy that mentions all these things. That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? Binary extension modules depend on the version of Python that they were compiled against (a different micro version should be ok, AFAIK). Pure Python extension modules not necessarily depend on the version of Python they were packaged for. There's a tricky situation wrt. byte-compilation in postinst, but currently, this shouldn't hurt us (since on upgrading python-base, site-packages is re-byte-compiled by an compileall.py call). s/not well-behaved/buggy/: Any binary Python extension package that doesn't depend on 'python-base = X.Y, python-base X.Y+1' is buggy (a few weeks ago I asked in debian-python for volunteers that filed bug reports against those packages; don't know about the current status, though). A pure Python extension package that installs stuff in /usr/lib/pythonX.Y and doesn't have this dependency is also buggy. A pure Python extension package that installs stuff in site-python doesn't need a versioned dependency. Gregor
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 05:33:37PM +0200, Gregor Hoffleit wrote: On Thu, Jul 05, 2001 at 07:56:57AM -0700, Neil Schemenauer wrote: That's our current setup (well-behaved packages should have a dependency on python-base = 1.5, python-base 1.6). Look at the mess we're now running into, now that we want to upgrade this to Python 2.1.1. All packages have to be recompiled at once. What's a well-behaved package? Extension modules depend on the version of Python that they were compiled against. If you upgrade the interpreter you must upgrade the extensions. How are you planning on avoiding this? I guess I missed your point here. Binary extensions certainly need to be rebuilt once for every Python feature version. Pure Python packages not necessarily would need to be rebuilt (if the code was cross-version compatible). That was the point of my original inquiry on python-dev. Possible solutions for cross-version compatible code would be installation in a version-neutral directory (e.g. /usr/lib/python/site-packages) and either adding this to sys.path (has been depreciated in python-dev) or somehow arranging symlinks into the actual versioned site-package directories. In this case, we would have to setup something comparable to the Debian emacsen's registration system. Gregor
Re: Status report on python2 transition
Gregor Hoffleit wrote: Pure Python packages not necessarily would need to be rebuilt (if the code was cross-version compatible). It almost always is. Python tries very hard to remain source compatible across releases. I've been using Python for 9 years and can only think of one case were my code broke after an upgrade. Possible solutions for cross-version compatible code would be installation in a version-neutral directory (e.g. /usr/lib/python/site-packages) and either adding this to sys.path (has been depreciated in python-dev) or somehow arranging symlinks into the actual versioned site-package directories. I assume we are talking about cross-version compatible _library_ code now (ie. libraries implemented in Python). In that case, something like /usr/lib/python/site-packages sounds like a reasonable solution to me. Note that Python is smart enough to check the version numbers of bytecompiled code. For example if foo.py and foo.pyc both exist but the .pyc file is an old version then the .py file is used. This slows down the import a bit but causes no other ill effects. I think it may be enough to have Python library code install itself in /usr/lib/python/site-packages and be compiled by the currently installed version of Python. If python-base is upgraded then its postinst recompiles all of the code in site-packages. Could it be that simple? Maybe I've missed something. Neil
Re: Status report on python2 transition
In article [EMAIL PROTECTED] you wrote: I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of scripts that use #!/usr/bin/env python. If we consider the possibility that somebody installs non-compatible Python versions in the path, then these are bugs in that packages. Thanks for pointing this out! I guess we really, really need a Debian Python policy that mentions all these things. Especially since #!/usr/bin/env python is recommended in the Python FAQ (section 4.63 -- http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is it bad to use in general, or just bad to use on Debian systems? Aq. -- There have to be people like me, so people like you can look so clever. -- Max Jaeger, Fatherland
Re: Status report on python2 transition
On Thu, Jul 05, 2001 at 06:43:08PM +0100, Aquarius wrote: | In article [EMAIL PROTECTED] you wrote: | I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of | scripts that use #!/usr/bin/env python. If we consider the possibility | that somebody installs non-compatible Python versions in the path, then | these are bugs in that packages. Thanks for pointing this out! | | I guess we really, really need a Debian Python policy that mentions all | these things. | | Especially since #!/usr/bin/env python is recommended in the Python | FAQ (section 4.63 -- | http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is | it bad to use in general, or just bad to use on Debian systems? It is bad only when you wanted to use a different version of python than the one that is first in the PATH (which is what /usr/bin/env will tell you when you ask for 'python'). The whole discussion here is how to simultaneously deal with different, potentially incompatible or known to be incompatible, versions. I don't have any real solutions. Maybe as a reference it would be a good idea to mention what I have done for Java on my cygwin (win2k) box at work : I have jdk 1.1.8, jdk 1.2.2, and jdk 1.3.0 installed into various directories. As is common with windows and java everything for that version lies under that base directory (ie d:\jdk1.1.8). I have some shell scripts in my path called 'java1' 'java2' and 'java3'. I have a symlink from 'java' to the version I want to be the default. Then I can pick which JVM to run the java code by simply switching the command slightly. The shell scripts set the JAVA_HOME environment variable to match the selected JVM. I also added a command line switch, --classpath, that _appends_ the jar files to the existing classpath (which would include commonly used libraries such as swing or 3rd party libs used by our product). Maybe that will help someone devise a plan (policy) for Python on Debian. If you want those shell scripts, I am more than happy to provide them. They are really simplistic hacks, though. -D
Re: Status report on python2 transition
Aquarius wrote: Especially since #!/usr/bin/env python is recommended in the Python FAQ (section 4.63 -- http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is it bad to use in general, or just bad to use on Debian systems? Depends. If your writing a script or program to distribute then the env trick is a good idea since you don't know were Python is installed. For Python programs/scripts that come with Debian we know that Python is /usr/bin/python. Neil
Re: Status report on python2 transition
On Thu, 5 Jul 2001, D-Man wrote: On Thu, Jul 05, 2001 at 06:43:08PM +0100, Aquarius wrote: | In article [EMAIL PROTECTED] you wrote: | I just browsed /usr/bin and /usr/sbin, and indeed there are plenty of | scripts that use #!/usr/bin/env python. If we consider the possibility | that somebody installs non-compatible Python versions in the path, then | these are bugs in that packages. Thanks for pointing this out! | | I guess we really, really need a Debian Python policy that mentions all | these things. | | Especially since #!/usr/bin/env python is recommended in the Python | FAQ (section 4.63 -- | http://www.python.org/cgi-bin/faqw.py?req=showfile=faq04.063.htp). Is | it bad to use in general, or just bad to use on Debian systems? It is bad only when you wanted to use a different version of python than the one that is first in the PATH (which is what /usr/bin/env will tell you when you ask for 'python'). The problem with both the FAQ and Debian's Python is that they have been assuming nobody will ever have more than one Python on the system, and it will always be as recent as the most recent program... as long as the language is backwards compatible there should be no problem -- too bad that is not always the case. The correct way to do it is: #!/usr/bin/env python[major.minor] using this heuristic: If the program will run with any version of Python currently in use then have python execute it, otherwise use the lowest numbered version you know it will run on, or the version of Python you are using if you do not know when the feature was introduced. The nice thing about the Python documentation is that it usually tells you when a feature is new to that version... which makes it not too difficult to decide between using a plain python and a versioned one in the hatch(she)-bang line, without needing to know the history of Python. Is PyChecker smart enough to determine the lowest numbered version of Python a program will run with... that is the kind of functionality our beloved Python maintainers need to make this necessary transition a little less painful (or just let the bug reports flow in, after all, it is gonna be in unstable ;). The whole discussion here is how to simultaneously deal with different, potentially incompatible or known to be incompatible, versions. I don't have any real solutions. Maybe as a reference it would be a good idea to mention what I have done for Java on my cygwin (win2k) box at work : ... Maybe that will help someone devise a plan (policy) for Python on Debian. If you want those shell scripts, I am more than happy to provide them. They are really simplistic hacks, though. Different versions of Python can co-exist quite well (if you don't mind a duplication of .py's), so there should be no need to run a switcher script or wrapper. I do it this way... - build py-whatever in /usr/local - goto the Tools/scripts dir and modify the fixps.py program to change all hatch-bang lines to use a versioned python executable. - get rid of the python[major.minor] -- python hardlink in /usr/local/bin (it is first in my PATH) - use the convention I outlined at the start of this message You can now choose your default Python with a symlink, and every program you write will work no matter which Python is the default... and if Debian's Python maintainers adopt this scheme you will not need to worry about their programs breaking when you change the default interpreter. - Bruce
Re: Status report on python2 transition
On Thu, 5 Jul 2001, Bruce Sass wrote: The problem with both the FAQ and Debian's Python is that they have been assuming nobody will ever have more than one Python on the system, and it will always be as recent as the most recent program... as long as the language is backwards compatible there should be no problem -- too bad that is not always the case. Not always the case that only one version of Python is on the system that is, Python is backwards compatible in my experience. Sorry 'bout that.
Re: Status report on python2 transition
Gregor Hoffleit wrote: Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. This doesn't imply that Debian has to support multiple concurrent versions of Python _packages_. To me, it means Debian should play nicely if you want to install other versions of Python in /usr/local or wherever. Currently it does not. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. Woody should have one core Python package (python-base_2.1.1-X). Extension modules should have a versioned dependency on the interpreter (ie. python (= X.Y)). Pure Python packages should only have a dependency on python or perhaps python (= X.Y). Neil
