Re: Status report on python2 transition
Here's my two cents. It might be a bit rambling, given how late it is here... [...] Barry A. Warsaw [EMAIL PROTECTED]: Yes, definitely as both a Zope and Mailman developer wink I need multiple Python versions. But I suspect even normal users of the system will need multiple versions. Different Python-based apps are requiring their users to upgrade Python on their own schedule, so multiple versions will still be required. For the big applications like Zope and Mailman, how much is required in the way of non-core packages? It would be considerably easier if the only versioned Python packages were those compiled from the distribution tarball, and extra packages like python-gnome were only required to be packaged for the latest Python version in time for each Debian release. [...] Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. This seems like a reasonable idea, for code that works with all available Python versions. There's no need to change the Python interpreter to look in different places, or even to change compileall.py. The registration/update script could be in a python-common package that each of the different Python version packages depend on. [...] Gregor Hoffleit: We should look at the perl packages (they support concurrent versions) and a Emacsen (they have a system for registration of .el files that can be byte-compiled at installation time, and they support this for concurent and different Emacsen flavors). *Does* Perl support concurrent versions? All I seem to have available from the mirrors is 5.6.1. Thinking about the transition, it seems to me that relying on all the Python packages to change their dependencies on python-base is going to be hard to get right. It would still be possible to use apt-get to install a new version of python-base, and existing packages that only depend on = 1.5.2 won't be automatically upgraded. Perhaps python-base should be removed in favour of Python package names that include the first and second version number components, something like the perl-api-x.y packages. (python-base-2.1 or just python-2.1?) The python package could be something like the gcc package - mostly just an indication of the default version from the user's point of view, and something to make sure the latest version is available after a dist-upgrade. Though that leaves the problem of the packages currently depending on python... If it's any consolation while you're trying to work out a plan, just be glad Python isn't Essential: yes like parts of Perl. -- Carey Evans http://home.clear.net.nz/pages/c.evans/ Quiet, you'll miss the humorous conclusion.
Re: Status report on python2 transition
On Jul 04, Gregor Hoffleit wrote: First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. That is good news. Now for the bad news: I don't have any conclusive plan how to manage transitions for future Python feature releases, like 2.2, 2.3 and so on. Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. If we're going to support this in Debian regularly, then it get's even much more complicated than with our current setup of python-* and python2-*. Please have a look at the python-dev archives for the full discussions: http://mail.python.org/pipermail/python-dev/2001-July/015715.html My semi-well-thought-out solution: - python-* should provide a Standard Python for each Debian release. 2.1.1 seems a sensible target for woody. These packages should be invoked by /usr/bin/python and /usr/bin/pythonx.y - python-* should (virtual) provide python-x.y-*. - python-v.w-* (where v.w != x.y) can exist at the discretion of the Python maintainer. Provides /usr/bin/pythonv.w I recommend at least having 1.5.2 in woody. 2.0.1 may also be useful. I don't see any good reason for us to ship 1.6.1. - The complete version number of Python should appear in the version field. I recommend using X.Y(.Z)?[abcfp](N?)-rev; this also complies with Debian's sort order. .Z and N can be omitted if 0. However, f is mandatory for a final release, to maintain sort order. (Packages built from CVS could be 2.2a-2001.) - Packages that don't care what Python is installed can depend on python-*. Most simple Python packages fit here. If any features used are deprecated in the most recent X.Y release, they can depend on python-* ( X.Y+1). They can use #!/usr/bin/python. - Packages that provide Python-language library modules: - Depend on python-v.w-* | python-y.z-* | ... for all versions they can run on. - Provide /usr/lib/pythonX.Y/site-packages/... for all versions they can run on; I'd use symlinks if the code needn't change. - Invoke compileall.py for each version available. - Run python-wrapper-config * in postinst and postrm. - Packages that provide binary-linked library modules: - Depend on python-v.w-* | python-y.z-* | ... for all versions they can run on. - Build-depend on python-v.w-*, python-y.z-*, ... for all versions they can run on. - Provide /usr/lib/pythonX.Y/site-packages/... for all versions they can run on. - If two releases share the same API version, the same binaries may be shared in site-packages. (Not sure if this happens in practice.) - Follow the above policy with regards to any Python-language modules. - Run python-wrapper-config * in postinst and postrm. - Python packages that need particular modules: This is sticky. - Depend on the needed packages to provide the modules. - Depend on python-* (= X.Y) if it uses features unique to X.Y+ - In postinst, figure out the python-x.y packages that are installed and provide all necessary modules and fulfill the dependencies for this package. Write this to a file. (This is handled by python-wrapper-config *) We need to register each package that cares about this in a file (/var/lib/python/needs-wrapper *) for module postinst/rm. We are guaranteed to have at least one Python version that works if one module is involved by dependencies. If multiple modules are involved, we may have zero working versions (which is a bug) and will be detected by this step, causing package install to fail. How to do this: - Executables use #!/usr/bin/python-wrapper (*) - Have a file /var/lib/python/wrapper-config (*) with executable: version1, version2, ... This is written by the postinst. - Have a python-wrapper that reads $1, figures out which executable is involved, and invokes the appropriate python. (It can probably be in Python itself... perhaps provided by python-x.y-base and managed by alternatives.) - This figuring out will have to be done by each python-x.y-* and each modules package on install and remove too. - Remove this cruft in the postrm. (*) Rename this to something better if you like. The main disadvantage is that people compiling binary-linked library modules will need multiple Pythons installed; of course, unless some magic API fix comes along, they would anyway. It also doesn't include any way to accomodate locally-built Pythons, unless they are built and installed as
Status report on python2 transition
First of all the good news: You have managed to talk me into making the big step, and going right to the 2.1.1 CVS branch. Thomas Wouters (release czar for Python 2.1.1) assured me that 2.1.1 will be released before the freeze, and Guido heavily supported that. Now for the bad news: I don't have any conclusive plan how to manage transitions for future Python feature releases, like 2.2, 2.3 and so on. Until now I had the impression that in general it's not necessary to have more than one Python version on your machine at the same time (except perhaps you're a Python core developer). Feedback from python-dev though was that it's definitely necessary to allow and support multiple concurrent versions of Python even on production machines. If we're going to support this in Debian regularly, then it get's even much more complicated than with our current setup of python-* and python2-*. Please have a look at the python-dev archives for the full discussions: http://mail.python.org/pipermail/python-dev/2001-July/015715.html The discussion -- - Python has now a reliable, well-behaved version numbering policy: - feature releases: may introduce new features, may change the bytecode format. The minor (middle) version number changes (e.g. 1.5.2 - 1.6, 2.0 - 2.1). - micro-versions: bug fixes only, bytecode format can't vary. Only the micro component of the version number changes (e.g. 1.5.1 - 1.5.2 or 2.0 - 2.0.1). - site-python is depreciated; Python code should go into a version specific directory (originally I thought I might use site-python for all packages--would make Python updates very much easier): Greg Ward [EMAIL PROTECTED]: Oh yeah, another thing I vaguely recall from the pre-Distutils-0.1 era: Guido doesn't (didn't?) like site-python and wanted to deprecate it. ... BTW, I'm skeptical about keeping .py and .pyc code in a non-Python-version-specific directory (ie. site-python). Debian's bytecode-recompilation at installation time scheme sounds cool, but the desire/need to have multiple Python versions available kind of nixes it. Fred L. Drake, Jr. [EMAIL PROTECTED]: Another reason not to use site-python is that it is actually still hard to write cross-version Python code -- there are enough differences that any substantial volume of code (and in Python, you don't need many KLoC to get substantial code!) is bound to encounter a few, especially if you get used to using only Python 2.0+ -- it's easy to get used to features like string methods, list comprehensions, and augmented assignment! The site-packages directory was introduced to avoid the deficiencies of the site-python directory. Guido van Rossum [EMAIL PROTECTED]: Indeed, /usr/local/lib/pythonmajor.minor/site-packages/ is where site packages should go. - Support for multiple concurrent Python versions on the same system is necessary: Barry A. Warsaw [EMAIL PROTECTED]: GH Seriously, does anybody (besides the Python developers) feel a GH need to have multiple Python versions on the same system ? Yes, definitely as both a Zope and Mailman developer wink I need multiple Python versions. But I suspect even normal users of the system will need multiple versions. Different Python-based apps are requiring their users to upgrade Python on their own schedule, so multiple versions will still be required. Fred L. Drake, Jr. [EMAIL PROTECTED]: Another excellent reason to support multiple versions! As more widely distributed applications are written using Python and don't want to include the interpreter, this becomes a more noticable issue. - If we want to support concurrent Python versions and still don't want to have multiple packages for each Python extension, a setup like the Debian Emacsen system might be a solution: Thomas Wouters [EMAIL PROTECTED]: None are compatible. This might change, but I don't think so -- I think the CVS tree already has a different bytecode magic than 2.1, though I haven't checked. Perhaps what Gregor wants is a set of symlinks in each python version's site-packages directory, to a system-wide one, and a 'register-python-version' script like the emacs/xemacs stuff has that adds those symlinks. That way, the .pyc/.pyo versions would remain in the version-specific directory. Conclusions --- I have no final conclusion what that means for our Debian Python policy. For one, I've changed my mind and accepted that there's a need to support multiple concurrent Python versions in Debian. The way we're doing this currently for python-* and python2-* is certainly no good--it means duplication of work, and it's a mess when it comes to installing a new version. If it's possible to do this before woody, by any means I would go that way. Everything else will be a mess later on. We should look at the perl packages (they support
