Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread M.-A. Lemburg
Martin v. Löwis wrote: -1 on always using wchar_t as well. Python's default is UCS2 and the stable ABI should not change that. It's not really Python's default. It is what configure.in does by default. Python's default, on Linux, is UCS-4. No, the default is UCS2 on all platforms and in

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread Amaury Forgeot d'Arc
Hi, 2010/9/7 M.-A. Lemburg m...@egenix.com: Ok. I'm fine with excluding Py_UNICODE from the stable ABI. However, I think in the long run, I guess more support for wchar_t will then be needed in the API, e.g. more convenient argument parsing. Sure, we could add that. Just to be clear: does

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread M.-A. Lemburg
Amaury Forgeot d'Arc wrote: Hi, 2010/9/7 M.-A. Lemburg m...@egenix.com: Ok. I'm fine with excluding Py_UNICODE from the stable ABI. However, I think in the long run, I guess more support for wchar_t will then be needed in the API, e.g. more convenient argument parsing. Sure, we could add

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-04 Thread Martin v. Löwis
-1 on always using wchar_t as well. Python's default is UCS2 and the stable ABI should not change that. It's not really Python's default. It is what configure.in does by default. Python's default, on Linux, is UCS-4. I also think that this information is not relevant for the stable ABI:

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread Ronald Oussoren
On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote: - wide-unicode: this is a tricky one. I'm tempted to say that the stable ABI should always use a Py_UNICODE that matches the platform's wchar_t. Alternative proposals are welcome. Sizeof(wchar_t) is 4 on OSX, but the Apple frameworks use a

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread M.-A. Lemburg
Ronald Oussoren wrote: On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote: - wide-unicode: this is a tricky one. I'm tempted to say that the stable ABI should always use a Py_UNICODE that matches the platform's wchar_t. Alternative proposals are welcome. Sizeof(wchar_t) is 4 on OSX, but

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread Barry Warsaw
On Aug 31, 2010, at 11:22 AM, M.-A. Lemburg wrote: BTW: Wasn't one of the main reasons for having versioned .so files the idea to be able to have UCS2 and UCS4 versions installed side-by-side ? Yes. This isn't an issue for PEP 3149 because it adds a flag to the shared library file name for wide

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-30 Thread Barry Warsaw
On Aug 28, 2010, at 12:29 PM, Martin v. Löwis wrote: The intention is that there is indeed just one stable ABI, so one configuration is the supported one, and that should be the default build. As for the specific settings, my analysis would be this: - pydebug: not supported by the stable ABI, as

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-28 Thread Martin v. Löwis
This leads me to a question: how do these configure options affect the PEP 384 stable ABI? That PEP is currently silent on the issue, while PEP 3149 appears to implicitly assume that abi3 completely specifies the ABI. It's a great question - perhaps Martin can chime in? It may be that

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Matthias Klose
On 26.07.2010 22:53, Ralf Schmitt wrote: Barry Warsawba...@python.org writes: That's fine, but it's not the way Debian/Ubuntu works today. PEP 3149 adoption will definitely remove significant complication for deploying multiple Python versions at the same time on those systems. You're just

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Ralf Schmitt
Matthias Klose d...@ubuntu.com writes: Not true. Package managers like dpkg/apt-get, rpm/yum and maybe others do this for ages. And yes, the added complexity of package managers does lead to increased robustness. but how does sharing things lead to increased robustness (even if it might be

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Barry Warsaw
On Jul 26, 2010, at 10:53 PM, Ralf Schmitt wrote: Some of the things that need to be adapted are e.g. Makefiles (basically anything that assumes modules have a certain name), all of the freezers (cxFreeze, py2exe, ...). The biggest problem probably will be that an import will load the wrong

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Barry Warsaw
On Jul 27, 2010, at 01:54 PM, Ralf Schmitt wrote: Matthias Klose d...@ubuntu.com writes: Not true. Package managers like dpkg/apt-get, rpm/yum and maybe others do this for ages. And yes, the added complexity of package managers does lead to increased robustness. but how does sharing things

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Barry Warsaw
On Jul 24, 2010, at 09:54 AM, Ronald Oussoren wrote: I'd be okay including a configure option to allow you to add whatever you want after the implementation, version, and flags. E.g. something like: ./configure --with-abi-tag-extension=linux2 would lead to foo.cpython-32m-linux2.so,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Barry Warsaw
On Jul 24, 2010, at 11:59 PM, sch...@gmail.com wrote: Barry Warsaw ba...@python.org writes: On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: Doesn't anybody else think this is lost work for very little gain? My /usr/lib/python2.6/site-packages directory consumes 200MB on disk. I couldn't

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Ralf Schmitt
Barry Warsaw ba...@python.org writes: Do you have concrete examples? Without that it's just speculation I can't do much to address. Is the problem big or small? Easy to work around or not? Some of the things that need to be adapted are e.g. Makefiles (basically anything that assumes

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-24 Thread Ronald Oussoren
On 23 Jul, 2010, at 23:19, Barry Warsaw wrote: On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-24 Thread schmir
Barry Warsaw ba...@python.org writes: On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: Doesn't anybody else think this is lost work for very little gain? My /usr/lib/python2.6/site-packages directory consumes 200MB on disk. I couldn't care less if my /usr/lib/python2.5/site-packages

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 22 Jul, 2010, at 15:40, Barry Warsaw wrote: Abstract PEP 3147 [1]_ described an extension to Python's import machinery that improved the sharing of Python source code, by allowing more than one byte compilation file (.pyc) to be co-located with each source file. This PEP

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: I guess this is not an explicit goal of this PEP, but the structure is very close to supporting multiple system architectures at the same time. I regularly develop code that needs to run on Windows, Linux and OSX and it is very convenient to

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 23 Jul, 2010, at 11:02, Barry Warsaw wrote: On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: I guess this is not an explicit goal of this PEP, but the structure is very close to supporting multiple system architectures at the same time. I regularly develop code that needs to run on

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe start with the platform name, e.g. foo.linux2-cpython-32m. This isn't a strong

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Nick Coghlan
On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw ba...@python.org wrote: Python implementations *MAY* include additional flags in the file name tag as appropriate.  For example, on POSIX systems these flags will also contribute to the file name:  * ``--with-pydebug`` (flag: ``d``)  *

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe start with the platform name,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread schmir
Ronald Oussoren ronaldousso...@mac.com writes: On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: What about the architecture (i386, amd64)? With every increase in length I start to get more concerned. We could encode the platform and architecture, but that gets into cryptic territory.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe start with the platform name,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: Doesn't anybody else think this is lost work for very little gain? My /usr/lib/python2.6/site-packages directory consumes 200MB on disk. I couldn't care less if my /usr/lib/python2.5/site-packages consumed the same amount of disk space...

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 08:56 PM, Nick Coghlan wrote: On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw ba...@python.org wrote: Python implementations *MAY* include additional flags in the file name tag as appropriate.  For example, on POSIX systems these flags will also contribute to the file name:

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-22 Thread Barry Warsaw
On Jul 16, 2010, at 12:40 PM, Matthias Klose wrote: I like the proposal, but IMO it is too unspecific about the abi tag. Assume that an extension is built with such a configured python and then tried to run with an abi compatible configured python, but with a slightly different version tag, the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-21 Thread Barry Warsaw
On Jul 16, 2010, at 12:16 AM, Amaury Forgeot d'Arc wrote: 2010/7/15 Barry Warsaw ba...@python.org: The first draft of PEP 3149 is ready for review. I like it! Cool! I think it could mention the case where packages are not installed in the canonical directory, but placed elsewhere along the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Matthias Klose
On 15.07.2010 01:59, Barry Warsaw wrote: PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is passed as the API version if the extension was complied with ``Py_LIMITED_API``. This should be formalized into an official macro called ``PYTHON_ABI_VERSION`` to mirror

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Daniel Stutzbach
On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose d...@ubuntu.com wrote: 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP should define a non-configurable way for the tag. The tag should include all information which currently makes an extension ABI

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Matthias Klose
On 16.07.2010 15:43, Daniel Stutzbach wrote: On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klosed...@ubuntu.com wrote: 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP should define a non-configurable way for the tag. The tag should include all information which

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-15 Thread Amaury Forgeot d'Arc
Hello, 2010/7/15 Barry Warsaw ba...@python.org: The first draft of PEP 3149 is ready for review. I like it! I think it could mention the case where packages are not installed in the canonical directory, but placed elsewhere along the PYTHONPATH. This is how I deploy applications, for example,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-14 Thread Barry Warsaw
On Jun 25, 2010, at 08:35 AM, Nick Coghlan wrote: I like the idea, but I think summarising the rest of this discussion in its own (relatively short) PEP would be good (there are a few things that are tricky - exact versioning scheme, PEP 384 forward compatibility, impact on distutils,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-09 Thread Barry Warsaw
On Jul 08, 2010, at 09:14 AM, Georg Brandl wrote: Am 07.07.2010 23:04, schrieb Georg Brandl: I can see where this is going... writing it into PEP 384 would automatically get the change accepted? I'm definitely not trying to get it in subversively. :) I hit Send prematurely. I wanted to add

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-09 Thread Barry Warsaw
On Jul 08, 2010, at 01:47 AM, Matthias Klose wrote: On 07.07.2010 20:40, Barry Warsaw wrote: Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-08 Thread Georg Brandl
Am 07.07.2010 23:04, schrieb Georg Brandl: Am 07.07.2010 20:40, schrieb Barry Warsaw: Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Barry Warsaw
On Jul 01, 2010, at 07:02 AM, Scott Dial wrote: I decided to prove to myself that it was not a significant issue to have parallel directory structures in a .tar.bz2, and I was surprised to find it much worse at that then I had imagined. For example, # cd /usr/lib/python2.6/site-packages # tar

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Georg Brandl
Am 07.07.2010 20:40, schrieb Barry Warsaw: Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one difference from previous versions of the patch is

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread John Arbash Meinel
Scott Dial wrote: On 6/30/2010 2:53 PM, Barry Warsaw wrote: It might be amazing, but it's still a significant overhead. As I've described, multiply that by all the py files in all the distro packages containing Python source code, and then still try to fit it on a CDROM. I decided to prove

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Matthias Klose
On 07.07.2010 20:40, Barry Warsaw wrote: Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one difference from previous versions of the patch is that

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-01 Thread Scott Dial
On 6/30/2010 2:53 PM, Barry Warsaw wrote: It might be amazing, but it's still a significant overhead. As I've described, multiply that by all the py files in all the distro packages containing Python source code, and then still try to fit it on a CDROM. I decided to prove to myself that it

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
I'm trying to catch up on this thread, so I may collapse some responses or refer to points others have brought up. On Jun 24, 2010, at 05:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 26, 2010, at 10:22 PM, Matthias Klose wrote: On 24.06.2010 22:46, Barry Warsaw wrote: So, we could say that PEP 384 compliant extension modules would get written without a version specifier. IOW, we'd treat foo.so as using the ABI. It would then be up to the Python runtime to throw

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 04:53 AM, Scott Dial wrote: My suggestion was that a package that contains .so files should not be shared (e.g., the entire lxml package should be placed in a version-specific path). Matthias outlined some of the pitfalls with this approach. The motivation for this PEP was

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 26, 2010, at 06:50 PM, Scott Dial wrote: On 6/26/2010 4:06 PM, Matthias Klose wrote: On 25.06.2010 22:12, James Y Knight wrote: On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: Placing .so files together does not simplify that install process in any way. You will still have to handle

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 11:58 AM, Brett Cannon wrote: Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. You must still compile the package multiple times for each relevant version of python (with special

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 03:42 PM, Scott Dial wrote: On 6/25/2010 2:58 PM, Brett Cannon wrote: I assume you are talking about PEP 3147. You're right that the PEP was for pyc files and that's it. No one is talking about rewriting the PEP. Yes, I am making reference to PEP 3147. I make reference to

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the version number

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread M.-A. Lemburg
Barry Warsaw wrote: On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? If you want to make it so a system can install a package in just one location to be used by multiple Python installations,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote: The Python default installation dir for libs (including site-packages) is $prefix/lib/pythonX.X, so you already have separate and properly versioned directory paths. What difference would the extra version on the .so file make in such a setup ?

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 25.06.2010 22:12, James Y Knight wrote: On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 26.06.2010 02:19, Nick Coghlan wrote: On Sat, Jun 26, 2010 at 6:12 AM, James Y Knightf...@fuhm.net wrote: However, then you have to also consider python packages made up of multiple distro packages -- like twisted or zope. Twisted includes some C extensions in the core package. But then

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 24.06.2010 22:46, Barry Warsaw wrote: On Jun 24, 2010, at 02:28 PM, Barry Warsaw wrote: On Jun 24, 2010, at 01:00 PM, Benjamin Peterson wrote: 2010/6/24 Barry Warsawba...@python.org: On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: 2010/6/24 Barry Warsawba...@python.org: Please

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 25.06.2010 20:58, Brett Cannon wrote: On Fri, Jun 25, 2010 at 01:53, Scott Dial Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. You must still compile the package multiple times for each relevant

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Scott Dial
On 6/26/2010 4:06 PM, Matthias Klose wrote: On 25.06.2010 22:12, James Y Knight wrote: On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. This is a good

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location (since it can't actually be shared) Because python

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/24/2010 9:18 PM, Greg Ewing wrote: Scott Dial wrote: But the only motivation for doing this with .pyc files is that the .py files are able to be shared, In an application made up of a mixture of pure Python and extension modules, the .py files are able to be shared too. Seems to me

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Brett Cannon
On Fri, Jun 25, 2010 at 01:53, Scott Dial scott+python-...@scottdial.com wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/25/2010 2:58 PM, Brett Cannon wrote: I assume you are talking about PEP 3147. You're right that the PEP was for pyc files and that's it. No one is talking about rewriting the PEP. Yes, I am making reference to PEP 3147. I make reference to that PEP because this change is of the same order

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread James Y Knight
On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Barry Warsaw
On Jun 24, 2010, at 11:37 PM, Éric Araujo wrote: Your plan seems good. Adding keyword arguments should not create compatibility issues, and I suspect the impact on the code of build_ext may be actually quite small. I’ll try to review your patch even though I don’t know C or compiler oddities, but

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Nick Coghlan
On Sat, Jun 26, 2010 at 6:12 AM, James Y Knight f...@fuhm.net wrote: However, then you have to also consider python packages made up of multiple distro packages -- like twisted or zope. Twisted includes some C extensions in the core package. But then there are other twisted modules (installed

[Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
This is a follow up to PEP 3147. That PEP, already implemented in Python 3.2, allows for Python source files from different Python versions to live together in the same directory. It does this by putting a magic tag in the .pyc file name and placing the .pyc file in a __pycache__ directory.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Benjamin Peterson
2010/6/24 Barry Warsaw ba...@python.org: Please let me know what you think.  I'm happy to just commit this to the py3k branch if there are no objections wink.  I don't think a new PEP is in order, but an update to PEP 3147 might make sense. How will this interact with PEP 384 if that is

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Daniel Stutzbach
On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw ba...@python.org wrote: The idea is to put the Python version number in the shared library file name, and extend .so lookup to find these extended file names. So for example, we'd see foo.3.2.so instead, and Python would know how to dynload

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 08:50, Barry Warsaw ba...@python.org wrote: This is a follow up to PEP 3147.  That PEP, already implemented in Python 3.2, allows for Python source files from different Python versions to live together in the same directory.  It does this by putting a magic tag in the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 11:27, Guido van Rossum gu...@python.org wrote: On Thu, Jun 24, 2010 at 10:48 AM, Brett Cannon br...@python.org wrote: On Thu, Jun 24, 2010 at 08:50, Barry Warsaw ba...@python.org wrote: This is a follow up to PEP 3147.  That PEP, already implemented in Python 3.2,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Le 24/06/2010 17:50, Barry Warsaw (FLUFL) a écrit : Other possible approaches: * Extend the distutils API so that the .so file extension can be passed in, instead of being essentially hardcoded to what Python's Makefile contains. Third-party code rely on Distutils internal quirks, so it’s

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Le 24/06/2010 19:48, Brett Cannon a écrit : P.S.: I wish we could drop use of the 'module.so' variant at the same time, for consistency sake and to cut out a stat call, but I know that is asking too much. At least, looking for spam/__init__module.so could be avoided. It seems to me that the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 02:28 PM, Barry Warsaw wrote: On Jun 24, 2010, at 01:00 PM, Benjamin Peterson wrote: 2010/6/24 Barry Warsaw ba...@python.org: On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: 2010/6/24 Barry Warsaw ba...@python.org: Please let me know what you think.  I'm happy to

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 10:48 AM, Brett Cannon wrote: While the idea is fine with me since I won't have any of my directories cluttered with multiple .so files, I would still want to add some moniker showing that the version number represents the interpreter and not the .so file. If I read

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:27 AM, Guido van Rossum wrote: On Thu, Jun 24, 2010 at 10:48 AM, Brett Cannon br...@python.org wrote: While the idea is fine with me since I won't have any of my directories cluttered with multiple .so files, I would still want to add some moniker showing that the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 11:53, Éric Araujo mer...@netwok.org wrote: Le 24/06/2010 19:48, Brett Cannon a écrit : P.S.: I wish we could drop use of the 'module.so' variant at the same time, for consistency sake and to cut out a stat call, but I know that is asking too much. At least, looking

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:05 AM, Daniel Stutzbach wrote: On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw ba...@python.org wrote: The idea is to put the Python version number in the shared library file name, and extend .so lookup to find these extended file names. So for example, we'd see

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Fred Drake
On Thu, Jun 24, 2010 at 4:55 PM, Barry Warsaw ba...@python.org wrote: Which is probably another reason not to use foo.so.X.Y for Python extension modules. Clearly, foo.so.3.2 is the man page for the foo.so.3 system call. The ABI ident definitely has to be elsewhere. -Fred -- Fred L.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 08:50 PM, Éric Araujo wrote: Le 24/06/2010 17:50, Barry Warsaw (FLUFL) a écrit : Other possible approaches: * Extend the distutils API so that the .so file extension can be passed in, instead of being essentially hardcoded to what Python's Makefile contains.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:50 AM, Barry Warsaw wrote: Please let me know what you think. I'm happy to just commit this to the py3k branch if there are no objections wink. I don't think a new PEP is in order, but an update to PEP 3147 might make sense. Thanks for all the quick feedback. I've made

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Your plan seems good. Adding keyword arguments should not create compatibility issues, and I suspect the impact on the code of build_ext may be actually quite small. I’ll try to review your patch even though I don’t know C or compiler oddities, but Tarek will have the best insight and the final

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Scott Dial
On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? Specifically, it's the use case where we (Debian/Ubuntu) plan on installing all Python 3.x packages into /usr/lib/python3/dist-packages. As of PEP 3147, we can do that without collisions on the pyc files, but would

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread M.-A. Lemburg
Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the version number isn't enough. You also need to distinguish debug

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Nick Coghlan
On Fri, Jun 25, 2010 at 1:50 AM, Barry Warsaw ba...@python.org wrote: Please let me know what you think.  I'm happy to just commit this to the py3k branch if there are no objections wink.  I don't think a new PEP is in order, but an update to PEP 3147 might make sense. I like the idea, but I

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread James Y Knight
On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? Specifically, it's the use case where we (Debian/Ubuntu) plan on installing all Python 3.x packages into /usr/lib/python3/dist-packages. As of PEP 3147, we can do

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Greg Ewing
Scott Dial wrote: But the only motivation for doing this with .pyc files is that the .py files are able to be shared, In an application made up of a mixture of pure Python and extension modules, the .py files are able to be shared too. Seems to me that a similar motivation exists here as