Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-12 Thread Martin v. Löwis
Am 07.09.2010 19:46, schrieb M.-A. Lemburg: > "Martin v. Löwis" wrote: >>> -1 on always using wchar_t as well. Python's default is UCS2 and the >>> stable ABI should not change that. >> >> It's not really Python's default. It is what configure.in does by >> default. Python's default, on Linux, is U

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread M.-A. Lemburg
Amaury Forgeot d'Arc wrote: > Hi, > > 2010/9/7 M.-A. Lemburg : >>> Ok. I'm fine with excluding Py_UNICODE from the stable ABI. However, >>> I think in the long run, I guess more support for wchar_t will then >>> be needed in the API, e.g. more convenient argument parsing. >> >> Sure, we could add

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread Amaury Forgeot d'Arc
Hi, 2010/9/7 M.-A. Lemburg : >> Ok. I'm fine with excluding Py_UNICODE from the stable ABI. However, >> I think in the long run, I guess more support for wchar_t will then >> be needed in the API, e.g. more convenient argument parsing. > > Sure, we could add that. Just to be clear: does this mean

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-07 Thread M.-A. Lemburg
"Martin v. Löwis" wrote: >> -1 on always using wchar_t as well. Python's default is UCS2 and the >> stable ABI should not change that. > > It's not really Python's default. It is what configure.in does by > default. Python's default, on Linux, is UCS-4. No, the default is UCS2 on all platforms an

Re: [Python-Dev] versioned .so files for Python 3.2

2010-09-04 Thread Martin v. Löwis
> -1 on always using wchar_t as well. Python's default is UCS2 and the > stable ABI should not change that. It's not really Python's default. It is what configure.in does by default. Python's default, on Linux, is UCS-4. > I also think that this information is not relevant for the stable > ABI: E

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread Barry Warsaw
On Aug 31, 2010, at 11:22 AM, M.-A. Lemburg wrote: >BTW: Wasn't one of the main reasons for having versioned .so files >the idea to be able to have UCS2 and UCS4 versions installed >side-by-side ? Yes. This isn't an issue for PEP 3149 because it adds a flag to the shared library file name for wi

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread M.-A. Lemburg
Ronald Oussoren wrote: > > On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote: >> >> - wide-unicode: this is a tricky one. I'm tempted to say that the >> stable ABI should always use a Py_UNICODE that matches the platform's >> wchar_t. Alternative proposals are welcome. > > Sizeof(wchar_t) is 4 o

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-31 Thread Ronald Oussoren
On 28 Aug, 2010, at 12:29, Martin v. Löwis wrote: > > - wide-unicode: this is a tricky one. I'm tempted to say that the > stable ABI should always use a Py_UNICODE that matches the platform's > wchar_t. Alternative proposals are welcome. Sizeof(wchar_t) is 4 on OSX, but the Apple frameworks us

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-30 Thread Barry Warsaw
On Aug 28, 2010, at 12:29 PM, Martin v. Löwis wrote: >The intention is that there is indeed just one stable ABI, so one >configuration is the supported one, and that should be the "default" >build. > >As for the specific settings, my analysis would be this: >- pydebug: not supported by the stable

Re: [Python-Dev] versioned .so files for Python 3.2

2010-08-28 Thread Martin v. Löwis
>> This leads me to a question: how do these configure options affect the >> PEP 384 stable ABI? That PEP is currently silent on the issue, while >> PEP 3149 appears to implicitly assume that "abi3" completely specifies >> the ABI. > > It's a great question - perhaps Martin can chime in? It may b

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Barry Warsaw
On Jul 27, 2010, at 01:54 PM, Ralf Schmitt wrote: >Matthias Klose writes: > >> Not true. Package managers like dpkg/apt-get, rpm/yum and maybe >> others do this for ages. And yes, the added "complexity" of package >> managers does lead to increased robustness. > >but how does sharing things lead

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Barry Warsaw
On Jul 26, 2010, at 10:53 PM, Ralf Schmitt wrote: >Some of the things that need to be adapted are e.g. Makefiles >(basically anything that assumes modules have a certain name), all of >the freezers (cxFreeze, py2exe, ...). The biggest problem probably >will be that an import will load the wrong mo

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Ralf Schmitt
Matthias Klose writes: > Not true. Package managers like dpkg/apt-get, rpm/yum and maybe others > do this for ages. And yes, the added "complexity" of package managers > does lead to increased robustness. but how does sharing things lead to increased robustness (even if it might be managed by yo

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-27 Thread Matthias Klose
On 26.07.2010 22:53, Ralf Schmitt wrote: Barry Warsaw writes: That's fine, but it's not the way Debian/Ubuntu works today. PEP 3149 adoption will definitely remove significant complication for deploying multiple Python versions at the same time on those systems. You're just moving that compl

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Ralf Schmitt
Barry Warsaw writes: > > Do you have concrete examples? Without that it's just speculation I can't do > much to address. Is the problem big or small? Easy to work around or not? Some of the things that need to be adapted are e.g. Makefiles (basically anything that assumes modules have a certa

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Barry Warsaw
On Jul 24, 2010, at 11:59 PM, sch...@gmail.com wrote: >Barry Warsaw writes: > >> On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: >> >>>Doesn't anybody else think this is lost work for very little gain? My >>>/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >>>couldn't ca

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-26 Thread Barry Warsaw
On Jul 24, 2010, at 09:54 AM, Ronald Oussoren wrote: >> I'd be okay including a configure option to allow you to add >> whatever you want after the implementation, version, and flags. >> E.g. something like: >> >>./configure --with-abi-tag-extension=linux2 >> >> would lead to foo.cpython-32m

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-24 Thread schmir
Barry Warsaw writes: > On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: > >>Doesn't anybody else think this is lost work for very little gain? My >>/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >>couldn't care less if my /usr/lib/python2.5/site-packages consumed the >>

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-24 Thread Ronald Oussoren
On 23 Jul, 2010, at 23:19, Barry Warsaw wrote: > On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: > >> On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: >> I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 08:56 PM, Nick Coghlan wrote: >On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw >wrote: >> Python implementations *MAY* include additional flags in the file >> name tag as appropriate.  For example, on POSIX systems these flags >> will also contribute to the file name: >> >>  *

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 01:46 PM, sch...@gmail.com wrote: >Doesn't anybody else think this is lost work for very little gain? My >/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >couldn't care less if my /usr/lib/python2.5/site-packages consumed the >same amount of disk space...

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: >On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: > >>> I'd be open to adding the >>> platform name to the tag, but I'd probably define it as part of the >>> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >>> start with the pl

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread schmir
Ronald Oussoren writes: > On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: > >> >> What about the architecture (i386, amd64)? With every increase in length I >> start to get more concerned. We could encode the platform and architecture, >> but that gets into cryptic territory. OTOH, would you r

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: > On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: > >>> I'd be open to adding the >>> platform name to the tag, but I'd probably define it as part of the >>> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >>> start with the pla

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Nick Coghlan
On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw wrote: > Python implementations *MAY* include additional flags in the file name > tag as appropriate.  For example, on POSIX systems these flags will > also contribute to the file name: > >  * ``--with-pydebug`` (flag: ``d``) >  * ``--with-pymalloc``

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: >> I'd be open to adding the >> platform name to the tag, but I'd probably define it as part of the >> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >> start with the platform name, e.g. foo.linux2-cpython-32m. This >> isn't

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 23 Jul, 2010, at 11:02, Barry Warsaw wrote: > On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: > >> I guess this is not an explicit goal of this PEP, but the structure is >> very close to supporting multiple system architectures at the same >> time. I regularly develop code that needs to

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Barry Warsaw
On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: >I guess this is not an explicit goal of this PEP, but the structure is >very close to supporting multiple system architectures at the same >time. I regularly develop code that needs to run on Windows, Linux >and OSX and it is very convenient t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-23 Thread Ronald Oussoren
On 22 Jul, 2010, at 15:40, Barry Warsaw wrote: > > > Abstract > > > PEP 3147 [1]_ described an extension to Python's import machinery that > improved the sharing of Python source code, by allowing more than one > byte compilation file (.pyc) to be co-located with each source file. > >

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-22 Thread Barry Warsaw
On Jul 16, 2010, at 12:40 PM, Matthias Klose wrote: >I like the proposal, but IMO it is too unspecific about the abi tag. >Assume that an extension is built with such a configured python and >then tried to run with an abi compatible configured python, but with a >slightly different version tag, th

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-21 Thread Barry Warsaw
On Jul 16, 2010, at 12:16 AM, Amaury Forgeot d'Arc wrote: >2010/7/15 Barry Warsaw : >> The first draft of PEP 3149 is ready for review. > >I like it! Cool! >I think it could mention the case where packages are not installed >in the canonical directory, but placed elsewhere along the PYTHONPATH.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Matthias Klose
On 16.07.2010 15:43, Daniel Stutzbach wrote: On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose wrote: 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP should define a non-configurable way for the tag. The tag should include all information which currently makes

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Daniel Stutzbach
On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose wrote: > 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP > should define a non-configurable way for the tag. The tag should > include all information which currently makes an extension ABI > incompatible: >

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-16 Thread Matthias Klose
On 15.07.2010 01:59, Barry Warsaw wrote: PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is passed as the API version if the extension was complied with ``Py_LIMITED_API``. This should be formalized into an official macro called ``PYTHON_ABI_VERSION`` to mirror ``PYTHON_API_VERSI

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-15 Thread Amaury Forgeot d'Arc
Hello, 2010/7/15 Barry Warsaw : > The first draft of PEP 3149 is ready for review. I like it! I think it could mention the case where packages are not installed in the canonical directory, but placed elsewhere along the PYTHONPATH. This is how I deploy applications, for example, and the differen

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-14 Thread Barry Warsaw
On Jun 25, 2010, at 08:35 AM, Nick Coghlan wrote: >I like the idea, but I think summarising the rest of this discussion >in its own (relatively short) PEP would be good (there are a few >things that are tricky - exact versioning scheme, PEP 384 forward >compatibility, impact on distutils, articula

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-09 Thread Barry Warsaw
On Jul 08, 2010, at 01:47 AM, Matthias Klose wrote: >On 07.07.2010 20:40, Barry Warsaw wrote: >> Getting back to this after the US holiday. Thanks for running these >> numbers Scott. I've opened a bug in the Python tracker and attached >> my latest patch: >> >> http://bugs.python.org/issue9193 >

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-09 Thread Barry Warsaw
On Jul 08, 2010, at 09:14 AM, Georg Brandl wrote: >Am 07.07.2010 23:04, schrieb Georg Brandl: >> I can see where this is going... writing it into PEP 384 would >> automatically get the change accepted? I'm definitely not trying to get it in subversively. :) >I hit "Send" prematurely. I wanted t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-08 Thread Georg Brandl
Am 07.07.2010 23:04, schrieb Georg Brandl: > Am 07.07.2010 20:40, schrieb Barry Warsaw: > >> Getting back to this after the US holiday. Thanks for running these numbers >> Scott. I've opened a bug in the Python tracker and attached my latest patch: >> >> http://bugs.python.org/issue9193 >> >>

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Matthias Klose
On 07.07.2010 20:40, Barry Warsaw wrote: Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one difference from previous versions of the patch is that the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Nick Coghlan
On Thu, Jul 8, 2010 at 4:40 AM, Barry Warsaw wrote: > I'd like to get consensus as to whether folks feel that a PEP is needed.  My > own thought is that I'd rather not do a PEP specific to this change, but I > would update PEP 384 with the implications on .so versioning.  Please also > feel free t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread John Arbash Meinel
Scott Dial wrote: > On 6/30/2010 2:53 PM, Barry Warsaw wrote: >> It might be amazing, but it's still a significant overhead. As I've >> described, multiply that by all the py files in all the distro packages >> containing Python source code, and then still try to fit it on a CDROM. > > I decided

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Georg Brandl
Am 07.07.2010 20:40, schrieb Barry Warsaw: > Getting back to this after the US holiday. Thanks for running these numbers > Scott. I've opened a bug in the Python tracker and attached my latest patch: > > http://bugs.python.org/issue9193 > > The one difference from previous versions of the patc

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-07 Thread Barry Warsaw
On Jul 01, 2010, at 07:02 AM, Scott Dial wrote: >I decided to prove to myself that it was not a significant issue to >have parallel directory structures in a .tar.bz2, and I was surprised >to find it much worse at that then I had imagined. For example, > ># cd /usr/lib/python2.6/site-packages ># t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-07-01 Thread Scott Dial
On 6/30/2010 2:53 PM, Barry Warsaw wrote: > It might be amazing, but it's still a significant overhead. As I've > described, multiply that by all the py files in all the distro packages > containing Python source code, and then still try to fit it on a CDROM. I decided to prove to myself that it

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote: >The Python default installation dir for >libs (including site-packages) is $prefix/lib/pythonX.X, so you >already have separate and properly versioned directory paths. > >What difference would the extra version on the .so file make in >such a set

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread M.-A. Lemburg
Barry Warsaw wrote: > On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: > >> Scott Dial wrote: >>> On 6/24/2010 5:09 PM, Barry Warsaw wrote: > What use case does this address? > If you want to make it so a system can install a package in just one > location to be used by multiple

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 12:35 AM, M.-A. Lemburg wrote: >Scott Dial wrote: >> On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? >>> If you want to make it so a system can install a package in just one location to be used by multiple Python installations, then the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 03:42 PM, Scott Dial wrote: >On 6/25/2010 2:58 PM, Brett Cannon wrote: >> I assume you are talking about PEP 3147. You're right that the PEP was >> for pyc files and that's it. No one is talking about rewriting the >> PEP. > >Yes, I am making reference to PEP 3147. I make refer

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 26, 2010, at 10:45 PM, Matthias Klose wrote: >Having non-conflicting extension names is a schema which already is used on >some platforms (debug builds on Windows). The question for me is, if just a >renaming of the .so files is acceptable for upstream, or if distributors >should implement

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 11:58 AM, Brett Cannon wrote: >> Placing .so files together does not simplify that install process in any >> way. You will still have to handle such packages in a special way. You >> must still compile the package multiple times for each relevant version >> of python (with spec

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 26, 2010, at 06:50 PM, Scott Dial wrote: >On 6/26/2010 4:06 PM, Matthias Klose wrote: >> On 25.06.2010 22:12, James Y Knight wrote: >>> On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: Placing .so files together does not simplify that install process in any way. You will still have

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 25, 2010, at 04:53 AM, Scott Dial wrote: >My suggestion was that a package that contains .so files should not be >shared (e.g., the entire lxml package should be placed in a >version-specific path). Matthias outlined some of the pitfalls with this approach. >The motivation for this PEP wa

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
On Jun 26, 2010, at 10:22 PM, Matthias Klose wrote: >On 24.06.2010 22:46, Barry Warsaw wrote: >> So, we could say that PEP 384 compliant extension modules would get written >> without a version specifier. IOW, we'd treat foo.so as using the ABI. It >> would then be up to the Python runtime to th

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-30 Thread Barry Warsaw
I'm trying to catch up on this thread, so I may collapse some responses or refer to points others have brought up. On Jun 24, 2010, at 05:53 PM, Scott Dial wrote: >If the package has .so files that aren't compatible with other version >of python, then what is the motivation for placing that in a

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Scott Dial
On 6/26/2010 4:06 PM, Matthias Klose wrote: > On 25.06.2010 22:12, James Y Knight wrote: >> On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: >>> Placing .so files together does not simplify that install process in any >>> way. You will still have to handle such packages in a special way. >> >> This i

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 25.06.2010 20:58, Brett Cannon wrote: On Fri, Jun 25, 2010 at 01:53, Scott Dial Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way. You must still compile the package multiple times for each relevant versi

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 24.06.2010 22:46, Barry Warsaw wrote: On Jun 24, 2010, at 02:28 PM, Barry Warsaw wrote: On Jun 24, 2010, at 01:00 PM, Benjamin Peterson wrote: 2010/6/24 Barry Warsaw: On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: 2010/6/24 Barry Warsaw: Please let me know what you think. I'm

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 26.06.2010 02:19, Nick Coghlan wrote: On Sat, Jun 26, 2010 at 6:12 AM, James Y Knight wrote: However, then you have to also consider python packages made up of multiple distro packages -- like twisted or zope. Twisted includes some C extensions in the core package. But then there are other t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-26 Thread Matthias Klose
On 25.06.2010 22:12, James Y Knight wrote: On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation f

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Nick Coghlan
On Sat, Jun 26, 2010 at 6:12 AM, James Y Knight wrote: > However, then you have to also consider python packages made up of multiple > distro packages -- like twisted or zope. Twisted includes some C extensions > in the core package. But then there are other twisted modules (installed > under a "t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Barry Warsaw
On Jun 24, 2010, at 11:37 PM, Éric Araujo wrote: >Your plan seems good. Adding keyword arguments should not create >compatibility issues, and I suspect the impact on the code of build_ext >may be actually quite small. I’ll try to review your patch even though I >don’t know C or compiler oddities,

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread James Y Knight
On Jun 25, 2010, at 4:53 AM, Scott Dial wrote: On 6/24/2010 8:23 PM, James Y Knight wrote: On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location (sinc

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/25/2010 2:58 PM, Brett Cannon wrote: > I assume you are talking about PEP 3147. You're right that the PEP was > for pyc files and that's it. No one is talking about rewriting the > PEP. Yes, I am making reference to PEP 3147. I make reference to that PEP because this change is of the same ord

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Brett Cannon
On Fri, Jun 25, 2010 at 01:53, Scott Dial wrote: > On 6/24/2010 8:23 PM, James Y Knight wrote: >> On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: >>> If the package has .so files that aren't compatible with other version >>> of python, then what is the motivation for placing that in a shared >>> lo

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/24/2010 9:18 PM, Greg Ewing wrote: > Scott Dial wrote: > >> But the only motivation for doing this with .pyc files is that the .py >> files are able to be shared, > > In an application made up of a mixture of pure Python and > extension modules, the .py files are able to be shared too. > See

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-25 Thread Scott Dial
On 6/24/2010 8:23 PM, James Y Knight wrote: > On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: >> If the package has .so files that aren't compatible with other version >> of python, then what is the motivation for placing that in a shared >> location (since it can't actually be shared) > > Because

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Greg Ewing
Scott Dial wrote: But the only motivation for doing this with .pyc files is that the .py files are able to be shared, In an application made up of a mixture of pure Python and extension modules, the .py files are able to be shared too. Seems to me that a similar motivation exists here as well.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread James Y Knight
On Jun 24, 2010, at 5:53 PM, Scott Dial wrote: On 6/24/2010 5:09 PM, Barry Warsaw wrote: What use case does this address? Specifically, it's the use case where we (Debian/Ubuntu) plan on installing all Python 3.x packages into /usr/lib/python3/dist-packages. As of PEP 3147, we can do th

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Nick Coghlan
On Fri, Jun 25, 2010 at 1:50 AM, Barry Warsaw wrote: > Please let me know what you think.  I'm happy to just commit this to the py3k > branch if there are no objections .  I don't think a new PEP is in > order, but an update to PEP 3147 might make sense. I like the idea, but I think summarising t

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread M.-A. Lemburg
Scott Dial wrote: > On 6/24/2010 5:09 PM, Barry Warsaw wrote: >>> What use case does this address? >> >>> If you want to make it so a system can install a package in just one >>> location to be used by multiple Python installations, then the version >>> number isn't enough. You also need to distin

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Scott Dial
On 6/24/2010 5:09 PM, Barry Warsaw wrote: >> What use case does this address? > > Specifically, it's the use case where we (Debian/Ubuntu) plan on installing > all Python 3.x packages into /usr/lib/python3/dist-packages. As of PEP 3147, > we can do that without collisions on the pyc files, but wo

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Your plan seems good. Adding keyword arguments should not create compatibility issues, and I suspect the impact on the code of build_ext may be actually quite small. I’ll try to review your patch even though I don’t know C or compiler oddities, but Tarek will have the best insight and the final wor

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:50 AM, Barry Warsaw wrote: >Please let me know what you think. I'm happy to just commit this to the py3k >branch if there are no objections . I don't think a new PEP is in >order, but an update to PEP 3147 might make sense. Thanks for all the quick feedback. I've made so

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 08:50 PM, Éric Araujo wrote: >Le 24/06/2010 17:50, Barry Warsaw (FLUFL) a écrit : >> Other possible approaches: >> * Extend the distutils API so that the .so file extension can be passed in, >>instead of being essentially hardcoded to what Python's Makefile contains. > >Th

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Fred Drake
On Thu, Jun 24, 2010 at 4:55 PM, Barry Warsaw wrote: > Which is probably another reason not to use foo.so.X.Y for Python extension > modules. Clearly, foo.so.3.2 is the man page for the foo.so.3 system call. The ABI ident definitely has to be elsewhere. -Fred -- Fred L. Drake, Jr. "A s

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:05 AM, Daniel Stutzbach wrote: >On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw wrote: > >> The idea is to put the Python version number in the shared library file >> name, >> and extend .so lookup to find these extended file names. So for example, >> we'd >> see foo.3.2.so

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 11:53, Éric Araujo wrote: > Le 24/06/2010 19:48, Brett Cannon a écrit : >> P.S.: I wish we could drop use of the 'module.so' variant at the same >> time, for consistency sake and to cut out a stat call, but I know that >> is asking too much. > > At least, looking for spam/_

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 11:27 AM, Guido van Rossum wrote: >On Thu, Jun 24, 2010 at 10:48 AM, Brett Cannon wrote: >> While the idea is fine with me since I won't have any of my >> directories cluttered with multiple .so files, I would still want to >> add some moniker showing that the version number r

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 10:48 AM, Brett Cannon wrote: >While the idea is fine with me since I won't have any of my >directories cluttered with multiple .so files, I would still want to >add some moniker showing that the version number represents the >interpreter and not the .so file. If I read "foo.3.

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 02:28 PM, Barry Warsaw wrote: >On Jun 24, 2010, at 01:00 PM, Benjamin Peterson wrote: > >>2010/6/24 Barry Warsaw : >>> On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: >>> 2010/6/24 Barry Warsaw : > Please let me know what you think.  I'm happy to just commit this

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Le 24/06/2010 19:48, Brett Cannon a écrit : > P.S.: I wish we could drop use of the 'module.so' variant at the same > time, for consistency sake and to cut out a stat call, but I know that > is asking too much. At least, looking for spam/__init__module.so could be avoided. It seems to me that the

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Éric Araujo
Le 24/06/2010 17:50, Barry Warsaw (FLUFL) a écrit : > Other possible approaches: > * Extend the distutils API so that the .so file extension can be passed in, >instead of being essentially hardcoded to what Python's Makefile contains. Third-party code rely on Distutils internal quirks, so it’

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 11:27, Guido van Rossum wrote: > On Thu, Jun 24, 2010 at 10:48 AM, Brett Cannon wrote: >> On Thu, Jun 24, 2010 at 08:50, Barry Warsaw wrote: >>> This is a follow up to PEP 3147.  That PEP, already implemented in Python >>> 3.2, >>> allows for Python source files from dif

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 01:00 PM, Benjamin Peterson wrote: >2010/6/24 Barry Warsaw : >> On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: >> >>>2010/6/24 Barry Warsaw : Please let me know what you think.  I'm happy to just commit this to the py3k branch if there are no objections .  I d

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Guido van Rossum
On Thu, Jun 24, 2010 at 10:48 AM, Brett Cannon wrote: > On Thu, Jun 24, 2010 at 08:50, Barry Warsaw wrote: >> This is a follow up to PEP 3147.  That PEP, already implemented in Python >> 3.2, >> allows for Python source files from different Python versions to live >> together >> in the same dir

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Benjamin Peterson
2010/6/24 Barry Warsaw : > On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: > >>2010/6/24 Barry Warsaw : >>> Please let me know what you think.  I'm happy to just commit this to the >>> py3k branch if there are no objections .  I don't think a new PEP is >>> in order, but an update to PEP 314

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
On Jun 24, 2010, at 10:58 AM, Benjamin Peterson wrote: >2010/6/24 Barry Warsaw : >> Please let me know what you think.  I'm happy to just commit this to the >> py3k branch if there are no objections .  I don't think a new PEP is >> in order, but an update to PEP 3147 might make sense. > >How will

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Brett Cannon
On Thu, Jun 24, 2010 at 08:50, Barry Warsaw wrote: > This is a follow up to PEP 3147.  That PEP, already implemented in Python 3.2, > allows for Python source files from different Python versions to live together > in the same directory.  It does this by putting a magic tag in the .pyc file > name

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Daniel Stutzbach
On Thu, Jun 24, 2010 at 10:50 AM, Barry Warsaw wrote: > The idea is to put the Python version number in the shared library file > name, > and extend .so lookup to find these extended file names. So for example, > we'd > see foo.3.2.so instead, and Python would know how to dynload both that and >

Re: [Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Benjamin Peterson
2010/6/24 Barry Warsaw : > Please let me know what you think.  I'm happy to just commit this to the py3k > branch if there are no objections .  I don't think a new PEP is in > order, but an update to PEP 3147 might make sense. How will this interact with PEP 384 if that is implemented? -- Rega

[Python-Dev] versioned .so files for Python 3.2

2010-06-24 Thread Barry Warsaw
This is a follow up to PEP 3147. That PEP, already implemented in Python 3.2, allows for Python source files from different Python versions to live together in the same directory. It does this by putting a magic tag in the .pyc file name and placing the .pyc file in a __pycache__ directory. Dist