Re: [Python-Dev] PEP 384 status

2010-09-12 Thread Martin v. Löwis
Am 07.09.2010 19:48, schrieb M.-A. Lemburg: Martin v. Löwis wrote: This sounds like the issues such a mix can cause are mostly theoretical and don't really bother much in practice, so PEP 384 on Windows does have a chance :-) Actually, the CRT issues (FILE* in particular) have been causing

Re: [Python-Dev] PEP 384 status

2010-09-12 Thread Martin v. Löwis
On http://bugs.python.org/issue9778 you elaborated on what the PEP would entail in its current state: “No, vice versa. The PEP promises that the ABI won't change until Python 4. For any change that might break the ABI, either a backwards-compatible solution needs to be found, or the change

Re: [Python-Dev] PEP 384 status

2010-09-12 Thread Antoine Pitrou
On Sun, 12 Sep 2010 19:38:33 +0200 Martin v. Löwis mar...@v.loewis.de wrote: On http://bugs.python.org/issue9778 you elaborated on what the PEP would entail in its current state: “No, vice versa. The PEP promises that the ABI won't change until Python 4. For any change that might break

Re: [Python-Dev] PEP 384 status

2010-09-12 Thread Martin v. Löwis
That said, looking at the PEP, I was wondering whether fields such as ob_type, ob_refcnt, ob_size have to be directly accessible, rather than through a macro-turned-into-a-function such as Py_REFCNT(). That they are macros still is primarily for performance reasons. For ob_type, that may be

Re: [Python-Dev] PEP 384 status

2010-09-08 Thread Nick Coghlan
On Wed, Sep 8, 2010 at 8:34 AM, David Cournapeau courn...@gmail.com wrote: I would turn the question around: what are the cases where you manage to mix CRT and not getting any issues ? This has never worked in my own experience in the context of python extensions, I've done it quite a bit over

Re: [Python-Dev] PEP 384 status

2010-09-08 Thread David Cournapeau
On Wed, Sep 8, 2010 at 5:19 PM, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Sep 8, 2010 at 8:34 AM, David Cournapeau courn...@gmail.com wrote: I would turn the question around: what are the cases where you manage to mix CRT and not getting any issues ? This has never worked in my own

Re: [Python-Dev] PEP 384 status

2010-09-08 Thread Nick Coghlan
On Wed, Sep 8, 2010 at 6:34 PM, David Cournapeau courn...@gmail.com wrote: In other words, the problem mainly arises when you need to integrate libraries which you can not recompile with the compiler used by python, because the code is not visual-studio compatible, or because the library is

Re: [Python-Dev] PEP 384 status

2010-09-08 Thread David Cournapeau
On Wed, Sep 8, 2010 at 7:59 PM, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Sep 8, 2010 at 6:34 PM, David Cournapeau courn...@gmail.com wrote: In other words, the problem mainly arises when you need to integrate libraries which you can not recompile with the compiler used by python, because

Re: [Python-Dev] PEP 384 status

2010-09-07 Thread M.-A. Lemburg
Martin v. Löwis wrote: This sounds like the issues such a mix can cause are mostly theoretical and don't really bother much in practice, so PEP 384 on Windows does have a chance :-) Actually, the CRT issues (FILE* in particular) have been causing real crashes for many years, for many

Re: [Python-Dev] PEP 384 status

2010-09-07 Thread David Cournapeau
On Wed, Sep 8, 2010 at 2:48 AM, M.-A. Lemburg m...@egenix.com wrote: Martin v. Löwis wrote: This sounds like the issues such a mix can cause are mostly theoretical and don't really bother much in practice, so PEP 384 on Windows does have a chance :-) Actually, the CRT issues (FILE* in

Re: [Python-Dev] PEP 384 status

2010-09-06 Thread Antoine Pitrou
Hello Martin, On Sat, 28 Aug 2010 12:04:10 +0200 Martin v. Löwis mar...@v.loewis.de wrote: I have now started an initial patch for PEP 384, in the pep-0384 branch. [...] On http://bugs.python.org/issue9778 you elaborated on what the PEP would entail in its current state: “No, vice versa. The

Re: [Python-Dev] PEP 384 status

2010-09-04 Thread Martin v. Löwis
It would be interesting to know how, in practice, these FILE pointers come to life. In my experience they are generally obtained via fopen. I think that experience can't be generalized. I personally guess that in most cases, the FILE* being passed across CRT boundaries is stdout. If that is

Re: [Python-Dev] PEP 384 status

2010-09-04 Thread Martin v. Löwis
This sounds like the issues such a mix can cause are mostly theoretical and don't really bother much in practice, so PEP 384 on Windows does have a chance :-) Actually, the CRT issues (FILE* in particular) have been causing real crashes for many years, for many people. Regards, Martin

Re: [Python-Dev] PEP 384 status

2010-09-04 Thread Martin v. Löwis
What I think would be a mistake would be to define the API in terms of Windows-specific quirks. All this discussion seems bent on satisfying the needs of Windows developers at the expense of non-Windows developers. FILE* is a perfectly standard C feature (and a widely-used one at that). If

Re: [Python-Dev] PEP 384 status

2010-09-04 Thread Martin v. Löwis
Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a different compiler version than Python was compiled with. It's only dangerous *if* you compile with a different compiler. That's why we take serious

Re: [Python-Dev] PEP 384 status

2010-09-04 Thread Antoine Pitrou
On Sat, 04 Sep 2010 15:13:55 +0200 Martin v. Löwis mar...@v.loewis.de wrote: Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a different compiler version than Python was compiled with. It's only

Re: [Python-Dev] PEP 384 status

2010-09-03 Thread Nick Coghlan
On Fri, Sep 3, 2010 at 12:44 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote: On 02/09/10 09:04, Nick Coghlan wrote: I think it would be better if everything dealing with FILE* was a macro rather than a function, yes. How would that help? Macros (and, as Antoine pointed out, inline

Re: [Python-Dev] PEP 384 status

2010-09-02 Thread Nick Coghlan
On Wed, Sep 1, 2010 at 10:54 PM, Antoine Pitrou solip...@pitrou.net wrote: Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a different compiler version than Python was compiled with. So, if we were following

Re: [Python-Dev] PEP 384 status

2010-09-02 Thread Greg Ewing
On 02/09/10 09:04, Nick Coghlan wrote: I think it would be better if everything dealing with FILE* was a macro rather than a function, yes. How would that help? -- Greg ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Nick Coghlan
On Wed, Sep 1, 2010 at 8:42 AM, Antoine Pitrou solip...@pitrou.net wrote: After all, we don't usually try to workaround platform-specific bugs (not as a low level such as the C API level); at worse, we mention their existence in the docs. You persist in viewing the allowance of multiple C

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Antoine Pitrou
Le mercredi 01 septembre 2010 à 22:43 +1000, Nick Coghlan a écrit : On Wed, Sep 1, 2010 at 8:42 AM, Antoine Pitrou solip...@pitrou.net wrote: After all, we don't usually try to workaround platform-specific bugs (not as a low level such as the C API level); at worse, we mention their

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Michael Foord
On 01/09/2010 13:54, Antoine Pitrou wrote: [snip...] Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a different compiler version than Python was compiled with. So, if we were following you, we should rip out

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Nick Coghlan
On Wed, Sep 1, 2010 at 10:54 PM, Antoine Pitrou solip...@pitrou.net wrote: Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a different compiler version than Python was compiled with. So, if we were following

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Antoine Pitrou
On Thu, 2 Sep 2010 07:04:31 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Wed, Sep 1, 2010 at 10:54 PM, Antoine Pitrou solip...@pitrou.net wrote: Please consider this: even without relying on PEP 384, using FILE* is /already/ dangerous; because you might compile an extension with a

Re: [Python-Dev] PEP 384 status

2010-09-01 Thread Nick Coghlan
On Thu, Sep 2, 2010 at 7:19 AM, Antoine Pitrou solip...@pitrou.net wrote: Maybe I've missed your answer, but what would prevent the inline solution from working? Only PEP 7 (since standard support for inline is a C99 feature) On the other hand, if gcc (including cygwin/mingw) and MSVC support

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Antoine Pitrou
On Tue, 31 Aug 2010 11:12:17 +0900 David Cournapeau courn...@gmail.com wrote: Hmm... that last point is a bit of any issue actually, since it also flows the other way (changes made via the locale module won't be visible to any extension modules using a different C runtime). So I suspect

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread M.-A. Lemburg
Michael Foord wrote: On 30/08/2010 17:35, Barry Warsaw wrote: On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote: Since the Linkage section of PEP 384 specifically states the availability of a generic python3.dll that dynamically redirects to the relevant python3y.dll to allow an extension

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Amaury Forgeot d'Arc
Hi, 2010/8/31 Antoine Pitrou solip...@pitrou.net: David Cournapeau courn...@gmail.com wrote: As far as IO is concerned, FILE* is just a special case of a more generic issue, though, so maybe this could be a bit reworded. For example, file descriptor cannot be shared between runtimes either.

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Antoine Pitrou
So it means that, for example, a FileIO object couldn't be shared between runtimes either? How about a socket object? Do you want to forbid FileIO and socket objects as part of the API? Python objects don't have this concern: all methods of FileIO are implemented in a single file

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Neil Hodgson
M.-A. Lemburg: Is it possible to have multiple versions of the lib C loaded on Windows ? Yes. It is possible not only to mix C runtimes from different vendors but different variants from a single vendor. Historically, each vendor has shipped their own C runtime libraries. This was also

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Daniel Stutzbach
On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg m...@egenix.com wrote: Is it possible to have multiple versions of the lib C loaded on Windows ? Yes, and it's a pretty common situation. The fopen() that I call within a DLL may not be the same fopen() called by another DLL. When writing a

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread M.-A. Lemburg
Daniel Stutzbach wrote: On Tue, Aug 31, 2010 at 4:54 AM, M.-A. Lemburg m...@egenix.com wrote: Is it possible to have multiple versions of the lib C loaded on Windows ? Yes, and it's a pretty common situation. The fopen() that I call within a DLL may not be the same fopen() called by

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread M.-A. Lemburg
Neil Hodgson wrote: M.-A. Lemburg: Is it possible to have multiple versions of the lib C loaded on Windows ? Yes. It is possible not only to mix C runtimes from different vendors but different variants from a single vendor. Historically, each vendor has shipped their own C

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Greg Ewing
Daniel Stutzbach wrote: Likewise, a FILE * isn't safe to pass around, unless I can guarantee that the application really is one big happy family compiled against the same version of the C library. Given that, it seems to me that it's a mistake for Python to provide any APIs that take a FILE*

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Nick Coghlan
On Wed, Sep 1, 2010 at 7:49 AM, M.-A. Lemburg m...@egenix.com wrote: Yes, and it's a pretty common situation.   The fopen() that I call within a DLL may not be the same fopen() called by another DLL.  When writing a DLL for Windows, the API must be designed with the assumption that anything

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Nick Coghlan
On Tue, Aug 31, 2010 at 2:15 PM, Mark Hammond skippy.hamm...@gmail.com wrote: It would be interesting to know how, in practice, these FILE pointers come to life.  In my experience they are generally obtained via fopen. If that is broadly true, then a middle-ground may be for Python to expose

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Antoine Pitrou
On Wed, 01 Sep 2010 10:23:42 +1200 Greg Ewing greg.ew...@canterbury.ac.nz wrote: Daniel Stutzbach wrote: Likewise, a FILE * isn't safe to pass around, unless I can guarantee that the application really is one big happy family compiled against the same version of the C library. Given

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Greg Ewing
M.-A. Lemburg wrote: But isn't exactly that a major problem for Python ? An extension written for a different MS CRT version would not be able to safely free a buffer allocated by the Python DLL. Python provides its own set of memory alloc/free functions to deal with this exact problem. For

Re: [Python-Dev] PEP 384 status

2010-08-31 Thread Adal Chiriliuc
On Wed, Sep 1, 2010 at 1:42 AM, Antoine Pitrou solip...@pitrou.net wrote: What I think would be a mistake would be to define the API in terms of Windows-specific quirks. All this discussion seems bent on satisfying the needs of Windows developers at the expense of non-Windows developers. FILE*

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Nick Coghlan
On Mon, Aug 30, 2010 at 7:43 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 30 Aug 2010 07:31:34 +1000 Nick Coghlan ncogh...@gmail.com wrote: Since part of the point of PEP 384 is to support multiple versions of the C runtime in a single process, [...] I think that's quite a

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Antoine Pitrou
Le lundi 30 août 2010 à 22:18 +1000, Nick Coghlan a écrit : FILE* is very different from the other things you mention. Function call conventions and binary representations are defined in the C standard. FILE*, on the other hand, is explicitly called out as an opaque reference completely

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Barry Warsaw
On Aug 30, 2010, at 07:31 AM, Nick Coghlan wrote: Since part of the point of PEP 384 is to support multiple versions of the C runtime in a single process Is it? That's certainly not explicit in the Rationale section of PEP 384. It seems to me that the PEP is all about inoculating extension

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Barry Warsaw
On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote: Since the Linkage section of PEP 384 specifically states the availability of a generic python3.dll that dynamically redirects to the relevant python3y.dll to allow an extension module to run against any 3.2 or later Python version as a goal of the

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Michael Foord
On 30/08/2010 17:35, Barry Warsaw wrote: On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote: Since the Linkage section of PEP 384 specifically states the availability of a generic python3.dll that dynamically redirects to the relevant python3y.dll to allow an extension module to run against any

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Nick Coghlan
On Tue, Aug 31, 2010 at 12:30 AM, Barry Warsaw ba...@python.org wrote: On Aug 30, 2010, at 07:31 AM, Nick Coghlan wrote: Since part of the point of PEP 384 is to support multiple versions of the C runtime in a single process Is it?  That's certainly not explicit in the Rationale section of PEP

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Nick Coghlan
On Tue, Aug 31, 2010 at 12:47 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: An extension compiled for one version of Python will be linked against the version of the C runtime used by that version of Python (if it is compiled with the same version of Visual Studio of course). If the

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread David Cournapeau
On Tue, Aug 31, 2010 at 6:54 AM, Nick Coghlan ncogh...@gmail.com wrote: Hmm... that last point is a bit of any issue actually, since it also flows the other way (changes made via the locale module won't be visible to any extension modules using a different C runtime). So I suspect mixing C

Re: [Python-Dev] PEP 384 status

2010-08-30 Thread Mark Hammond
On 31/08/2010 7:54 AM, Nick Coghlan wrote: On Tue, Aug 31, 2010 at 12:47 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: An extension compiled for one version of Python will be linked against the version of the C runtime used by that version of Python (if it is compiled with the same

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Antoine Pitrou
On Sun, 29 Aug 2010 09:20:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Four options come to mind: - just leave it out of the limited API, extensions can do their own thing to print objects - leave PyObject_Print out of the limited API, but create a PyObject_PrintEx that takes a Python

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Nick Coghlan
On Sun, Aug 29, 2010 at 6:24 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 29 Aug 2010 09:20:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Four options come to mind: - just leave it out of the limited API, extensions can do their own thing to print objects - leave PyObject_Print

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Antoine Pitrou
On Sun, 29 Aug 2010 18:41:45 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Sun, Aug 29, 2010 at 6:24 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 29 Aug 2010 09:20:56 +1000 Nick Coghlan ncogh...@gmail.com wrote: Four options come to mind: - just leave it out of the limited

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Nick Coghlan
On Sun, Aug 29, 2010 at 7:10 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 29 Aug 2010 18:41:45 +1000 Nick Coghlan ncogh...@gmail.com wrote: I believe both that option, and my third option, would run into trouble due to the potential for errno confusion. I don't understand. What's

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Antoine Pitrou
On Sun, 29 Aug 2010 22:16:57 +1000 Nick Coghlan ncogh...@gmail.com wrote: However, since even platforms other than Windows aren't immune to version upgrades of the standard C runtime, I'm still more comfortable with the idea that the strict ABI should refuse to pass FILE* pointers across

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread James Y Knight
On Aug 29, 2010, at 8:16 AM, Nick Coghlan wrote: However, since even platforms other than Windows aren't immune to version upgrades of the standard C runtime Aren't they? I don't know of any other platform that lets you have two versions of libc linked into a single address space. Linux has

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Nick Coghlan
On Mon, Aug 30, 2010 at 1:43 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 29 Aug 2010 22:16:57 +1000 Nick Coghlan ncogh...@gmail.com wrote: (actually, I'm baffled that Windows has such problems, and I would suggest that it's not Python's job to shield Windows application developers

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread Antoine Pitrou
On Mon, 30 Aug 2010 07:31:34 +1000 Nick Coghlan ncogh...@gmail.com wrote: Since part of the point of PEP 384 is to support multiple versions of the C runtime in a single process, [...] I think that's quite a maximalist goal. The point of PEP 384 should be to define a standard API for Python,

Re: [Python-Dev] PEP 384 status

2010-08-29 Thread David Cournapeau
On Mon, Aug 30, 2010 at 6:43 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 30 Aug 2010 07:31:34 +1000 Nick Coghlan ncogh...@gmail.com wrote: Since part of the point of PEP 384 is to support multiple versions of the C runtime in a single process, [...] I think that's quite a

[Python-Dev] PEP 384 status

2010-08-28 Thread Martin v. Löwis
I have now started an initial patch for PEP 384, in the pep-0384 branch. This has the following features: - modules can be compiled under Py_LIMITED_API - Tools/scripts/abitype.py converts C code containing static PyTypeObject definitions to use the new API for type definitions. The following

Re: [Python-Dev] PEP 384 status

2010-08-28 Thread Benjamin Peterson
2010/8/28 Martin v. Löwis mar...@v.loewis.de: I have now started an initial patch for PEP 384, in the pep-0384 branch. This has the following features: - modules can be compiled under Py_LIMITED_API - Tools/scripts/abitype.py converts C code containing static  PyTypeObject definitions to use

Re: [Python-Dev] PEP 384 status

2010-08-28 Thread Martin v. Löwis
This is from tp_new and tp_dealloc, right? I think we should probably provide assessors PyObject_Alloc and PyObject_FreeObject. Correct, and yes, that sounds like a good approach. - PyObject_Print is used, but can't be supported, as it uses a FILE* parameter I thought tp_print was

Re: [Python-Dev] PEP 384 status

2010-08-28 Thread Nick Coghlan
On Sun, Aug 29, 2010 at 4:52 AM, Martin v. Löwis mar...@v.loewis.de wrote: This is from tp_new and tp_dealloc, right? I think we should probably provide assessors PyObject_Alloc and PyObject_FreeObject. Correct, and yes, that sounds like a good approach. - PyObject_Print is used, but can't