Re: [Python-Dev] PEP 384 status
On Mon, Aug 30, 2010 at 7:43 AM, Antoine Pitrou wrote: > On Mon, 30 Aug 2010 07:31:34 +1000 > Nick Coghlan wrote: >> Since part of the point of >> PEP 384 is to support multiple versions of the C runtime in a single >> process, [...] > > I think that's quite a maximalist goal. The point of PEP 384 should be > to define a standard API for Python, (hopefully) spanning multiple > versions. Whether the API effectively guarantees a standard ABI can > only depend on whether the system itself hasn't changed its own > conventions (including, for example, function call conventions, or the > binary representation of standard C types). FILE* is very different from the other things you mention. Function call conventions and binary representations are defined in the C standard. FILE*, on the other hand, is explicitly called out as an opaque reference completely under the control of the specific C runtime implementation. Since the Linkage section of PEP 384 specifically states the availability of a generic "python3.dll" that dynamically redirects to the relevant "python3y.dll" to allow an extension module to run against any 3.2 or later Python version as a goal of the PEP, I would say that allowing mixing of C runtimes is definitely one of the PEP's goals. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
Le lundi 30 août 2010 à 22:18 +1000, Nick Coghlan a écrit : > > FILE* is very different from the other things you mention. Function > call conventions and binary representations are defined in the C > standard. FILE*, on the other hand, is explicitly called out as an > opaque reference completely under the control of the specific C > runtime implementation. > > Since the Linkage section of PEP 384 specifically states the > availability of a generic "python3.dll" that dynamically redirects to > the relevant "python3y.dll" to allow an extension module to run > against any 3.2 or later Python version as a goal of the PEP, I would > say that allowing mixing of C runtimes is definitely one of the PEP's > goals. Well, allowing it is one thing. Guaranteeing that it will always work is another one. That said, I have nothing against Python providing such a guarantee, as long as it doesn't complicate the work of developers on other platforms who don't care about Windows. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Aug 30, 2010, at 07:31 AM, Nick Coghlan wrote: >Since part of the point of PEP 384 is to support multiple versions of the C >runtime in a single process Is it? That's certainly not explicit in the Rationale section of PEP 384. It seems to me that the PEP is all about inoculating extension builders from changes in Python's ABI across Python versions. That's a worthy goal. I'm not sure the same can be said about allowing multiple versions of the C runtime in a linked process. But then I'm developing on Windows these days, and I think for all practical purposes, it's a moot point on *nix. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote: >Since the Linkage section of PEP 384 specifically states the >availability of a generic "python3.dll" that dynamically redirects to >the relevant "python3y.dll" to allow an extension module to run >against any 3.2 or later Python version as a goal of the PEP, I would >say that allowing mixing of C runtimes is definitely one of the PEP's >goals. It should be explicit about that then, and provide detail about why the runtime is relevant to Windows programmers (and probably not relevant in practice for *nix programmers). -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On 30/08/2010 17:35, Barry Warsaw wrote: On Aug 30, 2010, at 10:18 PM, Nick Coghlan wrote: Since the Linkage section of PEP 384 specifically states the availability of a generic "python3.dll" that dynamically redirects to the relevant "python3y.dll" to allow an extension module to run against any 3.2 or later Python version as a goal of the PEP, I would say that allowing mixing of C runtimes is definitely one of the PEP's goals. It should be explicit about that then, and provide detail about why the runtime is relevant to Windows programmers (and probably not relevant in practice for *nix programmers). An extension compiled for one version of Python will be linked against the version of the C runtime used by that version of Python (if it is compiled with the same version of Visual Studio of course). If the extension binary is to remain compatible with a later version of Python, compiled against a different version of the C runtime, then it *must* be possible for multiple C runtimes to be loaded. If the stable ABI doesn't allow this then binaries will *still* have to be recompiled when we update the version of Visual Studio used to compile Python - defeating the purpose of the PEP. Right? If this is the case then I agree that it should be explicit in the PEP. All the best, Michael -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Released: Python 2.6.6
On Aug 28, 2010, at 01:12 AM, Georg Brandl wrote: >I don't think I'll want to bother with porting doc fixes to the 2.6 >branch. Thanks for the feedback everyone. We will not be porting doc fixes to release26-maint. I would be open to a doc fix that was specifically addressing a security concern, but we'll handle that the same way we handle all security related fixes to retired branches. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
On Aug 28, 2010, at 12:29 PM, Martin v. Löwis wrote: >The intention is that there is indeed just one stable ABI, so one >configuration is the supported one, and that should be the "default" >build. > >As for the specific settings, my analysis would be this: >- pydebug: not supported by the stable ABI, as it changes the layout > of PyObject, which is an exposed structure > More specifically: Py_DEBUG, Py_TRACEREFS and Py_REF_DEBUG are > all incompatible with the stable ABI >- pymalloc: I fail to see the impact on the ABI. All allocator > macros become function calls under Py_LIMITED_API, otherwise, > there shouldn't be any need to have different versions of that. >- wide-unicode: this is a tricky one. I'm tempted to say that the > stable ABI should always use a Py_UNICODE that matches the platform's > wchar_t. Alternative proposals are welcome. Thanks Martin. I have updated PEP 3149 with these thoughts, but I'll leave it up to you to update PEP 384. I haven't heard a peep since my last RFC on PEP 3149. Guido, would you care to pronounce on the PEP, or designate someone who can do so (remembering that Martin is off-line for a while)? If acceptable, I'd like to get this into the tree before 3.2 alpha 2, currently scheduled for September 5. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Windows buildbots MSVC RTL popups
Since the recent history of my two Windows buildbots has turned ugly, I figured I'd mention that they both (XP and Windows 7) have started generating quite a few GUI C++ RTL runtime pop-up assertions, which has been throwing a wrench into things until they get manually cleared. I first noticed them Sunday night but they were probably there 24-48 hours at that point. These are R6034 "An application has made an attempt to load the C runtime library incorrectly" I glanced through recent commits and I don't think I see anything obviously related but was wondering if anyone who might have been committing changes in the past few days might have an thought? The source filename is truncated in the dialog (what genius at MS thought that would be the one field worth truncating?!) so I am not absolutely sure if they are limited to a specific branch or not. I've thrown on an autoit script to try to automatically acknowledge these dialogs so hopefully the affected test runs will just fail rather than timing out and blocking later runs. -- David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Windows buildbots MSVC RTL popups
With the help of bbreport, I collected these informations: - 3 tests are freezing randomly : test_bz2, test_codecs and test_tarfile - it happens on XP-4 and Windows7 builders (branches 2.7 and 3.1 only, trunk is not affected) - it seems to happen since revisions 84345 and 84346 which fixed issue # 1868 my .2 cents, -- Florent Xicluna 2010/8/30 David Bolen : > Since the recent history of my two Windows buildbots has turned ugly, > I figured I'd mention that they both (XP and Windows 7) have started > generating quite a few GUI C++ RTL runtime pop-up assertions, which > has been throwing a wrench into things until they get manually > cleared. I first noticed them Sunday night but they were probably > there 24-48 hours at that point. These are R6034 "An application has > made an attempt to load the C runtime library incorrectly" > > I glanced through recent commits and I don't think I see anything > obviously related but was wondering if anyone who might have been > committing changes in the past few days might have an thought? The > source filename is truncated in the dialog (what genius at MS thought > that would be the one field worth truncating?!) so I am not absolutely > sure if they are limited to a specific branch or not. > > I've thrown on an autoit script to try to automatically acknowledge > these dialogs so hopefully the affected test runs will just fail > rather than timing out and blocking later runs. > > -- David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Tue, Aug 31, 2010 at 12:30 AM, Barry Warsaw wrote: > On Aug 30, 2010, at 07:31 AM, Nick Coghlan wrote: > >>Since part of the point of PEP 384 is to support multiple versions of the C >>runtime in a single process > > Is it? That's certainly not explicit in the Rationale section of PEP 384. > > It seems to me that the PEP is all about inoculating extension builders from > changes in Python's ABI across Python versions. That's a worthy goal. I'm > not sure the same can be said about allowing multiple versions of the C > runtime in a linked process. But then I'm developing on Windows these days, > and I think for all practical purposes, it's a moot point on *nix. Yeah, I found the Rationale section a little lacking in that regard as well. It was more of a description of the status quo rather than an attempt to justify the proposed changes. The actual proposal and benefits were down in the Linkage section. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Tue, Aug 31, 2010 at 12:47 AM, Michael Foord wrote: > An extension compiled for one version of Python will be linked against the > version of the C runtime used by that version of Python (if it is compiled > with the same version of Visual Studio of course). > > If the extension binary is to remain compatible with a later version of > Python, compiled against a different version of the C runtime, then it > *must* be possible for multiple C runtimes to be loaded. If the stable ABI > doesn't allow this then binaries will *still* have to be recompiled when we > update the version of Visual Studio used to compile Python - defeating the > purpose of the PEP. Right? > > If this is the case then I agree that it should be explicit in the PEP. Ah, I knew it was explicit in the PEP somewhere. The following is currently mentioned in the "Excluded Functions" section: "In addition, functions expecting FILE* are not part of the ABI, to avoid depending on a specific version of the Microsoft C runtime DLL on Windows." To make that explicit in the Rationale section, I would append a second sentence to the final paragraph of that section (the first sentence is already there): "With this PEP, it will be possible to reduce the dependency of binary extension modules on a specific Python feature release, and applications embedding Python can be made work with different releases. To ensure this is also achieved on Windows, the ABI will be designed to allow the Python binary and extension modules or an embedding application to depend on different versions of the C runtime DLL." I would also make the following modification to the above quoted paragraph from the Excluded Functions section of the PEP: "To avoid depending on a specific version of the Microsoft C runtime DLL on Windows while still providing a consistent, cross-platform API when PY_LIMITED_API is defined, the following functions are also excluded: - Any functions accepting or returning FILE* (as this is defined in the C standard as an implementation dependent opaque reference. On Windows, it is known to contain reference to runtime-specific global data) - Any functions reliant on global or thread local state expected to be modified by C standard library calls rather than other Python API calls (e.g. PyErr_FromErrno will be unavailable. Comparable functionality will be provided by a new function PyErr_FromErrnoEx, with the latter accepting an explicit errno parameter) Where practical, equivalent functionality for the excluded functions will instead be provided using preprocessor macros. This ensures that the use of the C runtime dependent types and data remains separate without significantly complicating extension module development and the API remains compatible with any C89 compiler. Note that locale dependent operations will still be available, they just may not see locale changes made via the C standard library calls. Locale changes will need to be made via Python API calls to the locale module to ensure they are seen correctly by the interpreter." Hmm... that last point is a bit of any issue actually, since it also flows the other way (changes made via the locale module won't be visible to any extension modules using a different C runtime). So I suspect mixing C runtimes is still going to come with the caveat of potential locale related glitches. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Tue, Aug 31, 2010 at 6:54 AM, Nick Coghlan wrote: > Hmm... that last point is a bit of any issue actually, since it also > flows the other way (changes made via the locale module won't be > visible to any extension modules using a different C runtime). So I > suspect mixing C runtimes is still going to come with the caveat of > potential locale related glitches. As far as IO is concerned, FILE* is just a special case of a more generic issue, though, so maybe this could be a bit reworded. For example, file descriptor cannot be shared between runtimes either. cheers, David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On 31/08/2010 7:54 AM, Nick Coghlan wrote: On Tue, Aug 31, 2010 at 12:47 AM, Michael Foord wrote: An extension compiled for one version of Python will be linked against the version of the C runtime used by that version of Python (if it is compiled with the same version of Visual Studio of course). If the extension binary is to remain compatible with a later version of Python, compiled against a different version of the C runtime, then it *must* be possible for multiple C runtimes to be loaded. If the stable ABI doesn't allow this then binaries will *still* have to be recompiled when we update the version of Visual Studio used to compile Python - defeating the purpose of the PEP. Right? If this is the case then I agree that it should be explicit in the PEP. Ah, I knew it was explicit in the PEP somewhere. The following is currently mentioned in the "Excluded Functions" section: "In addition, functions expecting FILE* are not part of the ABI, to avoid depending on a specific version of the Microsoft C runtime DLL on Windows." It would be interesting to know how, in practice, these FILE pointers come to life. In my experience they are generally obtained via fopen. If that is broadly true, then a middle-ground may be for Python to expose something like Py_fopen, Py_fclose and a PyFILE opaque "handle". API elements which currently take a FILE * could be exposed using a PyFILE * in the ABI. People who didn't care about this level of portability could continue to use the non-ABI FILE * functions, but people who do could use Py_fopen/Py_fclose in place of fopen/fclose but otherwise work as now. A downside is that as mentioned, in practice these FILE pointers may come from more than simply fopen, so few people would be able to leverage this. It also assumes that people open files before handing them to Python, but otherwise don't use that file - it would be a slippery-slope to wind up with Py_fread etc. Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Internal counter to debug leaking file descriptors
Hi, Is there any kind of internal file descriptor counter that can be queried to debug issues with leaking resources? It can be used in tests to check that all tests are finish with 0 opened descriptors. It will be very useful while porting Python applications from Unix to Windows. Unix is more tolerant to open files and can overwrite them and do other nasty things. See the thread from comment #17 - https://bugs.edge.launchpad.net/dulwich/+bug/557585/ - there is an example of mmap that starts holding file descriptor somewhere long before an error occurs. How could one debug this? Right now I have to use FileMon. It includes information about operated filenames, but no info about source code where this happens. It will be nice to have some kind of counter with filename information inside Python, so that it can be possible to get the full log of events without manually messing with external system-specific tools like FileMon. -- anatoly t. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
