[Python-Dev] lzma and 'x' mode open
I love the 'x' mode open in recent versions of Python. I just discovered that lzma.open doesn't support it. It seems there's an elif that explicitly checks the modes allowed. I added x and xb to the choices and now it seems to work as I would like. patch.lzma.py Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma and 'x' mode open
Please open a bug report on bugs.python.org so this doesn't get lost. -- Eric. On Oct 8, 2013, at 8:49 PM, Tim Heaney thea...@gmail.com wrote: I love the 'x' mode open in recent versions of Python. I just discovered that lzma.open doesn't support it. It seems there's an elif that explicitly checks the modes allowed. I added x and xb to the choices and now it seems to work as I would like. patch.lzma.py ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma and 'x' mode open
On Tue, 08 Oct 2013 20:49:17 -0400, Tim Heaney thea...@gmail.com wrote: I love the 'x' mode open in recent versions of Python. I just discovered that lzma.open doesn't support it. It seems there's an elif that explicitly checks the modes allowed. I added x and xb to the choices and now it seems to work as I would like. Please file an enhancement request at bugs.python.org. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma and 'x' mode open
Done. http://bugs.python.org/issue19201 I guess I should have known that. Sorry to bother python-dev with this. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] lzma
When I build python from sources I have no lzma support (module _lzma cannot be built). There are lzma packages installed in my Ubuntu 11.10 box: lzma, lzma-dev and lzma-sources. I can see lib files (headers are also can be found in linux-headers): andrew@tiktaalik ~/p/cpython locate lzma.so /usr/lib/liblzma.so.2 /usr/lib/liblzma.so.2.0.0 I can live with that but if somebody can point me the way to build python with lzma support please do. -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma
On Fri, Mar 16, 2012 at 07:30:31PM -0700, Andrew Svetlov wrote: When I build python from sources I have no lzma support (module _lzma cannot be built). I have liblzma-dev, liblzma2 and lzma packages installed on ubuntu. I am able to build and import lzma module. Thanks, Senthil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lzma
liblzma-dev has solved my problem. Thank you, Senthil. On Fri, Mar 16, 2012 at 7:38 PM, Senthil Kumaran sent...@uthcode.com wrote: On Fri, Mar 16, 2012 at 07:30:31PM -0700, Andrew Svetlov wrote: When I build python from sources I have no lzma support (module _lzma cannot be built). I have liblzma-dev, liblzma2 and lzma packages installed on ubuntu. I am able to build and import lzma module. Thanks, Senthil -- Thanks, Andrew Svetlov ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LZMA support has landed
Hey folks, I'm pleased to announce that as of changeset 74d182cf0187, the standard library now includes support for the LZMA compression algorithm (as well as the associated .xz and .lzma file formats). The new lzma module has a very similar API to the existing bz2 module; it should serve as a drop-in replacement for most use cases. If anyone has any feedback or any suggestions for improvement, please let me know. I'd like to ask the owners of (non-Windows) buildbots to install the XZ Utils development headers so that they can build the new module. For Debian-derived Linux distros, the relevant package is named liblzma-dev; on Fedora I believe the correct package is xz-devel. Binaries for OS X are available from the project's homepage at http://tukaani.org/xz/. Finally, a big thanks to everyone who contributed feedback during this module's development! Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA support has landed
2011/11/29 Nadeem Vawda nadeem.va...@gmail.com I'm pleased to announce that as of changeset 74d182cf0187, the standard library now includes support for the LZMA compression algorithm Congratulations! I'd like to ask the owners of (non-Windows) buildbots to install the XZ Utils development headers so that they can build the new module. And don't worry about Windows builbots, they will automatically download the XZ prebuilt binaries from the usual place. (svn export http://svn.python.org/projects/external/xz-5.0.3) Next step: add support for tar.xz files (issue5689)... -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA support has landed
Congrats, this is an excellent feature. On Wed, Nov 30, 2011 at 10:34 AM, Amaury Forgeot d'Arc amaur...@gmail.com wrote: 2011/11/29 Nadeem Vawda nadeem.va...@gmail.com I'm pleased to announce that as of changeset 74d182cf0187, the standard library now includes support for the LZMA compression algorithm Congratulations! I'd like to ask the owners of (non-Windows) buildbots to install the XZ Utils development headers so that they can build the new module. And don't worry about Windows builbots, they will automatically download the XZ prebuilt binaries from the usual place. (svn export http://svn.python.org/projects/external/xz-5.0.3) Next step: add support for tar.xz files (issue5689)... -- Amaury Forgeot d'Arc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/anacrolix%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA support has landed
On Tue, Nov 29, 2011 at 4:59 PM, Nadeem Vawda nadeem.va...@gmail.com wrote: liblzma-dev; on Fedora I believe the correct package is xz-devel. xz-devel is right. I just verified a build of the new module on a fresh F16 system. -- Meador ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
Another update - I've added proper documentation. Now the code should be pretty much complete - all that's missing is the necessary bits and pieces to build it on Windows. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
I've posted an updated patch to the bug tracker, with a complete implementation of the lzma module, including 100% test coverage for the LZMAFile class (which is implemented entirely in Python). It doesn't include ReST documentation (yet), but the docstrings are quite detailed. Please take a look and let me know what you think. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Aug 27, 2011, at 10:36 PM, Nadeem Vawda wrote: I talked to Antoine about this on IRC; he didn't seem to think a PEP would be necessary. But a summary of the discussion on the tracker issue might still be a useful thing to have, given how long it's gotten. I agree with Antoine - no PEP should be necessary. A well reviewed and tested module should do it. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
I've updated the issue http://bugs.python.org/issue6715 with a patch containing my work so far - the LZMACompressor and LZMADecompressor classes, along with some tests. These two classes should provide a fairly complete interface to liblzma; it will be possible to implement LZMAFile on top of them, entirely in Python. Note that the C code does no I/O; this will be handled by LZMAFile. Please take a look, and let me know what you think. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
I just want to talk about it - for now. python-ideas is a better place to just talk than python-dev. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
Dan Stromberg, 27.08.2011 21:58: On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan wrote: On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda wrote: On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I'm not too sure about that - PEP 399 explicitly says that using ctypes is frowned upon, and doesn't mention anywhere that it should be used in this sort of situation. Note to self: do not comment on python-dev at 2 am, as one's ability to read PEPs correctly apparently suffers :) Consider my comment withdrawn, you're quite right that PEP 399 actually says this is precisely the case where an exemption is a reasonable idea. Although I believe it's likely that PyPy will wrap it with ctypes anyway :) I'd like to better understand why ctypes is (sometimes) frowned upon. Is it the brittleness? Tendency to segfault? Maybe unwieldy code and slow execution on CPython? Note that there's a ctypes backend for Cython being written as part of a GSoC, so it should eventually become possible to write C library wrappers in Cython and have it generate a ctypes version to run on PyPy. That, together with the IronPython backend that is on its way, would give you a way to write fast wrappers for at least three of the major four Python implementations, without sacrificing readability or speed in one of them. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 10:36 PM, Dan Stromberg drsali...@gmail.com wrote: On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum gu...@python.org wrote: On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg drsali...@gmail.com wrote: IMO, we really, really need some common way of accessing C libraries that works for all major Python variants. We have one. It's called writing an extension module. And yet Cext's are full of CPython-isms. I have to apologize, I somehow misread your all Python variants as a mixture of all CPython versions and all platforms where CPython runs. While I have no desire to continue this discussion, you are most welcome to do so. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
Guido van Rossum wrote: On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg drsali...@gmail.com wrote: IMO, we really, really need some common way of accessing C libraries that works for all major Python variants. We have one. It's called writing an extension module. I think Dan means some way of doing this without having to hand-craft a different one for each Python implementation. If we're really serious about the idea that Python is not CPython, this seems like a reasonable thing to want. Currently the Python universe is very much centred around CPython, with the other implementations perpetually in catch-up mode. My suggestion on how to address this would be something akin to Pyrex or Cython. I gather that there has been some work recently on adding different back-ends to Cython to generate code for different Python implementations. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LZMA compression support in 3.3
Hello all, I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker http://bugs.python.org/issue6715, where there seems to be a consensus that this is a desirable feature. What are your thoughts? The proposed module's API will be very similar to that of the bz2 module; the only differences will be additional keyword arguments to some functions, for specifying container formats and detailed compressor options. The implementation will also be similar to bz2 - basic compressor and decompressor classes written in C, with convenience functions and a file interface implemented on top of those in Python. I've already done some work on the C parts of the module; I'll push that to my sandbox http://hg.python.org/sandbox/nvawda/ in the next day or two. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker http://bugs.python.org/issue6715, where there seems to be a consensus that this is a desirable feature. What are your thoughts? The proposed module's API will be very similar to that of the bz2 module; the only differences will be additional keyword arguments to some functions, for specifying container formats and detailed compressor options. +1 for adding and +1 for keeping a similar interface. Cheers Ross ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
The implementation will also be similar to bz2 - basic compressor and decompressor classes written in C, with convenience functions and a file interface implemented on top of those in Python. When I reviewed lzma, I found that this approach might not be appropriate. lzma has many more options and aspects that allow tuning and selection, and a Python LZMA library should provide the same feature set as the underlying C library. So I would propose that a very thin C layer is created around the C library that focuses on the actual algorithms, and that any higher layers (in particular file formats) are done in Python. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 4:50 PM, Martin v. Löwis mar...@v.loewis.de wrote: The implementation will also be similar to bz2 - basic compressor and decompressor classes written in C, with convenience functions and a file interface implemented on top of those in Python. When I reviewed lzma, I found that this approach might not be appropriate. lzma has many more options and aspects that allow tuning and selection, and a Python LZMA library should provide the same feature set as the underlying C library. So I would propose that a very thin C layer is created around the C library that focuses on the actual algorithms, and that any higher layers (in particular file formats) are done in Python. I probably shouldn't have used the word basic here - these classes expose all the features of the underlying library. I was rather trying to underscore that the rest of the module is implemented in terms of these two classes. As for file formats, these are handled by liblzma itself; the extension module just selects which compressor/decompressor initializer function to use depending on the value of the format argument. Our code won't contain anything along the lines of GzipFile; all of that work is done by the underlying C library. Rather, the LZMAFile class will be like BZ2File - just a simple filter that passes the read/written data through a LZMACompressor or LZMADecompressor as appropriate. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
As for file formats, these are handled by liblzma itself; the extension module just selects which compressor/decompressor initializer function to use depending on the value of the format argument. Our code won't contain anything along the lines of GzipFile; all of that work is done by the underlying C library. Rather, the LZMAFile class will be like BZ2File - just a simple filter that passes the read/written data through a LZMACompressor or LZMADecompressor as appropriate. This is exactly what I worry about. I think adding file I/O to bz2 was a mistake, as this doesn't integrate with Python's IO library (it used to, but now after dropping stdio, they were incompatible. Indeed, for Python 3.2, BZ2File has been removed from the C module, and lifted to Python. IOW, the _lzma C module must not do any I/O, neither directly nor indirectly (through liblzma). The approach of gzip.py (doing IO and file formats in pure Python) is exactly right. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 5:15 PM, Martin v. Löwis mar...@v.loewis.de wrote: As for file formats, these are handled by liblzma itself; the extension module just selects which compressor/decompressor initializer function to use depending on the value of the format argument. Our code won't contain anything along the lines of GzipFile; all of that work is done by the underlying C library. Rather, the LZMAFile class will be like BZ2File - just a simple filter that passes the read/written data through a LZMACompressor or LZMADecompressor as appropriate. This is exactly what I worry about. I think adding file I/O to bz2 was a mistake, as this doesn't integrate with Python's IO library (it used to, but now after dropping stdio, they were incompatible. Indeed, for Python 3.2, BZ2File has been removed from the C module, and lifted to Python. IOW, the _lzma C module must not do any I/O, neither directly nor indirectly (through liblzma). The approach of gzip.py (doing IO and file formats in pure Python) is exactly right. It is not my intention for the _lzma C module to do I/O - that will be done by the LZMAFile class, which will be written in Python. My comparison with bz2 was in reference to the state of the module after it was rewritten for issue 5863. Saying anything along the lines of GzipFile was a bad choice of wording; what I meant is that the LZMAFile class won't handle the problem of picking apart the .xz and .lzma container formats. That is handled by liblzma (operating entirely on in-memory buffers). It will do _only_ I/O, in a similar fashion to the BZ2File class (as of changeset 2cb07a46f4b5, to avoid ambiguity ;) ). Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sun, Aug 28, 2011 at 1:15 AM, Martin v. Löwis mar...@v.loewis.de wrote: This is exactly what I worry about. I think adding file I/O to bz2 was a mistake, as this doesn't integrate with Python's IO library (it used to, but now after dropping stdio, they were incompatible. Indeed, for Python 3.2, BZ2File has been removed from the C module, and lifted to Python. IOW, the _lzma C module must not do any I/O, neither directly nor indirectly (through liblzma). The approach of gzip.py (doing IO and file formats in pure Python) is exactly right. PEP 399 also comes into play - we need a pure Python version for PyPy et al (or a plausible story for why an exception should be granted). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
It is not my intention for the _lzma C module to do I/O - that will be done by the LZMAFile class, which will be written in Python. My comparison with bz2 was in reference to the state of the module after it was rewritten for issue 5863. Ok. I'll defer my judgement then until actual code is to review. Not sure whether you already have this: supporting the tarfile module would be nice. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sun, 28 Aug 2011 01:36:50 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Sun, Aug 28, 2011 at 1:15 AM, Martin v. Löwis mar...@v.loewis.de wrote: This is exactly what I worry about. I think adding file I/O to bz2 was a mistake, as this doesn't integrate with Python's IO library (it used to, but now after dropping stdio, they were incompatible. Indeed, for Python 3.2, BZ2File has been removed from the C module, and lifted to Python. IOW, the _lzma C module must not do any I/O, neither directly nor indirectly (through liblzma). The approach of gzip.py (doing IO and file formats in pure Python) is exactly right. PEP 399 also comes into play - we need a pure Python version for PyPy et al (or a plausible story for why an exception should be granted). The plausible story being that we basically wrap an existing library? I don't think PyPy et al have pure Python versions of the zlib or OpenSSL, do they? If we start taking PEP 399 conformance to such levels, we might as well stop developing CPython. cheers Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sun, Aug 28, 2011 at 1:40 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 28 Aug 2011 01:36:50 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Sun, Aug 28, 2011 at 1:15 AM, Martin v. Löwis mar...@v.loewis.de wrote: This is exactly what I worry about. I think adding file I/O to bz2 was a mistake, as this doesn't integrate with Python's IO library (it used to, but now after dropping stdio, they were incompatible. Indeed, for Python 3.2, BZ2File has been removed from the C module, and lifted to Python. IOW, the _lzma C module must not do any I/O, neither directly nor indirectly (through liblzma). The approach of gzip.py (doing IO and file formats in pure Python) is exactly right. PEP 399 also comes into play - we need a pure Python version for PyPy et al (or a plausible story for why an exception should be granted). The plausible story being that we basically wrap an existing library? I don't think PyPy et al have pure Python versions of the zlib or OpenSSL, do they? If we start taking PEP 399 conformance to such levels, we might as well stop developing CPython. It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 5:42 PM, Martin v. Löwis mar...@v.loewis.de wrote: Not sure whether you already have this: supporting the tarfile module would be nice. Yes, got that - issue 5689. Also of interest is issue 5411 - adding .xz support to distutils. But I think that these are separate projects that should wait until the lzma module is finalized. On Sat, Aug 27, 2011 at 5:40 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 28 Aug 2011 01:36:50 +1000 Nick Coghlan ncogh...@gmail.com wrote: PEP 399 also comes into play - we need a pure Python version for PyPy et al (or a plausible story for why an exception should be granted). The plausible story being that we basically wrap an existing library? I don't think PyPy et al have pure Python versions of the zlib or OpenSSL, do they? If we start taking PEP 399 conformance to such levels, we might as well stop developing CPython. Indeed, PEP 399 specifically notes that exemptions can be granted for modules that wrap external C libraries. On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan ncogh...@gmail.com wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I'm not too sure about that - PEP 399 explicitly says that using ctypes is frowned upon, and doesn't mention anywhere that it should be used in this sort of situation. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda nadeem.va...@gmail.com wrote: On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan ncogh...@gmail.com wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I'm not too sure about that - PEP 399 explicitly says that using ctypes is frowned upon, and doesn't mention anywhere that it should be used in this sort of situation. Note to self: do not comment on python-dev at 2 am, as one's ability to read PEPs correctly apparently suffers :) Consider my comment withdrawn, you're quite right that PEP 399 actually says this is precisely the case where an exemption is a reasonable idea. Although I believe it's likely that PyPy will wrap it with ctypes anyway :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sun, 28 Aug 2011 01:52:51 +1000 Nick Coghlan ncogh...@gmail.com wrote: The plausible story being that we basically wrap an existing library? I don't think PyPy et al have pure Python versions of the zlib or OpenSSL, do they? If we start taking PEP 399 conformance to such levels, we might as well stop developing CPython. It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I think you're taking this too seriously. Our extension modules (_bz2, _ssl...) are *already* optional even on CPython. If the library or its development headers are not available on the system, building these extensions is simply skipped, and the test suite passes nonetheless. The only required libraries for passing the tests being basically the libc and the zlib. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
PEP 399 also comes into play - we need a pure Python version for PyPy et al (or a plausible story for why an exception should be granted). No, we don't. We can grant an exception, which I'm very willing to do. The PEP lists wrapping a specific C-based library as a plausible reason. It's acceptable for the Python version to use ctypes Hmm. To me, *that's* unacceptable. In the specific case, having a pure-Python implementation would be acceptable to me, but I'm skeptical that anybody is willing to produce one. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, 27 Aug 2011 18:50:40 +0200 Antoine Pitrou solip...@pitrou.net wrote: On Sun, 28 Aug 2011 01:52:51 +1000 Nick Coghlan ncogh...@gmail.com wrote: The plausible story being that we basically wrap an existing library? I don't think PyPy et al have pure Python versions of the zlib or OpenSSL, do they? If we start taking PEP 399 conformance to such levels, we might as well stop developing CPython. It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I think you're taking this too seriously. Our extension modules (_bz2, _ssl...) are *already* optional even on CPython. If the library or its development headers are not available on the system, building these extensions is simply skipped, and the test suite passes nonetheless. The only required libraries for passing the tests being basically the libc and the zlib. ...and, apparently, pyexpat... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On 8/27/2011 9:47 AM, Nadeem Vawda wrote: I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the trackerhttp://bugs.python.org/issue6715, where there seems to be a consensus that this is a desirable feature. What are your thoughts? As I read the discussion, the idea has been more or less accepted in principle. However, the current patch is not and needs changes. The proposed module's API will be very similar to that of the bz2 module; the only differences will be additional keyword arguments to some functions, for specifying container formats and detailed compressor options. I believe Antoine suggested a PEP. It should summarize the salient points in the long tracker discussion into a coherent exposition and flesh out the details implied above. (Perhaps they are already in the proposed doc addition.) The implementation will also be similar to bz2 - basic compressor and decompressor classes written in C, with convenience functions and a file interface implemented on top of those in Python. I would follow Martin's suggestions, including doing all i/o with the io module and the following: So I would propose that a very thin C layer is created around the C library that focuses on the actual algorithms, and that any higher layers (in particular file formats) are done in Python. If we minimize the C code we add and maximize what is done in Python, that would maximize the ease of porting to other implementations. This would conform to the spirit of PEP 399. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan ncogh...@gmail.com wrote: On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda nadeem.va...@gmail.com wrote: On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan ncogh...@gmail.com wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but the Python version should still exist. I'm not too sure about that - PEP 399 explicitly says that using ctypes is frowned upon, and doesn't mention anywhere that it should be used in this sort of situation. Note to self: do not comment on python-dev at 2 am, as one's ability to read PEPs correctly apparently suffers :) Consider my comment withdrawn, you're quite right that PEP 399 actually says this is precisely the case where an exemption is a reasonable idea. Although I believe it's likely that PyPy will wrap it with ctypes anyway :) I'd like to better understand why ctypes is (sometimes) frowned upon. Is it the brittleness? Tendency to segfault? If yes, is there a way of making ctypes less brittle - say, by carefully matching it against a specific version of a .so/.dll before starting to make heavy use of said .so/.dll? FWIW, I have a partial implementation of a module that does xz from Python using ctypes. It only does in-memory compression and decompression (not stream compression or decompression to or from a file), because that was all I needed for my current project, but it runs on CPython 2.x, CPython 3.x, and PyPy. I don't think it runs on Jython, but I've not looked at that carefully - my code falls back on subprocess if ctypes doesn't appear to be all there. It's at http://stromberg.dnsalias.org/svn/xz_mod/trunk/xz_mod.py ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
I'd like to better understand why ctypes is (sometimes) frowned upon. Is it the brittleness? Tendency to segfault? That, and Python should work completely if ctypes is not available. FWIW, I have a partial implementation of a module that does xz from Python using ctypes. So does it work on Sparc/Solaris? On OpenBSD? On ARM-Linux? Does it work if the xz library is installed into /opt/sfw/xz? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 9:47 PM, Terry Reedy tjre...@udel.edu wrote: On 8/27/2011 9:47 AM, Nadeem Vawda wrote: I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the trackerhttp://bugs.python.org/issue6715, where there seems to be a consensus that this is a desirable feature. What are your thoughts? As I read the discussion, the idea has been more or less accepted in principle. However, the current patch is not and needs changes. Please note that the code I'm talking about is not the same as the patches by Per Øyvind Karlsen that are attached to the tracker issue. I have been doing a completely new implementation of the module, specifically to address the concerns raised by Martin and Antoine. (As for why I haven't posted my own changes yet - I'm currently an intern at Google, and they want me to run my code by their open-source team before releasing it into the wild. Sorry for the delay and the confusion.) The proposed module's API will be very similar to that of the bz2 module; the only differences will be additional keyword arguments to some functions, for specifying container formats and detailed compressor options. I believe Antoine suggested a PEP. It should summarize the salient points in the long tracker discussion into a coherent exposition and flesh out the details implied above. (Perhaps they are already in the proposed doc addition.) I talked to Antoine about this on IRC; he didn't seem to think a PEP would be necessary. But a summary of the discussion on the tracker issue might still be a useful thing to have, given how long it's gotten. The implementation will also be similar to bz2 - basic compressor and decompressor classes written in C, with convenience functions and a file interface implemented on top of those in Python. I would follow Martin's suggestions, including doing all i/o with the io module and the following: So I would propose that a very thin C layer is created around the C library that focuses on the actual algorithms, and that any higher layers (in particular file formats) are done in Python. If we minimize the C code we add and maximize what is done in Python, that would maximize the ease of porting to other implementations. This would conform to the spirit of PEP 399. As stated in my earlier response to Martin, I intend to do this. Aside from I/O, though, there's not much that _can_ be done in Python - the rest is basically just providing a thin wrapper for the C library. On Sat, Aug 27, 2011 at 9:58 PM, Dan Stromberg drsali...@gmail.com wrote: I'd like to better understand why ctypes is (sometimes) frowned upon. Is it the brittleness? Tendency to segfault? The problem (as I understand it) is that ABI changes in a library will cause code that uses it via ctypes to break without warning. With an extension module, you'll get a compile failure if you rely on things that change in an incompatible way. With a ctypes wrapper, you just get incorrect answers, or segfaults. If yes, is there a way of making ctypes less brittle - say, by carefully matching it against a specific version of a .so/.dll before starting to make heavy use of said .so/.dll? This might be feasible for a specific application running in a controlled environment, but it seems impractical for something as widely-used as the stdlib. Having to include a whitelist of acceptable library versions would be a substantial maintenance burden, and (compatible) new versions would not work until the library whitelist gets updated. Cheers, Nadeem ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 1:21 PM, Martin v. Löwis mar...@v.loewis.dewrote: I'd like to better understand why ctypes is (sometimes) frowned upon. Is it the brittleness? Tendency to segfault? That, and Python should work completely if ctypes is not available. What are the most major platforms ctypes doesn't work on? It seems like there should be some way of coming up with an xml file describing the types of the various bits of data and formal arguments - perhaps using gccxml or something like it. FWIW, I have a partial implementation of a module that does xz from Python using ctypes. So does it work on Sparc/Solaris? On OpenBSD? On ARM-Linux? Does it work if the xz library is installed into /opt/sfw/xz? So far, I've only tried it on a couple of Linuxes and Cygwin. I intend to try it on a large number of *ix variants in the future, including OS/X and Haiku. I doubt I'll test OpenBSD, but I'm likely to test on FreeBSD and Dragonfly again. With regard to /opt/sfw/xz, if ctypes.util.find_library(library) is smart enough to look there, then yes, xz_mod should find libxz there. On Cygwin, ctypes.util.find_library() wasn't smart enough to find a Cygwin DLL, so I coded around that. But it finds the library OK on the Linuxes I've tried so far. (This is part of a larger project, a backup program. The backup program has been tested on a large number of OS's, but I've not done another broad round of testing yet since adding the ctypes+xz code) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg drsali...@gmail.com wrote: It seems like there should be some way of coming up with an xml file describing the types of the various bits of data and formal arguments - perhaps using gccxml or something like it. The problem is that you would need to do this check at runtime, every time you load up the library - otherwise, what happens if the user upgrades their installed copy of liblzma? And we can't expect users to have the liblzma headers installed, so we'd have to try and figure out whether the library was ABI-compatible from the shared object alone; I doubt that this is even possible. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda nadeem.va...@gmail.comwrote: On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg drsali...@gmail.com wrote: It seems like there should be some way of coming up with an xml file describing the types of the various bits of data and formal arguments - perhaps using gccxml or something like it. The problem is that you would need to do this check at runtime, every time you load up the library - otherwise, what happens if the user upgrades their installed copy of liblzma? And we can't expect users to have the liblzma headers installed, so we'd have to try and figure out whether the library was ABI-compatible from the shared object alone; I doubt that this is even possible. I was thinking about this as I was getting groceries a bit ago. Why -can't- we expect the user to have liblzma headers installed? Couldn't it just be a dependency in the package management system? BTW, gcc-xml seems to be only for C++ (?), but long ago, around the time people were switching from KR to Ansi C, there were programs like mkptypes that could parse a .c/.h and output prototypes. It seems we could do something like this on module init. IMO, we really, really need some common way of accessing C libraries that works for all major Python variants. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, 27 Aug 2011 15:14:15 -0700 Dan Stromberg drsali...@gmail.com wrote: On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda nadeem.va...@gmail.comwrote: On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg drsali...@gmail.com wrote: It seems like there should be some way of coming up with an xml file describing the types of the various bits of data and formal arguments - perhaps using gccxml or something like it. The problem is that you would need to do this check at runtime, every time you load up the library - otherwise, what happens if the user upgrades their installed copy of liblzma? And we can't expect users to have the liblzma headers installed, so we'd have to try and figure out whether the library was ABI-compatible from the shared object alone; I doubt that this is even possible. I was thinking about this as I was getting groceries a bit ago. Why -can't- we expect the user to have liblzma headers installed? Couldn't it just be a dependency in the package management system? Package managers, under Linux, often split development files (headers, etc.) from runtime binaries. Also, under Windows, most users don't have development stuff installed at all. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
Why -can't- we expect the user to have liblzma headers installed? Couldn't it just be a dependency in the package management system? Please give it up. You just won't convince that list that ctypes is a viable approach for the standard library. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 3:26 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 27 Aug 2011 15:14:15 -0700 Dan Stromberg drsali...@gmail.com wrote: On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda nadeem.va...@gmail.com wrote: On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg drsali...@gmail.com wrote: It seems like there should be some way of coming up with an xml file describing the types of the various bits of data and formal arguments - perhaps using gccxml or something like it. The problem is that you would need to do this check at runtime, every time you load up the library - otherwise, what happens if the user upgrades their installed copy of liblzma? And we can't expect users to have the liblzma headers installed, so we'd have to try and figure out whether the library was ABI-compatible from the shared object alone; I doubt that this is even possible. I was thinking about this as I was getting groceries a bit ago. Why -can't- we expect the user to have liblzma headers installed? Couldn't it just be a dependency in the package management system? Package managers, under Linux, often split development files (headers, etc.) from runtime binaries. Well, uh, yeah. Not sure what your point is. 1) We could easily work with the dev / nondev distinction by taking a dependency on the -dev version of whatever we need, instead of the nondev version. 2) It's a rather arbitrary distinction that's being drawn between dev and nondev today. There's no particular reason why the line couldn't be drawn somewhere else. Also, under Windows, most users don't have development stuff installed at all. Yes... But if the nature of what development stuff is were to change, they'd have different stuff. Also, we wouldn't have to parse the .h's every time a module is loaded - we could have a timestamp file (or database) indicating when we last parsed a given .h. Also, we could query the package management system for the version of lzma that's currently installed on module init. Also, we could include our own version of lzma. Granted, this was a mess when zlib needed to be patched, but even this one might be worth it for the improved library unification across Python implementations. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, 27 Aug 2011 16:19:01 -0700 Dan Stromberg drsali...@gmail.com wrote: 2) It's a rather arbitrary distinction that's being drawn between dev and nondev today. There's no particular reason why the line couldn't be drawn somewhere else. Sure. Now please convince Linux distributions first, because this particular subthread is going nowhere. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou solip...@pitrou.net wrote: Sure. Now please convince Linux distributions first, because this particular subthread is going nowhere. I hope you're not a solipsist. Anyway, if the mere -discussion- of embracing a standard and safe way of making C libraries callable from all the major Python implementations is going nowhere before the discussion has even gotten started, I fear for Python's future. Repeat aloud to yourself: Python != CPython. Python != CPython. Python != CPython. Has this topic been discussed to death? If so, then say so. It's rude to try to kill the thread summarily before it gets started, sans discussion, sans explanation, sans commentary on whether new additions to the topic have surfaced or not. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 27 Aug 2011 16:19:01 -0700 Dan Stromberg drsali...@gmail.com wrote: 2) It's a rather arbitrary distinction that's being drawn between dev and nondev today. There's no particular reason why the line couldn't be drawn somewhere else. Sure. Now please convince Linux distributions first, because this particular subthread is going nowhere. Interesting. You seem to want to throw an arbitrary barrier between Python, the language, and accomplishing something important for said language. Care to tell me why I'm wrong? I'm all ears. I'll note that you've deleted: 1) We could easily work with the dev / nondev distinction by taking a dependency on the -dev version of whatever we need, instead of the nondev version. ...which makes it more than apparent that we needn't convince Linux distributors of #2, which you seem to prefer to focus on. Why was it in your best interest to delete #1, without even commenting on it? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg drsali...@gmail.com wrote: IMO, we really, really need some common way of accessing C libraries that works for all major Python variants. We have one. It's called writing an extension module. ctypes is a crutch because it doesn't realistically have access to the header files. It's a fine crutch for PyPy, which doesn't have much of an alternative. It's also a fine crutch for people who need something to run *now*. It's a horrible strategy for the standard library. If you have a better proposal please do write it up. But so far you are mostly exposing your ignorance and insisting dramatically that you be educated. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LZMA compression support in 3.3
On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum gu...@python.org wrote:
On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg drsali...@gmail.com
wrote:
IMO, we really, really need some common way of accessing C libraries that
works for all major Python variants.
We have one. It's called writing an extension module.
And yet Cext's are full of CPython-isms.
I've said in the past that Python has been lucky in that it had only a
single implementation for a long time, but still managed to escape becoming
too defined by the idiosyncrasies of that implementation - that's quite
impressive, and is probably our best indication that Python has had
leadership with foresight. In the language proper, I'd say I still believe
this, but Cext's are sadly not a good example.
ctypes is a crutch because it doesn't realistically have access to the
header files.
Well, actually, header files are pretty easy to come by. I bet you've
installed them yourself many times. In fact, you've probably even
automatically brought some of them in via a package management system of one
form or another without getting your hands dirty.
As a thought experiment, imagine having a ctypes configuration system that
looks around a computer for .h's and .so's (etc) with even 25% of the effort
expended by GNU autoconf. Instead of building the results into a bunch of
.o's, the results are saved in a .ct file or something. If you build-in
some reasonable default locations to look in, plus the equivalent of some
-I's and -L's (and maybe -rpath's) as needed, you probably end up with a
pretty comparable system.
(typedef's might be a harder problem - that's particularly worth discussing,
IMO - your chance to nip this in the bud with a reasoned explanation why
they can't be handled well!)
It's a fine crutch for PyPy, which doesn't have much of
an alternative.
Wait - a second ago I thought I was to believe that C extension modules were
the one true way of interfacing with C code across all major
implementations? Are we perhaps saying that CPython is the major
implementation, and that we want it to stay that way?
I personally feel that PyPy has arrived as a major implementation. The
backup program I've been writing in my spare time runs great on PyPy (and
the CPython's from 2.5.x, and pretty well on Jython). And PyPy has been
maturing very rapidly ('just wish they'd do 3.x!).
It's also a fine crutch for people who need something
to run *now*. It's a horrible strategy for the standard library.
I guess I'm coming to see this as dogma.
If ctypes is augmented with type information and/or version information and
where to find things, wouldn't it Become safe and convenient? Or do you
have other concerns?
Make a list of things that can go wrong with ctypes modules. Now make a
list of things that can wrong with C extension modules. Aren't they really
pretty similar - missing .so, .so in a weird place, and especially: .so with
a changed interface? C really isn't a very safe language - not like
http://en.wikipedia.org/wiki/Turing_%28programming_language%29 or
something. Perhaps it's a little easier to mess things up with ctypes today
(a recompile doesn't fix, or at least detect, as many problems), but isn't
it at least worth Thinking about how that situation could be improved?
If you have a better proposal please do write it up. But so far you
are mostly exposing your ignorance and insisting dramatically that you
be educated.
I'm not sure why you're trying to avoid having a discussion. I think it's
premature to dive into a proposal before getting other people's thoughts.
Frankly, 100 people tend to think better than one - at least, if the 100
people feel like they can talk.
I'm -not- convinced ctypes are the way forward. I just want to talk about
it - for now. ctypes have some significant advantages - if we can find a
way to eliminate and/or ameliorate their disadvantages, they might be quite
a bit nicer than Cext's.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
