Re: [Python-Dev] LZMA compression support in 3.3

Dan Stromberg, 27.08.2011 21:58: On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan wrote: On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda wrote: On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan wrote: It's acceptable for the Python version to use ctypes in the case of wrapping an existing library, but

Re: [Python-Dev] LZMA compression support in 3.3

> I just want to talk about it - for now. python-ideas is a better place to just talk than python-dev. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 8:57 PM, Guido van Rossum wrote: > On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg > wrote: > > IMO, we really, really need some common way of accessing C libraries that > > works for all major Python variants. > > We have one. It's called writing an extension module. > A

Re: [Python-Dev] Should we move to replace re with regex?

On 8/27/2011 11:54 PM, Guido van Rossum wrote: If so, it would be like the decimal module, which closely tracks the IEEE decimal standard, rather than the binary float standard. Well, I would hope that for each "major" Python version (i.e. 3.2, 3.3, 3.4, ...) we would pick a specific version o

[Python-Dev] Ctypes and the stdlib (was Re: LZMA compression support in 3.3)

Dan, I once had the more or less the same opinion/question as you with regard to ctypes, but I now see at least 3 problems. 1) It seems hard to write it correctly. There are currently 47 open ctypes issues, with 9 being feature requests, leaving 38 behavior-related issues. Tom Heller has not b

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 8:59 PM, Ezio Melotti wrote: > On Sat, Aug 27, 2011 at 4:56 AM, Antoine Pitrou wrote: >> >> On Sat, 27 Aug 2011 04:37:21 +0300 >> Ezio Melotti wrote: >> > >> > I'm not sure it's worth doing an extensive review of the code, a better >> > approach might be to require extens

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 4:56 AM, Antoine Pitrou wrote: > On Sat, 27 Aug 2011 04:37:21 +0300 > Ezio Melotti wrote: > > > > I'm not sure it's worth doing an extensive review of the code, a better > > approach might be to require extensive test coverage (and a review of > > tests). If the code se

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 3:14 PM, Dan Stromberg wrote: > IMO, we really, really need some common way of accessing C libraries that > works for all major Python variants. We have one. It's called writing an extension module. ctypes is a crutch because it doesn't realistically have access to the he

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 5:48 PM, Terry Reedy wrote: > Many of the things regex does differently might be called either bug fixes > or feature changes, depending on one's viewpoint. Regex should definitely > not be 'bug-compatible'. Well, as you said, it depends on one's viewpoint. If there's a bu

Re: [Python-Dev] Should we move to replace re with regex?

On Sun, Aug 28, 2011 at 3:48 AM, Terry Reedy wrote: > > These are reasons why both Ezio and I suggested on the tracker adding regex > without deleting re. (I personally would not mind just replacing re with > regex, but then I have no legacy re code to break. So I am not suggesting > that out of

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou wrote: > On Sat, 27 Aug 2011 16:19:01 -0700 > Dan Stromberg wrote: > > 2) It's a rather arbitrary distinction that's being drawn between dev and > > nondev today. There's no particular reason why the line couldn't be > drawn > > somewhere else. >

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:27 PM, Antoine Pitrou wrote: > > Sure. Now please convince Linux distributions first, because this > particular subthread is going nowhere. > I hope you're not a solipsist. Anyway, if the mere -discussion- of embracing a standard and safe way of making C libraries call

Re: [Python-Dev] Should we move to replace re with regex?

On 8/27/2011 7:39 PM, Greg Ewing wrote: Nick Coghlan wrote: The next step needed is for someone to volunteer to write and champion a PEP that: Would it be feasible and desirable to modify regex so that it *is* backwards-compatible with re, with a view to making it a drop-in replacement at som

Re: [Python-Dev] Should we move to replace re with regex?

Nick Coghlan wrote: The next step needed is for someone to volunteer to write and champion a PEP that: Would it be feasible and desirable to modify regex so that it *is* backwards-compatible with re, with a view to making it a drop-in replacement at some point? If not, the PEP should discuss

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 16:19:01 -0700 Dan Stromberg wrote: > 2) It's a rather arbitrary distinction that's being drawn between dev and > nondev today. There's no particular reason why the line couldn't be drawn > somewhere else. Sure. Now please convince Linux distributions first, because this part

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 3:26 PM, Antoine Pitrou wrote: > On Sat, 27 Aug 2011 15:14:15 -0700 > Dan Stromberg wrote: > > > On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda >wrote: > > > > > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > > > wrote: > > > > It seems like there should be some way o

Re: [Python-Dev] LZMA compression support in 3.3

> Why -can't- we expect the user to have liblzma headers installed? > Couldn't it just be a dependency in the package management system? Please give it up. You just won't convince that list that ctypes is a viable approach for the standard library. Regards, Martin ___

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 15:14:15 -0700 Dan Stromberg wrote: > On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda wrote: > > > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > > wrote: > > > It seems like there should be some way of coming up with an xml file > > > describing the types of the various bi

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 2:38 PM, Nadeem Vawda wrote: > On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg > wrote: > > It seems like there should be some way of coming up with an xml file > > describing the types of the various bits of data and formal arguments - > > perhaps using gccxml or somethin

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 10:41 PM, Dan Stromberg wrote: > It seems like there should be some way of coming up with an xml file > describing the types of the various bits of data and formal arguments - > perhaps using gccxml or something like it. The problem is that you would need to do this check

Re: [Python-Dev] Add from __experimental__ import bla [was: Should we move to replace re with regex?]

On 07:57 pm, digitalx...@gmail.com wrote: In the thread about replacing re with regex someone mentioned adding to __future__ which isnt a great idea as future APIs are already solidified, they just live there to give developer time to adapt their code. The idea of a __experimental__ area is good

Re: [Python-Dev] Add from __experimental__ import bla [was: Should we move to replace re with regex?]

Le samedi 27 août 2011 21:57:26, Dj Gilcrease a écrit : > The idea of a __experimental__ area is good for any pep's or > stliib additions that are somewhat controversial (API isnt agreed on, > code may take a while to integrate properly, developer wants some time > to hash out any edge case bugs or

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 1:21 PM, "Martin v. Löwis" wrote: > > I'd like to better understand why ctypes is (sometimes) frowned upon. > > > > Is it the brittleness? Tendency to segfault? > > That, and Python should work completely if ctypes is not available. > What are the most major platforms cty

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 9:47 PM, Terry Reedy wrote: > On 8/27/2011 9:47 AM, Nadeem Vawda wrote: >> I'd like to propose the addition of a new module in Python 3.3. The 'lzma' >> module will provide support for compression and decompression using the >> LZMA >> algorithm, and the .xz and .lzma file

Re: [Python-Dev] LZMA compression support in 3.3

> I'd like to better understand why ctypes is (sometimes) frowned upon. > > Is it the brittleness? Tendency to segfault? That, and Python should work completely if ctypes is not available. > FWIW, I have a partial implementation of a module that does xz from > Python using ctypes. So does it w

[Python-Dev] Add from __experimental__ import bla [was: Should we move to replace re with regex?]

In the thread about replacing re with regex someone mentioned adding to __future__ which isnt a great idea as future APIs are already solidified, they just live there to give developer time to adapt their code. The idea of a __experimental__ area is good for any pep's or stliib additions that are s

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 9:04 AM, Nick Coghlan wrote: > On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda > wrote: > > On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan > wrote: > >> It's acceptable for the Python version to use ctypes in the case of > >> wrapping an existing library, but the Python ver

Re: [Python-Dev] Should we move to replace re with regex?

Dan Stromberg wrote: On Sat, Aug 27, 2011 at 9:53 AM, Brian Curtin wrote: On Sat, Aug 27, 2011 at 11:48, Dan Stromberg wrote: No, this was not the intent of __future__. The intent is that a feature is desirable but also backwards incompatible (e.g. introduces a new keyword) so that for 1 (so

Re: [Python-Dev] LZMA compression support in 3.3

On 8/27/2011 9:47 AM, Nadeem Vawda wrote: I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker

Re: [Python-Dev] Should we move to replace re with regex?

> I disagree. The first paragraph says this has something to do with new > keywords. It doesn't appear to say what we expect users to -do- with > it. Both are important. Well, users can use the new features... > Is it "You'd better try this, because it's going in eventually. If you > don't tr

Re: [Python-Dev] Should we move to replace re with regex?

On 2011-08-27, at 2:20 PM, Dan Stromberg wrote: > > On Sat, Aug 27, 2011 at 9:53 AM, Brian Curtin wrote: > On Sat, Aug 27, 2011 at 11:48, Dan Stromberg wrote: > No, this was not the intent of __future__. The intent is that a > feature is desirable but also backwards incompatible (e.g. introduce

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 9:53 AM, Brian Curtin wrote: > On Sat, Aug 27, 2011 at 11:48, Dan Stromberg wrote: >> >> No, this was not the intent of __future__. The intent is that a >>> feature is desirable but also backwards incompatible (e.g. introduces >>> a new keyword) so that for 1 (sometimes mo

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, 27 Aug 2011 18:50:40 +0200 Antoine Pitrou wrote: > On Sun, 28 Aug 2011 01:52:51 +1000 > Nick Coghlan wrote: > > > > > > The plausible story being that we basically wrap an existing library? > > > I don't think PyPy et al have pure Python versions of the zlib or > > > OpenSSL, do they? >

Re: [Python-Dev] Should we move to replace re with regex?

Am 27.08.2011 12:10, schrieb Antoine Pitrou: > On Sat, 27 Aug 2011 08:02:31 +0200 > "Martin v. Löwis" wrote: >>> I'm not sure it's worth doing an extensive review of the code, a better >>> approach might be to require extensive test coverage (and a review of >>> tests). >> >> I think it's worth.

Re: [Python-Dev] Software Transactional Memory for Python

Hi Armin, > This is basically dangerous, because it corresponds to taking lock > "GIL" and lock L, in that order, whereas the thread B takes lock L and > plays around with lock "GIL" in the opposite order.  I think a > reasonable solution to avoid deadlocks is simply not to use explicit > locks in

Re: [Python-Dev] LZMA compression support in 3.3

>>> PEP 399 also comes into play - we need a pure Python version for PyPy >>> et al (or a plausible story for why an exception should be granted). No, we don't. We can grant an exception, which I'm very willing to do. The PEP lists wrapping a specific C-based library as a plausible reason. > It's

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 11:48, Dan Stromberg wrote: > > No, this was not the intent of __future__. The intent is that a >> feature is desirable but also backwards incompatible (e.g. introduces >> a new keyword) so that for 1 (sometimes more) releases we require the >> users to use the __future__ i

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, 28 Aug 2011 01:52:51 +1000 Nick Coghlan wrote: > > > > The plausible story being that we basically wrap an existing library? > > I don't think PyPy et al have pure Python versions of the zlib or > > OpenSSL, do they? > > > > If we start taking PEP 399 conformance to such levels, we might a

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 9:19 AM, Guido van Rossum wrote: > On Fri, Aug 26, 2011 at 11:01 PM, Dan Stromberg > wrote: > [Steven] > >> Have then been any __future__ features that were added provisionally? > > > > I can't either, but ISTR hearing that from __future__ import was started > > with such

Re: [Python-Dev] Should we move to replace re with regex?

On Fri, Aug 26, 2011 at 11:01 PM, Dan Stromberg wrote: [Steven] >> Have then been any __future__ features that were added provisionally? > > I can't either, but ISTR hearing that from __future__ import was started > with such an intent.  Irrespective, it's hard to import something from > "future"

Re: [Python-Dev] Software Transactional Memory for Python

Hi Antoine, > You then risk deadlocks. Say: > (...) Yes, it is indeed not a solution that co-operates transparently and deadlock-freely with regular locks. You risk the same kind of deadlocks as you would when using only locks. The reason is similar to threads that try to acquire two locks in s

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:58 AM, Nadeem Vawda wrote: > On Sat, Aug 27, 2011 at 5:52 PM, Nick Coghlan wrote: >> It's acceptable for the Python version to use ctypes in the case of >> wrapping an existing library, but the Python version should still >> exist. > > I'm not too sure about that - PEP 3

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 5:42 PM, "Martin v. Löwis" wrote: > Not sure whether you already have this: supporting the tarfile module > would be nice. Yes, got that - issue 5689. Also of interest is issue 5411 - adding .xz support to distutils. But I think that these are separate projects that should

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:40 AM, Antoine Pitrou wrote: > On Sun, 28 Aug 2011 01:36:50 +1000 > Nick Coghlan wrote: >> On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" >> wrote: >> > This is exactly what I worry about. I think adding file I/O to bz2 was a >> > mistake, as this doesn't integrate

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, 28 Aug 2011 01:36:50 +1000 Nick Coghlan wrote: > On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" wrote: > > This is exactly what I worry about. I think adding file I/O to bz2 was a > > mistake, as this doesn't integrate with Python's IO library (it used > > to, but now after dropping s

Re: [Python-Dev] LZMA compression support in 3.3

> It is not my intention for the _lzma C module to do I/O - that will be done by > the LZMAFile class, which will be written in Python. My comparison with bz2 > was > in reference to the state of the module after it was rewritten for issue 5863. Ok. I'll defer my judgement then until actual code

Re: [Python-Dev] LZMA compression support in 3.3

On Sun, Aug 28, 2011 at 1:15 AM, "Martin v. Löwis" wrote: > This is exactly what I worry about. I think adding file I/O to bz2 was a > mistake, as this doesn't integrate with Python's IO library (it used > to, but now after dropping stdio, they were incompatible. Indeed, for > Python 3.2, BZ2File

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 5:15 PM, "Martin v. Löwis" wrote: >> As for file formats, these are handled by liblzma itself; the extension >> module >> just selects which compressor/decompressor initializer function to use >> depending >> on the value of the "format" argument. Our code won't contain a

Re: [Python-Dev] LZMA compression support in 3.3

> As for file formats, these are handled by liblzma itself; the extension module > just selects which compressor/decompressor initializer function to use > depending > on the value of the "format" argument. Our code won't contain anything along > the > lines of GzipFile; all of that work is done

Re: [Python-Dev] LZMA compression support in 3.3

On Sat, Aug 27, 2011 at 4:50 PM, "Martin v. Löwis" wrote: >> The implementation will also be similar to bz2 - basic compressor and >> decompressor classes written in C, with convenience functions and a file >> interface implemented on top of those in Python. > > When I reviewed lzma, I found that

Re: [Python-Dev] LZMA compression support in 3.3

> The implementation will also be similar to bz2 - basic compressor and > decompressor classes written in C, with convenience functions and a file > interface implemented on top of those in Python. When I reviewed lzma, I found that this approach might not be appropriate. lzma has many more option

Re: [Python-Dev] Software Transactional Memory for Python

On Sat, 27 Aug 2011 15:08:36 +0200 Armin Rigo wrote: > Hi Nick, > > On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan wrote: > > 1. How does the patch interact with C code that explicitly releases > > the GIL? (e.g. IO commands inside a "with atomic:" block) > > As implemented, any code in a "with

Re: [Python-Dev] LZMA compression support in 3.3

> I'd like to propose the addition of a new module in Python 3.3. The 'lzma' > module will provide support for compression and decompression using the LZMA > algorithm, and the .xz and .lzma file formats. The matter has already been > discussed on the tracker , whe

[Python-Dev] LZMA compression support in 3.3

Hello all, I'd like to propose the addition of a new module in Python 3.3. The 'lzma' module will provide support for compression and decompression using the LZMA algorithm, and the .xz and .lzma file formats. The matter has already been discussed on the tracker ,

Re: [Python-Dev] Software Transactional Memory for Python

Hi Nick, On Sat, Aug 27, 2011 at 2:40 PM, Nick Coghlan wrote: > 1. How does the patch interact with C code that explicitly releases > the GIL? (e.g. IO commands inside a "with atomic:" block) As implemented, any code in a "with atomic" is prevented from explicitly releasing and reacquiring the G

Re: [Python-Dev] Software Transactional Memory for Python

On Sat, Aug 27, 2011 at 8:45 PM, Armin Rigo wrote: > Hi all, > > About multithreading models: I recently made an observation which > might be obvious to some, but not to me, and as far as I know not to > most of us either.  I think that it's worth being pointed out :-) > > http://mail.python.org/p

Re: [Python-Dev] Should we move to replace re with regex?

On 26 Aug, 09:45 pm, gu...@python.org wrote: I just made a pass of all the Unicode-related bugs filed by Tom Christiansen, and found that in several, the response was "this is fixed in the regex module [by Matthew Barnett]". I started replying that I thought that we should fix the bugs in the re

[Python-Dev] Software Transactional Memory for Python

Hi all, About multithreading models: I recently made an observation which might be obvious to some, but not to me, and as far as I know not to most of us either. I think that it's worth being pointed out :-) http://mail.python.org/pipermail/pypy-dev/2011-August/008153.html A bientôt, Armin. _

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, 27 Aug 2011 08:02:31 +0200 "Martin v. Löwis" wrote: > > I'm not sure it's worth doing an extensive review of the code, a better > > approach might be to require extensive test coverage (and a review of > > tests). > > I think it's worth. It's really bad if only one developer fully > unde

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, 27 Aug 2011 09:18:14 +0200 "Martin v. Löwis" wrote: > Am 27.08.2011 08:33, schrieb Terry Reedy: > > On 8/26/2011 9:56 PM, Antoine Pitrou wrote: > > > >> Another "interesting" question is whether it's easy to port to the PEP > >> 393 string representation, if it gets accepted. > > > > Wil

Re: [Python-Dev] issue 6721 "Locks in python standard library should be sanitized on fork"

On 26 Aug 2011, at 16:53, Antoine Pitrou wrote: > > Hi, > >> I think that "deprecating" the use of threads w/ multiprocessing - or >> at least crippling it is the wrong answer. Multiprocessing needs the >> helper threads it uses internally to manage queues, etc. Removing that >> ability would r

Re: [Python-Dev] Should we move to replace re with regex?

On Sat, Aug 27, 2011 at 4:01 PM, Dan Stromberg wrote: > You're talking technically, which is important, but wasn't what I was > suggesting would be helped. > > Politically, and from a marketing standpoint, it's easier to withdraw a > feature you've given with a "Play with this, see if it works for

Re: [Python-Dev] PEP 393 Summer of Code Project

Am 27.08.2011 09:40, schrieb Steven D'Aprano: > Terry Reedy wrote: >> On 8/26/2011 8:23 PM, Antoine Pitrou wrote: >> I would only agree as long as it wasn't too much worse than O(1). O(log n) might be all right, but O(n) would be unacceptable, I think. >>> >>> It also depends a lot o

Re: [Python-Dev] PEP 393 Summer of Code Project

Terry Reedy wrote: On 8/26/2011 8:23 PM, Antoine Pitrou wrote: I would only agree as long as it wasn't too much worse than O(1). O(log n) might be all right, but O(n) would be unacceptable, I think. It also depends a lot on *actual* measured performance Amen. Some regard O(n*n) sorts to be,

Re: [Python-Dev] Should we move to replace re with regex?

Am 27.08.2011 08:33, schrieb Terry Reedy: > On 8/26/2011 9:56 PM, Antoine Pitrou wrote: > >> Another "interesting" question is whether it's easy to port to the PEP >> 393 string representation, if it gets accepted. > > Will the re module need porting also? That's a quality-of-implementation issu