Re: [Python-Dev] Optimization

2013-10-06 Thread Xavier Morel
On 2013-10-06, at 12:37 , Stephen J. Turnbull wrote: > > For me, the point about string "+=" being efficient (sometimes) isn't > that it is surprising compared to similar types, it's that it is > surprising for any immutable sequence type. It's clearly nitpicking, but ropes are immutable sequenc

Re: [Python-Dev] Optimization

2013-10-06 Thread Stephen J. Turnbull
Nick Coghlan writes: > I suspect "list" may have been the intended comparison there. > "array.array" is another appropriate comparison. Having bytearray > operations differ in algorithmic complexity from those two types > would be very strange and surprising I don't see why. If you really w

Re: [Python-Dev] Optimization

2013-10-05 Thread Serhiy Storchaka
06.10.13 02:37, Antoine Pitrou написав(ла): It's only strange because you don't understand the main use case for bytearrays. They may look like arrays of 8-bit integers but they are really used for buffers, so optimizing for stuff like FIFO operation makes sense. Could you please provide severa

Re: [Python-Dev] Optimization

2013-10-05 Thread Georg Brandl
Am 06.10.2013 01:37, schrieb Antoine Pitrou: > On Sun, 6 Oct 2013 09:32:30 +1000 > Nick Coghlan wrote: >> On 6 Oct 2013 08:59, "Antoine Pitrou" wrote: >> > >> > On Sat, 5 Oct 2013 15:35:30 -0700 >> > Raymond Hettinger wrote: >> > > >> > > Making bytearray's efficiently pop from the left side is

Re: [Python-Dev] Optimization

2013-10-05 Thread Antoine Pitrou
On Sun, 6 Oct 2013 09:32:30 +1000 Nick Coghlan wrote: > On 6 Oct 2013 08:59, "Antoine Pitrou" wrote: > > > > On Sat, 5 Oct 2013 15:35:30 -0700 > > Raymond Hettinger wrote: > > > > > > Making bytearray's efficiently pop from the left side is dubious. > > > This isn't a common idiom, nor should it

Re: [Python-Dev] Optimization

2013-10-05 Thread Nick Coghlan
On 6 Oct 2013 08:59, "Antoine Pitrou" wrote: > > On Sat, 5 Oct 2013 15:35:30 -0700 > Raymond Hettinger wrote: > > > > Making bytearray's efficiently pop from the left side is dubious. > > This isn't a common idiom, nor should it be. Even if all the > > other implementations could model this beha

Re: [Python-Dev] Optimization

2013-10-05 Thread Rob Cliffe
Sorry, I may have missed some earlier relevant parts of this discussion. But you appear to be saying that you don't want to optimize something because it would be hard to explain why it performed better. Eh?? Have I misunderstood? Rob Cliffe On 05/10/2013 23:35, Raymond Hettinger wrote: On O

Re: [Python-Dev] Optimization

2013-10-05 Thread Antoine Pitrou
On Sat, 5 Oct 2013 15:35:30 -0700 Raymond Hettinger wrote: > > Making bytearray's efficiently pop from the left side is dubious. > This isn't a common idiom, nor should it be. Even if all the > other implementations could model this behavior, it wouldn't > be a good idea to have bytearrays have

Re: [Python-Dev] Optimization

2013-10-05 Thread Raymond Hettinger
On Oct 5, 2013, at 12:42 PM, Serhiy Storchaka wrote: > Please remember me, what was common decision about CPython-only optimizations > which change computation complexity? IIRC, it is okay optimize C code for just about anything, but we don't want to alter the pure python code after from idiom

Re: [Python-Dev] Optimization

2013-10-05 Thread Serhiy Storchaka
06.10.13 00:06, Victor Stinner написав(ла): The str type is immutable, bytearray is not. It's easier to justify optimisations on mutable types, like overallocate lists for example. We can resize str or bytes if its refcount is 1. And resize is cheap in some circumstances. This optimization is

Re: [Python-Dev] Optimization

2013-10-05 Thread Victor Stinner
The str type is immutable, bytearray is not. It's easier to justify optimisations on mutable types, like overallocate lists for example. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscr

Re: [Python-Dev] Optimization

2013-10-05 Thread Antoine Pitrou
On Sun, 06 Oct 2013 00:02:14 +0300 Serhiy Storchaka wrote: > 05.10.13 23:11, Georg Brandl написав(ла): > > Am 05.10.2013 21:42, schrieb Serhiy Storchaka: > >> Please remember me, what was common decision about CPython-only > >> optimizations which change computation complexity? I.g. constant > >>

Re: [Python-Dev] Optimization

2013-10-05 Thread Serhiy Storchaka
05.10.13 23:11, Georg Brandl написав(ла): Am 05.10.2013 21:42, schrieb Serhiy Storchaka: Please remember me, what was common decision about CPython-only optimizations which change computation complexity? I.g. constant amortization time of += for byte objects and strings, or linear time of sum()

Re: [Python-Dev] Optimization

2013-10-05 Thread Antoine Pitrou
On Sat, 05 Oct 2013 22:11:43 +0200 Georg Brandl wrote: > Am 05.10.2013 21:42, schrieb Serhiy Storchaka: > > Please remember me, what was common decision about CPython-only > > optimizations which change computation complexity? I.g. constant > > amortization time of += for byte objects and string

Re: [Python-Dev] Optimization

2013-10-05 Thread Georg Brandl
Am 05.10.2013 21:42, schrieb Serhiy Storchaka: > Please remember me, what was common decision about CPython-only > optimizations which change computation complexity? I.g. constant > amortization time of += for byte objects and strings, or linear time of > sum() for sequences? This appears to be

[Python-Dev] Optimization

2013-10-05 Thread Serhiy Storchaka
Please remember me, what was common decision about CPython-only optimizations which change computation complexity? I.g. constant amortization time of += for byte objects and strings, or linear time of sum() for sequences? ___ Python-Dev mailing list

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-08-18 Thread Antoine Pitrou
Eric Smith trueblade.com> writes: > > I finally backported this to 2.6 in r65814. There's a similar 30% > speedup for the simplest cases. Unicode optimization is worse than > string optimization, because of the way int, long, and float formatters > work. This can be fixed, but I'm not sure

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-08-18 Thread Eric Smith
Eric Smith wrote: Eric Smith wrote: Eric Smith wrote: Nick Coghlan wrote: Secondly, the string % operator appears to have an explicit optimisation for the 'just return str(self)' case. This optimisation is missing from the new string format method. I'll see if I can optimize this case. 3.

Re: [Python-Dev] optimization required: .format() i s much slower than %

2008-05-31 Thread Antoine Pitrou
Simon Cross gmail.com> writes: > My tests show that the old-style % formatting is much faster when the > final string is 20 characters or less: > > $ ./python -m timeit "'|||...%s' % '12'" > 1000 loops, best of 3: 0.0764 usec per loop You are the victim of a constant-folding opt

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-30 Thread Eric Smith
Eric Smith wrote: Eric Smith wrote: Nick Coghlan wrote: Secondly, the string % operator appears to have an explicit optimisation for the 'just return str(self)' case. This optimisation is missing from the new string format method. I'll see if I can optimize this case. 3.0, from svn: $ ./py

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-30 Thread Nick Coghlan
Simon Cross wrote: It struct me as odd that this one case shows such a big difference while the others show less of one. I believe the %-formatting code has some optimisations in place where it realises it can just increment the reference count of the passed in string and return that, rather

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-29 Thread Simon Cross
On Tue, May 27, 2008 at 12:43 AM, Christian Heimes <[EMAIL PROTECTED]> wrote: > Ten minutes ago I raised a concern about speed differences between the > old style % formatting and the new .format() code. Some quick > benchmarking from Benjamin and me showed, that it's even worse than I > expected.

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-29 Thread Steve Holden
Nick Coghlan wrote: [EMAIL PROTECTED] wrote: Nick> $ ./python -m timeit "'' % ()" Nick> 100 loops, best of 3: 0.389 usec per loop vs. Nick> $ ./python -m timeit "'%s' % 'nothing'" Nick> 1000 loops, best of 3: 0.0736 usec per loop I think you need to use a tuple for the

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-29 Thread Nick Coghlan
[EMAIL PROTECTED] wrote: Nick> $ ./python -m timeit "'' % ()" Nick> 100 loops, best of 3: 0.389 usec per loop vs. Nick> $ ./python -m timeit "'%s' % 'nothing'" Nick> 1000 loops, best of 3: 0.0736 usec per loop I think you need to use a tuple for the second case to make

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-29 Thread skip
Nick> $ ./python -m timeit "'' % ()" Nick> 100 loops, best of 3: 0.389 usec per loop vs. Nick> $ ./python -m timeit "'%s' % 'nothing'" Nick> 1000 loops, best of 3: 0.0736 usec per loop I think you need to use a tuple for the second case to make it comparable to the first

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-27 Thread Eric Smith
Christian Heimes wrote: Antoine Pitrou schrieb: In order to avoid memory consumption issues there could be a centralized cache as for regular expressions. It makes it easier to handle eviction based on various parameters, and it saves a few bytes for string objects which are never used as a form

Re: [Python-Dev] optimization required: .format() is much slower than %

2008-05-27 Thread Christian Heimes
Antoine Pitrou schrieb: > In order to avoid memory consumption issues there could be a centralized cache > as for regular expressions. It makes it easier to handle eviction based on > various parameters, and it saves a few bytes for string objects which are > never > used as a formatting template.

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-18 Thread Frank Wierzbicki
On Fri, May 2, 2008 at 10:21 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Any Jython folk care to weigh in on this? If there are no major objections I > think I'm going to forge ahead with an independant Const() node. I suspect that having a marker for non-int non-str constants could also be used fo

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Nick Coghlan wrote: Steve Holden wrote: While not strictly related to the global statement, perhaps Adam refers to the possibility of optimizing away code with an assignment which would make a name be recognized as local? If you're worried about "yield" disappearing you should also be worried

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Nick Coghlan
Steve Holden wrote: While not strictly related to the global statement, perhaps Adam refers to the possibility of optimizing away code with an assignment which would make a name be recognized as local? If you're worried about "yield" disappearing you should also be worried about assignments di

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Adam Olsen
On Thu, May 8, 2008 at 5:54 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Adam Olsen wrote: >> >> On Thu, May 8, 2008 at 5:22 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: >> >>> >>> Nick Coghlan wrote: >>> There are a lot of micro-optimisations that are actually context independent, so mo

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Steve Holden
Thomas Lee wrote: Adam Olsen wrote: On Thu, May 8, 2008 at 5:22 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: Nick Coghlan wrote: There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "r

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Adam Olsen wrote: On Thu, May 8, 2008 at 5:22 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: Nick Coghlan wrote: There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "return None" with

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Adam Olsen
On Thu, May 8, 2008 at 5:22 PM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Nick Coghlan wrote: >> >> There are a lot of micro-optimisations that are actually context >> independent, so moving them before the symtable pass should be quite >> feasible - e.g. replacing "return None" with "return", stripp

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Thomas Lee
Nick Coghlan wrote: There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "return None" with "return", stripping dead code after a return statement, changing a "if not" statement into an

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Jeremy Hylton
On Thu, May 8, 2008 at 8:00 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Jeremy Hylton wrote: > > > On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: > > > > > Nick Coghlan wrote: > > > > > > > > > > As Thomas mentions in a later message, making it possible to annotate > > > >

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-08 Thread Nick Coghlan
Jeremy Hylton wrote: On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: Nick Coghlan wrote: As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-07 Thread Jeremy Hylton
On Wed, May 7, 2008 at 11:43 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Nick Coghlan wrote: > > > > > As Thomas mentions in a later message, making it possible to annotate > nodes would permit Functions to be annotated as being a generator at the AST > stage (currently it is left to the bytecode c

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-07 Thread Thomas Lee
Nick Coghlan wrote: As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the bytecode compiler's symtable generation pass to make that determination). Although I gues

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-07 Thread Nick Coghlan
Jeremy Hylton wrote: On Fri, May 2, 2008 at 1:38 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: Thomas Lee wrote: > Martin v. Löwis wrote: >>> This leaves us with a few options: >>> >> >> 5. Reuse/Abuse Num(object) for arbitrary constants. >>AFAICT, this should work out of the box.

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-06 Thread Jeremy Hylton
On Fri, May 2, 2008 at 1:38 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Thomas Lee wrote: > > Martin v. Löwis wrote: > >>> This leaves us with a few options: > >>> > >> > >> 5. Reuse/Abuse Num(object) for arbitrary constants. > >>AFAICT, this should work out of the box. > >> > >>

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-02 Thread Martin v. Löwis
Thomas Lee wrote: > Martin v. Löwis wrote: >>> This leaves us with a few options: >>> >> >> 5. Reuse/Abuse Num(object) for arbitrary constants. >>AFAICT, this should work out of the box. >> >> > Eek. It *does* seem like Num would work out of the box, but would this > be a good idea? No

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-05-02 Thread Thomas Lee
Martin v. Löwis wrote: This leaves us with a few options: 5. Reuse/Abuse Num(object) for arbitrary constants. AFAICT, this should work out of the box. Eek. It *does* seem like Num would work out of the box, but would this be a good idea? What about *replacing* Num with Const? Mig

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-04-30 Thread Martin v. Löwis
> This leaves us with a few options: 5. Reuse/Abuse Num(object) for arbitrary constants. AFAICT, this should work out of the box. > 1. A new AST expr node for constant values for types other than Str/Num > > I imagine this to be something like Const(PyObject* v), which is > effectively transl

Re: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-04-30 Thread Brett Cannon
On Wed, Apr 30, 2008 at 3:15 AM, Thomas Lee <[EMAIL PROTECTED]> wrote: > Hi all, > > I've been working on optimization of the AST, including the porting of the > old bytecode-level optimizations to the AST level. A few questions have come > up in the process of doing this, all of which are probabl

[Python-Dev] Optimization of Python ASTs: How should we deal with constant values?

2008-04-30 Thread Thomas Lee
Hi all, I've been working on optimization of the AST, including the porting of the old bytecode-level optimizations to the AST level. A few questions have come up in the process of doing this, all of which are probably appropriate for discussion on this list. The code I'm referring to here ca