On 14/02/13 02:22, Marius Gedminas wrote:
On Wed, Feb 13, 2013 at 11:10:26PM +1100, Steven D'Aprano wrote:
Best practice remains the same:
- we should still use join for repeated concatenations;
- we should still avoid + except for small cases which are not performance
critical;
- we should still teach beginners to use join;
- while this optimization is nice to have, we cannot rely on it being there
when it matters.
It's not just Jython and IronPython that can't make use of this optimization. It
can, and does, fail on CPython as well, as it is sensitive to memory
allocation details. See for example:
http://utcc.utoronto.ca/~cks/space/blog/python/ExaminingStringConcatOpt
and here for a cautionary tale about what can happen when the optimization fails
under CPython:
http://mail.python.org/pipermail/python-dev/2009-August/091125.html
Is that the right link? This thread doesn't mention +=, the bug
mentioned in the first email doesn't mention +=, and the fix mentioned
for that bug appears to be "let's not pass 0 as the buffer size of
sock.makefile()".
Did I skim too much?
Yes you skimmed too much :-)
The point is that use of += can cause *platform specific* O(N**2) performance in things
which don't obviously involve string concatenation. It would be nice if real world bugs
were as simple as "I concatenate a lot of strings with += and it's slow, how do I
fix it?" but in this case the bug report was that httplib was an order of magnitude
or more slower on Windows than Linux.
The thread continues into the following month. Here's the first pointer to the
problem:
http://mail.python.org/pipermail/python-dev/2009-September/091582.html
and a note that platform differences in realloc matter:
http://mail.python.org/pipermail/python-dev/2009-September/091583.html
and solution:
http://mail.python.org/pipermail/python-dev/2009-September/091588.html
--
Steven
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
