Re: Psycho question
On Aug 8, 7:18 pm, David C. Ullrich [EMAIL PROTECTED] wrote: The one thing that puzzles me about all the results is why // is so much slower than / inside that Psyco loop. Just an oversight. The optimization about '/' between integers was not copied for the case of '//' between integers. Fixed in the svn head :-) Armin -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
On 8 Aug, 20:36, John Krukoff [EMAIL PROTECTED] wrote: One possibility for the performance difference, is that as I understand it the psyco developer has moved on to working on pypy, and probably isn't interested in keeping psyco updated and optimized for new python syntax. More here on the current state of play with Psyco: http://www.europython.org/Talks%20and%20Themes/Abstracts#53 Somebody correct me if I'm wrong, but last I heard there's no expectation of a python 3.0 compatible version of psyco, either. I doubt it is what the presentation referenced above says on the matter. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: John Krukoff: One possibility for the performance difference, is that as I understand it the psyco developer has moved on to working on pypy, and probably isn't interested in keeping psyco updated and optimized for new python syntax. Somebody correct me if I'm wrong, but last I heard there's no expectation of a python 3.0 compatible version of psyco, either. But for me on the short term Python 3 is probably more important than pypy, and I'd like to keep using Psyco... I feel the same way. Maybe someone will do it... (I wonder how much work it would be to make something like Psyco that only accepts a small subset of the language.) Bye, bearophile -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], Delaney, Timothy (Tim) [EMAIL PROTECTED] wrote: David C. Ullrich wrote: f: 0.0158488750458 g: 0.000610113143921 h: 0.00200295448303 f: 0.0184948444366 g: 0.000257015228271 h: 0.00116610527039 I suspect you're hitting the point of diminishing returns with g, and any further investigations into optimisation are purely for fun and learning ;) No doubt. Hadn't meant to get into optimization, at least not here, but various people made various comments - when someone suggests this or that seems like I should try it. Curiously smug grin g is exactly how I'd planned on doing it before trying anything. The one thing that puzzles me about all the results is why // is so much slower than / inside that Psyco loop. Tim Delaney -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
On Fri, 2008-08-08 at 12:18 -0500, David C. Ullrich wrote: Curiously smug grin g is exactly how I'd planned on doing it before trying anything. The one thing that puzzles me about all the results is why // is so much slower than / inside that Psyco loop. Tim Delaney One possibility for the performance difference, is that as I understand it the psyco developer has moved on to working on pypy, and probably isn't interested in keeping psyco updated and optimized for new python syntax. Somebody correct me if I'm wrong, but last I heard there's no expectation of a python 3.0 compatible version of psyco, either. -- John Krukoff [EMAIL PROTECTED] Land Title Guarantee Company -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
John Krukoff: One possibility for the performance difference, is that as I understand it the psyco developer has moved on to working on pypy, and probably isn't interested in keeping psyco updated and optimized for new python syntax. Somebody correct me if I'm wrong, but last I heard there's no expectation of a python 3.0 compatible version of psyco, either. But for me on the short term Python 3 is probably more important than pypy, and I'd like to keep using Psyco... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], Erik Max Francis [EMAIL PROTECTED] wrote: David C. Ullrich wrote: Thanks. I would have guessed that I'd want low-level style code; that's the sort of thing I have in mind. In fact the only thing that seems likely to come up right now is looping through an array of bytes, modifying them. The plan is to use the array module first to convert a string or a list to an array, outside the accelerated part, then maybe do something like for j in range(len(bytes)/3): g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3 bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g If len(bytes) is large, you might want to use `xrange`, too. `range` creates a list which is not really what you need. I didn't follow the explanation, but I read in the docs that xrange can actually be slower under Psyco. This morning I learned that my guess that array.array was a good idea was correct: When I pass a list of ints to the routine above it gets accelerated by a factor of between 10 and 15, while if I pass an array it's closer to 50. This is so cool. Maybe I already said that. -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
On Aug 6, 8:52 pm, David C. Ullrich [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: David C. Ullrich: Thanks. If I can get it installed and it works as advertised this means I can finally (eventually) finish the process of dumping MS Windows: the only reason I need it right now is for the small number of Delphi programs I have for which straight Python is really not adequate. Been not looking forward to learning some C or Objective C (or whatever that Mac thing is) - if I can just accelerate a few Python routines that'll be great. To have better performance with Psyco you need low-level style code, generally not lazy, etc, and adopt some programming conventions, so you may have to rewrite your routines for max speed. Thanks. I would have guessed that I'd want low-level style code; that's the sort of thing I have in mind. In fact the only thing that seems likely to come up right now is looping through an array of bytes, modifying them. The plan is to use the array module first to convert a string or a list to an array, outside the accelerated part, then maybe do something like for j in range(len(bytes)/3): g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3 bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g then convert back to a list or string or whatever outside the accelerated function. [snip] A couple of points: 1. '/' with ints in Python 2.x returns an int, but from Python 3.x it'll return a float. You're recommended to use '//' for int division. 2. 'range' can accept a step value, so you can rewrite that as: for j in range(0, len(bytes), 3): g = (bytes[j] + bytes[j+1] + bytes[j+2])//3 # I think you also want // here bytes[j] = bytes[j+1] = bytes[j+2] = g -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article
[EMAIL PROTECTED],
MRAB [EMAIL PROTECTED] wrote:
On Aug 6, 8:52 pm, David C. Ullrich [EMAIL PROTECTED] wrote:
In article
[EMAIL PROTECTED],
[EMAIL PROTECTED] wrote:
David C. Ullrich:
Thanks. If I can get it installed and it works as advertised
this means I can finally (eventually) finish the process of
dumping MS Windows: the only reason I need it right now is for
the small number of Delphi programs I have for which straight
Python is really not adequate. Been not looking forward to
learning some C or Objective C (or whatever that Mac thing
is) - if I can just accelerate a few Python routines that'll
be great.
To have better performance with Psyco you need low-level style code,
generally not lazy, etc, and adopt some programming conventions, so
you may have to rewrite your routines for max speed.
Thanks. I would have guessed that I'd want low-level style code;
that's the sort of thing I have in mind. In fact the only thing
that seems likely to come up right now is looping through an
array of bytes, modifying them. The plan is to use the array
module first to convert a string or a list to an array, outside
the accelerated part, then maybe do something like
for j in range(len(bytes)/3):
g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3
bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g
then convert back to a list or string or whatever outside
the accelerated function.
[snip]
A couple of points:
1. '/' with ints in Python 2.x returns an int, but from Python 3.x
it'll return a float. You're recommended to use '//' for int division.
2. 'range' can accept a step value, so you can rewrite that as:
for j in range(0, len(bytes), 3):
g = (bytes[j] + bytes[j+1] + bytes[j+2])//3 # I think you also
want // here
bytes[j] = bytes[j+1] = bytes[j+2] = g
Not the issues I expected to be worrying about here, but thanks.
Of course the range(0, len(bytes), 3) is more elegant, and
it's probably faster in Python, but curiously it's much
slower under Psyco! Otoh xrange(0, len(bytes), 3) becomes
pretty fast again. So I conjecture that Psyco compiles
for j in range(l) just as a loop but actually constructs
an array for range(0, l, step).
Also very curiously, // inside the loop is much slower than
/ here (under Psyco). This one I'm not going to guess why...
Honest:
Ah - psyco does work with exec if the import psyco,
etc is inside the code being executed (right now it's
in its own namespace, hence a fresh import each time
- check whether this works with exec in default
namespaces).
Ie, this script works fine in DUShell:
from psyco import proxy, bind
def f(b):
for j in range(len(b)/3):
i = 3*j
g = (b[i] + b[i+1] + b[i+2])/3
b[i] = b[i+1] = b[i+2] = g
g = proxy(f)
def h(b):
for j in range(0,len(b),3):
#for i in range(len(b)/3):
#j = 3*i
g = (b[j] + b[j+1] + b[j+2])//3
b[j] = b[j+1] = b[j+2] = g
bind(h)
from time import time
fs = {'f':f, 'g':g, 'h':h}
def t(f,b):
F = fs[f]
st = time()
F(b)
et = time()
print %s: %s % (f, et-st)
b = range(3)
from array import array
c = array('i',b)
t('f',c)
t('g',c)
t('h',c)
t('f',c)
t('g',c)
t('h',c)
outputs
f: 0.0158488750458
g: 0.000610113143921
h: 0.00200295448303
f: 0.0184948444366
g: 0.000257015228271
h: 0.00116610527039
--
David C. Ullrich
--
http://mail.python.org/mailman/listinfo/python-list
RE: Psycho question
David C. Ullrich wrote: f: 0.0158488750458 g: 0.000610113143921 h: 0.00200295448303 f: 0.0184948444366 g: 0.000257015228271 h: 0.00116610527039 I suspect you're hitting the point of diminishing returns with g, and any further investigations into optimisation are purely for fun and learning ;) Tim Delaney -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], Erik Max Francis [EMAIL PROTECTED] wrote: David C. Ullrich wrote: Just heard about Psycho. I've often wondered why someone doesn't make something that does exactly what Psycho does - keen. Silly question: It's correct, is it not, that Psycho doesn't actually modify the Python installation, except by adding a module or two (so that code not using Psycho is absolutely unaffected)? That's correct. Hi, David! Thanks. If I can get it installed and it works as advertised this means I can finally (eventually) finish the process of dumping MS Windows: the only reason I need it right now is for the small number of Delphi programs I have for which straight Python is really not adequate. Been not looking forward to learning some C or Objective C (or whatever that Mac thing is) - if I can just accelerate a few Python routines that'll be great. Tentatively a very happy camper. See ya. -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
David C. Ullrich: Thanks. If I can get it installed and it works as advertised this means I can finally (eventually) finish the process of dumping MS Windows: the only reason I need it right now is for the small number of Delphi programs I have for which straight Python is really not adequate. Been not looking forward to learning some C or Objective C (or whatever that Mac thing is) - if I can just accelerate a few Python routines that'll be great. To have better performance with Psyco you need low-level style code, generally not lazy, etc, and adopt some programming conventions, so you may have to rewrite your routines for max speed. If some of your routines are too much slow there are many ways in Python to write faster modules, like Cython, Weave, Inline, Swig, SIP, ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D language is good for me (I have tried Pyrex too few times, but I have lost my patience trying to track down in a jungle of ugly auto- generated C code where some reference count updates happen. Writing D code is hugely faster/better for me. Even writing a C extension for Python from scratch may be better for me because there aren't hidden things happening everywhere. I presume other people don't share this problems of mine because there are lot of people using Cython now). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: David C. Ullrich: Thanks. If I can get it installed and it works as advertised this means I can finally (eventually) finish the process of dumping MS Windows: the only reason I need it right now is for the small number of Delphi programs I have for which straight Python is really not adequate. Been not looking forward to learning some C or Objective C (or whatever that Mac thing is) - if I can just accelerate a few Python routines that'll be great. To have better performance with Psyco you need low-level style code, generally not lazy, etc, and adopt some programming conventions, so you may have to rewrite your routines for max speed. Thanks. I would have guessed that I'd want low-level style code; that's the sort of thing I have in mind. In fact the only thing that seems likely to come up right now is looping through an array of bytes, modifying them. The plan is to use the array module first to convert a string or a list to an array, outside the accelerated part, then maybe do something like for j in range(len(bytes)/3): g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3 bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g then convert back to a list or string or whatever outside the accelerated function. Surely something like _that_ is exactly what Psyco is going to do well with, yes? (Ok, we're talking about image processing, in cases where I can't figure out how to get PIL to do whatever directly. So sometimes there will be double loops for row in range(width): for col in range(height): do_something[row*width + col] but at least for the things I can think of right now it shouldn't get much worse than that.) The things you mention below sound very interesting - I'm going to try Psyco first because unless I'm missing something I won't have to learn how to use it. Someday when it turns out to be not good enough I'll be in touch... If some of your routines are too much slow there are many ways in Python to write faster modules, like Cython, Weave, Inline, Swig, SIP, ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D language is good for me (I have tried Pyrex too few times, but I have lost my patience trying to track down in a jungle of ugly auto- generated C code where some reference count updates happen. Writing D code is hugely faster/better for me. Even writing a C extension for Python from scratch may be better for me because there aren't hidden things happening everywhere. I presume other people don't share this problems of mine because there are lot of people using Cython now). Bye, bearophile -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
In article [EMAIL PROTECTED], David C. Ullrich [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: David C. Ullrich: Thanks. If I can get it installed and it works as advertised this means I can finally (eventually) finish the process of dumping MS Windows: the only reason I need it right now is for the small number of Delphi programs I have for which straight Python is really not adequate. Been not looking forward to learning some C or Objective C (or whatever that Mac thing is) - if I can just accelerate a few Python routines that'll be great. To have better performance with Psyco you need low-level style code, generally not lazy, etc, and adopt some programming conventions, so you may have to rewrite your routines for max speed. Thanks. I would have guessed that I'd want low-level style code; that's the sort of thing I have in mind. In fact the only thing that seems likely to come up right now is looping through an array of bytes, modifying them. The plan is to use the array module first to convert a string or a list to an array, outside the accelerated part, then maybe do something like for j in range(len(bytes)/3): g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3 bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g then convert back to a list or string or whatever outside the accelerated function. Surely something like _that_ is exactly what Psyco is going to do well with, yes? teehee. Downloaded Psyco. The install actually worked. Tried exactly what's above with a list of 3 million ints. Didn't time it carefully, seemed to take about two seconds. Ran it again, in case the second run would be faster for some reason. Second was about the same. Said import psyco, etc. Ran the routine again, it returned in _no_ time, perceptually. This is so cool. Gonna find out whether a decorator that returns the accelerated function works, just for the fun of deciding what the name should be: @cool? @wheee? @wow? @dontblinkyoullmissit? (Ok, we're talking about image processing, in cases where I can't figure out how to get PIL to do whatever directly. So sometimes there will be double loops for row in range(width): for col in range(height): do_something[row*width + col] but at least for the things I can think of right now it shouldn't get much worse than that.) The things you mention below sound very interesting - I'm going to try Psyco first because unless I'm missing something I won't have to learn how to use it. Someday when it turns out to be not good enough I'll be in touch... If some of your routines are too much slow there are many ways in Python to write faster modules, like Cython, Weave, Inline, Swig, SIP, ShedSkin, etc. For bioinformatics purposes I have found that Pyd + D language is good for me (I have tried Pyrex too few times, but I have lost my patience trying to track down in a jungle of ugly auto- generated C code where some reference count updates happen. Writing D code is hugely faster/better for me. Even writing a C extension for Python from scratch may be better for me because there aren't hidden things happening everywhere. I presume other people don't share this problems of mine because there are lot of people using Cython now). Bye, bearophile -- David C. Ullrich -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
David C. Ullrich wrote: Thanks. I would have guessed that I'd want low-level style code; that's the sort of thing I have in mind. In fact the only thing that seems likely to come up right now is looping through an array of bytes, modifying them. The plan is to use the array module first to convert a string or a list to an array, outside the accelerated part, then maybe do something like for j in range(len(bytes)/3): g = (bytes[3*j] + bytes[3*j+1] + bytes[3*j+2])/3 bytes[3*j] = bytes[3*j+1] = bytes[3*j+2] = g If len(bytes) is large, you might want to use `xrange`, too. `range` creates a list which is not really what you need. -- Erik Max Francis [EMAIL PROTECTED] http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM, Y!M erikmaxfrancis You and I / We've seen it all / Chasing our hearts' desire -- The Russian and Florence, _Chess_ -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
Erik Max Francis: If len(bytes) is large, you might want to use `xrange`, too. `range` creates a list which is not really what you need. That's right for Python, but Psyco uses normal loops in both cases, you can time this code in the two situations: def foo1(n): count = 0 for i in range(n): count += 1 print count def foo2(n): count = 0 for i in xrange(n): count += 1 print count import psyco; psyco.full() N = 1 #foo1(N) foo2(N) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Psycho question
David C. Ullrich wrote: Just heard about Psycho. I've often wondered why someone doesn't make something that does exactly what Psycho does - keen. Silly question: It's correct, is it not, that Psycho doesn't actually modify the Python installation, except by adding a module or two (so that code not using Psycho is absolutely unaffected)? That's correct. Hi, David! -- Erik Max Francis [EMAIL PROTECTED] http://www.alcyone.com/max/ San Jose, CA, USA 37 18 N 121 57 W AIM, Y!M erikmaxfrancis Longevity has its place. But I'm not concerned about that now. -- Dr. Martin Luther King, Jr. -- http://mail.python.org/mailman/listinfo/python-list
