Re: numarray.array can be VERY slow
Edward C. Jones wrote: Steven Bethard wrote: As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) The code I posted was simplified from a larger program which I have now revised. But I still ask: why did it take 4.3 seconds to run? Is the simple answer not that repeated use of array.append is inefficient as python has to repeatedly re-allocate memory for the array? I had a similar problem as I ran a simulation and appended the results to a results set. Now I create a list of objects of the size required and the overwrite them. Much faster :) Jim Getting in there late cos I've been skiving -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Jim wrote: Edward C. Jones wrote: Steven Bethard wrote: As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) The code I posted was simplified from a larger program which I have now revised. But I still ask: why did it take 4.3 seconds to run? Is the simple answer not that repeated use of array.append is inefficient as python has to repeatedly re-allocate memory for the array? No, because he wasn't appending to arrays, he was appending to lists (each only 8 items long, so there really is no significant overhead). See my response in this thread for the answer. -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
numarray.array can be VERY slow
#! /usr/bin/env python Should this program take 4.3 seconds to run? I have a Linux PC with an AMD Athlon XP 2000+ chip (1.7Gh). I use Python 2.4.1 and numarray 1.2.3, both compliled from source. import time, numarray from numarray.numerictypes import * nested = [] for i in range(8): inner = [] for j in range(8): inner.append(numarray.ones((256,256), Float64)) nested.append(inner) t = time.clock() arr = numarray.array(nested) print time.clock() - t -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Yes because that is a bad way to things. There is no reason to be working with a list when it could be done directly with numarray. -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Edward C. Jones wrote: #! /usr/bin/env python Should this program take 4.3 seconds to run? I have a Linux PC with an AMD Athlon XP 2000+ chip (1.7Gh). I use Python 2.4.1 and numarray 1.2.3, both compliled from source. import time, numarray from numarray.numerictypes import * nested = [] for i in range(8): inner = [] for j in range(8): inner.append(numarray.ones((256,256), Float64)) nested.append(inner) t = time.clock() arr = numarray.array(nested) print time.clock() - t As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) ?? STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Steven Bethard wrote: As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) The code I posted was simplified from a larger program which I have now revised. But I still ask: why did it take 4.3 seconds to run? -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Edward C. Jones wrote: Steven Bethard wrote: As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) The code I posted was simplified from a larger program which I have now revised. But I still ask: why did it take 4.3 seconds to run? And I repeat, because you were using lists, not numarray. ;) If you want a numarray array, create it directly, don't create it from lists. What's your real code look like? Are you sure you need two nested loops? If you're sure, can you do something like import numarray as na arr = na.zeros((8, 8, 256, 256)) for i in xrange(8): for j in xrange(8): arr[i,j] = na.ones((256, 256)) where you allocate the array first and then fill it? STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray.array can be VERY slow
Edward C. Jones wrote: Steven Bethard wrote: As mentioned, this has nothing to do with numarray, and everything to do with your inexplicable use of lists. Why don't you just write this as: arr = numarray.ones((8, 8, 256, 256), Float64) The code I posted was simplified from a larger program which I have now revised. But I still ask: why did it take 4.3 seconds to run? Because lists of lists of arrays are difficult to work with. Lists of arrays are relatively easy to work with: numarray checks to see if all the items are compatible arrays. If they are, then the arrays are all concatenated and you have your final array fairly quickly. List of lists of arrays are more problematic. Currently, the code doesn't check that that the objects it sees are arrays, so it interprets the arrays as sequences of sequences, and uses the PySequence_* protocol. So you might as well have used list of lists of lists of lists. See the function setArrayFromSequence in libnumarraymodule.c . If you wish to fix this behavior, I'm sure that a patch would be welcomed. Further questions should probably go to the numarray mailing list. -- Robert Kern [EMAIL PROTECTED] In the fields of hell where the grass grows high Are the graves of dreams allowed to die. -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
