On 07/03/07, Daniel Mahler <[EMAIL PROTECTED]> wrote: > My problem is not space, but time. > I am creating a small array over and over, > and this is turning out to be a bottleneck. > My experiments suggest that problem is the allocation, > not the random number generation. > Allocating all the arrays as one n+1 dim and grabbing rows from it > is faster than allocating the small arrays individually. > I am iterating too many times to allocate everything at once though. > I can just do a nested loop > where create manageably large arrays in the outer > and grab the rows in the inner, > but I wanted something cleaner. > Besides, I thought avoiding allocation altogether would be even faster.
This doesn't make much practical difference, but allocation is definitely not the only difference between the two situations you describe - if the random number generator function does some significant setup or teardown, you are changing that as well. In fact, modern malloc()s are extremely fast on average (vastly faster than filling any significant amount of memory), so I would hesitate to blame malloc(). Creating numpy arrays, though, could easily be taking the extra time, though depending on how you extract the rows you may be creating just as many array objects. None of this answers your real question, of how to speed up your program. I will observe, though, that there are many places in numpy where a compromise between a simple for loop over all elements and a single operation on giant arrays is faster than either extreme; this is what numexpr is for, for example. Anne M. Archibald _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion