Magnus Lie Hetland wrote:
> Yeah, one-dimensional data -- simply a list of ints.
>
> I'll just go for a Python list for now, then, and look into it if
> performance becomes a problem. (It's not going to be a long list, but
> it may be traversed hundreds of thousands of times, so who knows... :->)
Funny you should write this. I was just composing a note in my head
about a proposal for a numpy "accumulating array". This idea was
inspired by a discussion at SciPY, in which we spent a LOT of time
during the numpy tutorial talking about how to accumulate values in an
array when you don't know how big the array needs to be when you start.
The "standard practice" is to accumulate in a python list, then convert
the final result into an array. This is a good idea because Python lists
are standard, well tested, efficient, etc.
However, as was pointed out in that lengthy discussion, if what you are
doing is accumulating is a whole bunch of numbers (ints, floats,
whatever), or particularly if you need to accumulate a data type that
plain python doesn't support, there is a lot of overhead involved: a
python float type is pretty heavyweight. If performance or memory use is
important, it might create issues.
What I have in mind is very simple. It would be:
- Only 1-d
- Support append() and extend() methods
- Support any valid numpy dtype
- which could even get you pseudo n-d arrays...
- maybe it would act like an array in other ways, I'm not so sure..
It could be written in pure python, using np.resize()(or concatenate,
or,...), and certainly written efficiently in Cython or even (no!) C.
Thoughts?
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
[email protected]
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
