Alexander Michael wrote: > I'm new to numpy and looking for advice on setting up and managing > array data for my particular problem. I'm collecting observations of P > properties for N objects over a rolling horizon of H sample times. I > could conceptually store the data in three-dimensional array with > shape (N,P,H) that would allow me to easily (and efficiently with > strided slices) compute the statistics over both N and H that I am > interested in. This is great, but the rub is that H, an interval of T, > is a rolling horizon. T is to large to fit in memory, so I need to > load up H, perform my calculations, pop the oldest N x P slice and > push the newest N x P slice into the data cube. What's the best way to > do this that will maintain fast computations along the one-dimensional > slices over N and H? Is there a commonly accepted idiom? > > Fundamentally, I see two solutions. The first would be to essentially > perform a memcpy to propagate the data. The second would be to manage > the N x P slices as H discontiguous memory blocks and merely reorder > the pointers with each new sample. Can I do either of these with > numpy?
The 'pointers reordering' can be very nicely done via deque from collections module - it behaves like a list, but has some additional methods like rotate(). I have used it successfully as a circular buffer for numpy arrays, but I can see that you need an efficient slicing over H, so a contiguous memory would be better for you IMHO. r. _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion