On a somewhat related note, I usually find I need to compute stats incrementally. To do this, a stat object is created so batches of samples can be fed to it sequentially.
I used to use an implementation based on boost::accumulator for this. More recently I'm using my own c++ code based on xtensor, exposed to python with xtensor-python and pybind11. The basic technique to find 2nd order stats is to keep 2 running sums, sum(x) and sum(x**2). It would be useful to have functionality for incremental stats like this in numpy, as well as other incremental operations (e.g., histogram). I frequently find I need to process large amounts of data in small batches at a time, generated by iterative monte-carlo simulations, for example.
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
