Phil, A big part of the problem as I understand it is that these rates are needed across a potentially quite large distributed system.
On Mon, Mar 14, 2011 at 1:42 PM, Phil Steitz <[email protected]> wrote: > On 3/14/11 12:34 PM, Luc Maisonobe wrote: > > Le 14/03/2011 15:33, Benson Margulies a écrit : > >> Please excuse the following ignorant question. > >> > >> I want to maintain summary statistics of a rate. At each 'event', I > >> know the number of characters and the time it took to process them, > >> and I want to maintain summary statistics for the rate of > >> chars/second. I imagine that I'm missing something basic, but I don't > >> see how to do this. > > You should define some windows width, either in terms of a time span > > (all events in the last n seconds) or in terms of number of events (last > > n events). > > > > In [math], we do not provide (yet) anything for maintaining such a data > > structure, you'll have to maintain the events in this slot by yourself, > > with something similar to a FIFO. > > I am not sure I understand what the problem is exactly, but if what > you need is simply "rolling" statistics, where a dataset of 0,...,n > values are maintained with the newest values replacing the oldest, > we do in fact support that in > o.a.c.math.descriptive.DescriptiveStatistics. > > Phil > > When you have your data available, each time a new event is added or > > removed from the ones that belong to the window, you can fetch compute > > the statistics you want on this data (min, max, mean, median, standard > > deviation ...) and wait for next addition/removel to recompute it again. > > > > Another thing we discussed some months ago (but did not implement yet) > > is a way to compute an approximation of percentiles in a flow of data > > without storing them. There is an interesting algorithm for it that was > > developed for the needs of telecommunication companies, I think it may > > be of interest to you. This would provide results like : currently 95% > > of the characters are processed in n milliseconds. would you be > > interested in us implementing this feature ? > > > > Luc > > > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
