Hi Jerome, Pity we didn't have opportunity to speak more about the topic, very interesting!, at RIPE. Here i'm with some thoughts about it, please see inline:
On Tue, Sep 25, 2012 at 03:42:10PM +0200, J?r?me Fleury wrote: > That means I have a lot of counters in my aggregation: if I have 150 > members on the IX, I have 150*149*2 (in & out) = roughly 45000 > counters to compute. Speaking of the dimensioning side: what would be retention time you have in mind for such counters? Ie. how many 5 mins, hourly, daily counters you would like to keep? This would give an upper bound to the number of counters to handle. > What I'm doing right now is parsing all the counters from the memory > plugin every 5 mins, and inserting them in RRD databases (classic). > It's working well so far, but not really efficient when I want to > present the stats *sorted* to my members, I have to compute data from > thousands of RRD databases before getting anything relevant. I believe the memory plugin + RRD approach is handy as it solves immediately the presentation part - plenty of pre-cooked tools to present counters in RRD format. Downside is, as you mention, that RRD is inefficient for what you want to do. That means scrapping RRD from the picture - should that happen, do you have already in mind a way to present data to your customers? > What I would appreciate would be to have pmacct compute the usual > 'daily, week, month, year, whatever' bandwidth rates for me so that I > would just have to parse this information and then display the graphs > sorted. Consider a viable strategy could be to leverage the fact pmacct can offer multiple views of the same traffic thanks to its pluggable architecture - this both on the spatial axis, ie. aggregate input data differently, and temporal axis, ie. aggregate input data the same way but at different time resolutions. To me it sounds the latter is precisely what you want - ie. multiple plugins, your call: SQL or noSQL, each aggregating data the exact same way but at different time resolutions, ie. 5 mins, hourly, daily. How do i find you on this idea? If positive i can draft some pmacct config snippet for you. Cheers, Paolo _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
