Hi Paolo, see my answers below.
On Wed, Oct 3, 2012 at 1:05 AM, Paolo Lucente <[email protected]> wrote: >> That means I have a lot of counters in my aggregation: if I have 150 >> members on the IX, I have 150*149*2 (in & out) = roughly 45000 >> counters to compute. > > Speaking of the dimensioning side: what would be retention time you > have in mind for such counters? Ie. how many 5 mins, hourly, daily > counters you would like to keep? This would give an upper bound to > the number of counters to handle. That is a good question. I would say only a few of each (2 to 10 ?), so as to keep memory consumption low, and pmacct stable. Stability is one the strength of pmacct, so let's keep it simple. >> What I'm doing right now is parsing all the counters from the memory >> plugin every 5 mins, and inserting them in RRD databases (classic). >> It's working well so far, but not really efficient when I want to >> present the stats *sorted* to my members, I have to compute data from >> thousands of RRD databases before getting anything relevant. > > I believe the memory plugin + RRD approach is handy as it solves > immediately the presentation part - plenty of pre-cooked tools to > present counters in RRD format. Downside is, as you mention, that > RRD is inefficient for what you want to do. That means scrapping > RRD from the picture - should that happen, do you have already in > mind a way to present data to your customers? I do have that, and it's working in production. Its limited access to our IX members only, but it works quite well, using the isotope.js library for the front-end (http://isotope.metafizzy.co) I can give you an access so you get the idea. The downside to this, as I said before, is that computing everything is very slow. >> What I would appreciate would be to have pmacct compute the usual >> 'daily, week, month, year, whatever' bandwidth rates for me so that I >> would just have to parse this information and then display the graphs >> sorted. > > Consider a viable strategy could be to leverage the fact pmacct can > offer multiple views of the same traffic thanks to its pluggable > architecture - this both on the spatial axis, ie. aggregate input > data differently, and temporal axis, ie. aggregate input data the > same way but at different time resolutions. To me it sounds the > latter is precisely what you want - ie. multiple plugins, your > call: SQL or noSQL, each aggregating data the exact same way but > at different time resolutions, ie. 5 mins, hourly, daily. How do > i find you on this idea? If positive i can draft some pmacct config > snippet for you. Sounds like what I'm looking for. But getting a rate instead of a counter is also important, because this is the information I need. Cheers, Jerome. _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
