Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hej
Tobias Oetiker wrote: > Yesterday you sent me mail regarding [rrd-users] VISIONARY: New tool to...: > > JI> So what am I suggesting? It's very short to express but probably not > JI> trivial to achieve: write a tool which efficiently and accurately enough > JI> might store, process and graph this kind of dynamicly changing data sets > JI> and which supports the data consolidation over time MRTG/RRDtool uses > JI> (store 5 min samples for 48 hours, store 30 min samples for 2 weeks > JI> etc).. > > doing this with rrdtool should not be too difficult ... think about > a directory tree with 3 levels ... each dir represents a byte of > the contributing hosts IP address the last byte woud be the file > name ... you can create rrd files as you go along ... Well, if I do this on an intranet WAN link going from the Americas to EMEA (Europe, Middle East, Africa) and we consider a large enterprise intranet, I will have a HUGE number or rrd files being created as I have a large number of destination hosts perhaps over 10 000. But if I try to do some trade-off even before I start writing to the RRD:s... OK, for each 5 minute period, process all the destination-datacount pairs so that hosts which contribute to less than, say, 1% of the total amount of traffic during those 5 minutes are put into the "low contributor RRD". Then I'll only have a maximum of 100 rrd files to access/create. Problem might be though, that the next 5 minutes, another large set of hosts might go above the "low contributor" threshold and thus warrant to have RRD-files of their own. Another tradeoff could be to forget about 5 minute sample periods and instead use 30 minute studies at the start. > > my question would rather be, what reports do you want to generate > from this data ... this will be the real challenge ... do you want Well, if we extend the scope from not just destinations for traffic, I want the tool to produce graphs with the same information as those produced by the Netmetrix product, but better (of course ;-). Netmetrix provides you with statistics for a link such as graphs showing the distribution of: * protocol usage during the day * top talkers during the day * top listeners during the day * top conversation during the day. (It also provides utilization graphs but that's not very intresting as RRDtool VERY easily store and graphs such statistics, as we all know). On each graph, the legend contains colors/patterns to distinguish the topmost candidates for each category. You can configure the reports to provide you with top 10 talkers or top 5 talkers or top 40 talkers etc... any number you like (too many messes up the graph, too few makes you miss some of the host). As you can tell, the legend and the color scheme cannot be decided on advance, as you don't know which hosts/protocols/conversations will be the top 10 (or 5 or 40) contributers to the traffic. It has to be decide upon rendering the image. > > to store the whole matrix ... source AND destination ? this would > make it probably unpractical with rrdtool, you might want to use a But still RRDtool has some intresting features I really do like and which I would like to see in the new tool and I think borrowing some design and code (if you don't mind of course) could be a good idea. > > SQL database (postgresql) for this ... but again, the question what Well, you could use a SQL database (like Oracle for instance) to actually store the data, but still you could use a tool similar to rrdtool to create the data collections, store and retrieve the data. Actually, you could have something very similar to an RRD but instead of storing values of how much utilization was inbound at each sample period, you store a reference to a data structure which contains what actually was seen during that sample period. (When we talk about SQL databases, you could use such a one for RRDtool as well if you want to). > > are you going to do with the data ? > Ok, here's an example.Well, consider the case you have a WAN link connecting an subsidiary to the rest or the corporate intranet. Suddenly some PC user decides to do some gigantic download. You want the graphs to display not only that the link is near 100% utilized inbound, you also want to see in some graphs WHO is using the major part of the link. Another example is if you have a number of "favourite download servers" somewhere on the intranet, and they become the source of the major portion of traffic traversing the link, you want to the stats to indicate that (so you can consider moving these servers or perhaps start replicate data to local download servers.) The two cases above does imply that not ALL the statistics about traffic over a link is used, we are mainly intrested in what are the biggest contributors. Which might imply that we may do trade-offs in statistic gathering and processing. > cheers > tobi > > -- > ______ __ _ > /_ __/_ / / (_) Oetiker, Timelord & SysMgr @ EE-Dept ETH-Zurich > / // _ \/ _ \/ / TEL: +41(0)1-6325286 FAX:...1517 ICQ: 10419518 > /_/ \.__/_.__/_/ [EMAIL PROTECTED] http://ee-staff.ethz.ch/~oetiker -- (Jakob Ilves) <[EMAIL PROTECTED]> {Oracle Global IT, Network Management Group} [Office as well as mobile phone: +46/8/477 3666 | Fax: +46/8/477 3572] - Intranet Home Page: http://jilves.se.oracle.com - -- Attached file removed by Listar and put at URL below -- -- Type: text/x-vcard -- Desc: Card for Jakob Ilves -- Size: 444 bytes -- URL : http://www.ee.ethz.ch/~slist/pantomime/20-jakob.ilves.vcf -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://www.ee.ethz.ch/~slist/rrd-users WebAdmin http://www.ee.ethz.ch/~slist/lsg2.cgi
