This is an interesting dilemma. I've had a lot of thoughts about this since I started replying to your mail.
One of the main features of rrdtool is its ability to consolidate the data over a period of time. Or age out the data if you decide not to use RRA's. You have to decide ahead of time the maximum amount of data you are ever going to want to store there. It seems from your email that you are planning to WORK AROUND that feature and use rrd files as an alternative to your mysql database. For instance, you want to be able to see a graph of 90 days of data that reflects the data in your mysql database. Is this correct? If this is true, and you decide to use pre-created rrd files, it seems to me you will have to create rrd files large enough to store the maximum amount of data anyone could ever want to see a graph of. If this is a possibility in your environment, then I think suggestion 2 is the way to go. Or even better, if you can save the pmacct data in the rrd files as you collect it--at the same time you store it in your rdbms. I don't know how you get the data into mysql (not familiar with pmacct), but if it is put there via a script you wrote, you can also write into it an rrdtool "adapter" that puts the same data into rrd files at the time it is collected. If you cannot predict how much data someone is going to need (for instance you never know if someone is going to want to view a graph with 2 years' worth of data) then you have no idea how large to create your rrdfiles and you have to somehow go with #1. Though I don't see how to get around the performance issue there. There is not some other tool that extracts data from a database and generates a graph directly from this data? Though I agree rrdtool has excellent graphing ability so I can see why you would want to leverage that. Anyway, I'd be interested to hear how you resolve this. Tanya Ruttenberg - RSIS Contractor OTSO/DNE/NMPEB [EMAIL PROTECTED] 410-965-9605 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kenneth Kalmer Sent: Tuesday, January 10, 2006 6:15 AM To: [email protected] Subject: [rrd-users] Strategies for data storage and graphing Guys This is my first post to the list, please excuse if me if it's not up to scratch... I currently collect network accounting data using pmacct and save it to MySQL. Due to the nature of the network we need to keep a massive amount of data for various reporting. I've been playing with rrdtool and a sample of the data with great results. I'd like to know what would be the best way to populate the rrd's from the data in the database to create graphs. I know there are plenty of ways, but speed is number one here. I'm also positive that I'm not first one facing this problem... Currently I'm faced with two possibilities, and would like any suggestions on better ways of doing this. 1) Create the rrd, populate and graph in one smooth step. Works fine, but to populate 90 days worth of 300 second step data can take up to 5 minutes... Not desirable... 2) At set intervals, simply populate pre-created rrd's with the new data since the last update, and when needed to graph add the remaining data on-demand... Keeping track of the updates won't be difficult... Haven't built this yet, but in theory it should yield much better performance. It boils down to needing rrd's excellent graphing capabilities, as well as blistering speed when generating reports. This must be the same problem the developers of cacti faced with all the data in MySQL, and rrdtools graphing... Speaking of which, I'm about to tear up the cacti code looking for insight, but any advise would be appreciated... Best -- Kenneth Kalmer [EMAIL PROTECTED] [EMAIL PROTECTED] stats http://fah-web.stanford.edu/cgi-bin/main.py?qtype=userpage&username=kenneth% 2Ekalmer -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdmin http://lists.ee.ethz.ch/lsg2.cgi -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdmin http://lists.ee.ethz.ch/lsg2.cgi
