hey folks, I've been using RRDTool for about 2 years now, and make pretty extensive use of it these days. I'm not a programmer so my abilities are limited but I am doing all the data work using the RRDs perl module and rrdcgi for the graphing.
To make adding databases easily I store each data point in it's own file. I am also interested in keeping every point of data, I was never really able to figure out the RRAs so I basically tell it to store everything for MIN/MAX/AVG/LAST, which may be overkill, not sure. anyways as I said I have a lot of RRD databases, around 15,000 at this point and they are updated at LEAST once per minute. My servers struggle at around 80 updates per second. Also I have around 300,000 lines of rrdcgi templates. I am curious if anyone else out there runs largish scale RRDs with lots of data points being collected with frequent updates and what kind of hardware they have. I've found that it's entirely I/O at this point. CPU stays 99% idle. I have 8 servers, the most recent ones are the most powerful: Dual 3Ghz Xeon 1MB cache 4GB memory 8x74GB 10k RPM Raptor disks in RAID10 (3ware raid) 7x74GB 10k RPM Raptor disks in RAID5 (3ware raid) Debian Sarge(3.1) 2.4.31 kernel (these servers are redundant, 2 systems monitoring the same datapoints) I get get as high as around 5,000 data updates a minute but the box itself is almost unusable on the command line, it can go 10-15 seconds while it waits on i/o. this doesn't seem to adversely impact the generation of the graphs all that much. I'm using RRTool 1.0.42 I believe, I've been meaning to upgrade but I have to go back and adjust all my cgis since I use some options that cause newer versions(up to 1.0.48 at least) to generate errors, and with my frantic work weeks i haven't had the time. just lookin for stories/experiences with high load RRD usage. Maybe there is something I can do to improve performance without having to get a rack full of disks :) I've already cut back on the amount of data points being collected and the frequency of updates. At this rate it is completely possible by this time next year I will have upwards of 50,000 RRD databases. How well does rrdtool work with NFS? some of my newer systems will be backed by NetApps, I tried running RRDTool over NFS(linux-to-linux) with full locking enabled but it didn't work. it said it updated successfully but all of the datapoints were 0. Didn't spend much time researching why at the time. I don't have any NetApps yet to test. All of my current systems write to local disk. thanks nate (I do plan to release my work to the world at some point once I can get it cleaned up, maybe even write a good HOWTO for RRDTool like I did with MRTG) -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdmin http://lists.ee.ethz.ch/lsg2.cgi
