Mark Plaksin said: > Who has the most RRD files (or total data sources in RRDs)? How frequently > are they updated? What do you use to update them?
At my last company(just left it) I made a custom RRD system that had about 20-30k of RRD files spread accross 8 servers, each datafile was updated at least once per minute, there was about 5k that were updated more often than that(ranging from 5 seconds to 45 seconds). The servers were taxed quite hard, the systems were not very responsive on the command line. system specs for the latest round of systems was: Dual 3Ghz Xeon 1MB cache 4GB memory 8x74GB 10k RPM Raptor disks in RAID10 (3ware raid) 7x74GB 10k RPM Raptor disks in RAID5 (3ware raid) Debian Sarge(3.1) 2.4.31 kernel (these servers are redundant, 2 systems monitoring the same datapoints) (these systems struggled with disk I/O with about 5k files updating every minute, load was typically in the 5.0-7.0 range, didn't appear to impact end user experience using rrdcgi but using the shell was quite painful) > > We are using a home-grown system to update over 20,000 RRD files. Each RRD > file has a single data source. RRD files are using a total of 45G of disk > space. my datafiles consumed around 1.3TB of space I think. My standard was to keep 2 years worth of data so the typical datafile size was around 65MB. I am sure it wasn't the best way to store the data but I couldn't find any real clear easy to understand docs on how to store stuff in RRAs so I just stored every data point and never went back and re-evaluated things. I created: RRA:(MIN|AVG|MAX|LAST):0.5:1:1051200 > Clearly we're doing something wrong :) I'm looking for pointers about > making our system more efficient. While it is possible that our machine > has a poor IO system, let's pretend it doesn't and just talk about the most > efficient way to get data into a large number of RRD files. We're > investigating the IO system separately. > > Besides the size of the RRD files, does the number of rows per RRA impact > performance? I asked a similar question a few months back (9/17/2005 - "scaling RRDTool to the sky..and beyond") and did not get any responses(that I noticed). What sort of hardware are you running? nate -- Unsubscribe mailto:[EMAIL PROTECTED] Help mailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdmin http://lists.ee.ethz.ch/lsg2.cgi
