On Tue, Dec 1, 2009 at 11:42 AM, Kjetil Torgrim Homme <[email protected]> wrote: > Peter Tribble writes: >> It's probably fair to say that sar is heavily used in some >> organizations (and less so in others). We collect it anyway, so I've >> been doing some data gathering based on it, which exposes some of its >> limitations: >> >> 1. No networking. This is an absolute killer. The concept of >> networking seems to have been somewhat, ahem, neglected. > > I just have to ask, have you looked into statistics collectors like > Munin? http://www.munin-monitoring.org
Oh yes. I think that I'm looking for something much more basic and lower in the stack. But you could easily imagine munin (and any of that class of tools) being able to chew through the saved data. >> What if we just scrapped the current sar collector (sadc) and just >> saved kstat -p output (or something like it) instead? > > Munin wouldn't be a replacement for that, the plugins are pretty > hardcoded in what is graphed, although they are easy to extend. Getting rid of the hard-coding is one thing I'm really keen on getting away from. >> Is this practical? How big is the data? >> >> On one of my machines, a thor, a regular sar datapoint is about 64k. >> The kstat -p output is about a meg, but zip gets that down to 160k, and >> 7z does significantly better. On a T5140, the sar data is about 20k and >> kstat about 2 meg, so there the difference is much larger. But I feel the >> idea has promise - in the best case we're almost there already, without >> optimizing the saved format or trimming unnecessary data. > > on a random machine of mine, I got 980k raw and 137k compressed > (gzip -9). collecting this data every 5 minutes from, say, a > medium-sized installation of a hundred hosts, you need 4 GB storage per > day or 1.4 TB per year. Seems about right. That's what my current sar archive is heading towards. > I think that is impractically large. Munin, Orca, Cacti all use RRD to > consolidate data, so that sample rate decreases with age. would it be > possible to do the same with kstat output? my guess is that it is > difficult without knowing a priori what kind of value we're > consolidating (is it a counter or a rate?). many values are neither, > complicating a general collector further. It depends what you do with it. Consider something like current sar usage, then you just keep the base data for 10 days or so. So you can do arbitrary analysis back for 10 days. You could keep summaries beyond that, or import it into other tools, or just keep rrd-style data. Personally, I wouldn't be overly concerned with keeping the raw data for a year - it really isn't completely impractical. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ _______________________________________________ sysadmin-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/sysadmin-discuss
