My install: 800 switches and 700 servers 141,738 rrd files My load average is usually under 4. Most of my scans finish in less than 10 seconds, with a few stragglers. UI is very fast. I still have some blocking issues, i thappens when modeling.
My biggest bottleneck was disk i/o. Updating 140k rrd files took a while and the system was always in a wait state. Using faster/more disks really wasn't an option. Fortunately I have 64GB of ram in the system. If I lose a days worth of data it won't kill me. Here are a few things that sped up the server: Created a ram disk for the rrd files. Every hour tar and zip it to disk. Recreate the ram disk at boot time and load rrd files from backup. Takes a couple extra minutes to boot. Server rarely gets rebooted. ram disk is a LVM volume and a snapshot is taken and that is what gets tarred. Once the majority of the load was taken off of the disk MySQL was a little slower than expected. The status table was converted to a memory table. Once again if I lose some status data to a reboot it is not a deal breaker. Cleaned out devices with errors and very chatty machines. I still have a few slower devices that slow my scans down. rc.local creates the ram disk and copies the rrd files. You need to have a link from /opt/zenoss/perf to /mnt/rrd. snaprrd is run hourly from cron and it creates the snapshot and backup. A few config files: rc.local: ------------------------------------------------------------- blockdev --setra 16384 /dev/cciss/c0d0 #create pv : pvcreate /dev/ram0 /dev/ram1 /dev/ram2 #create vg : vgcreate vg1 /dev/ram0 /dev/ram1 /dev/ram2 #create lv : lvcreate -L 18G -n ramlv vg1 #create fs: mkfs.ext2 /dev/vg1/ramlv #mount lv : mount -o noatime /dev/vg1/ramlv /mnt/rrd #restore rrd cd /mnt/rrd && tar -zxf /opt/zenoss/rrd.tar #start zenoss /etc/init.d/zenoss start ------------------------------------------------------------- snaprrd: ------------------------------------------------------------- #!/bin/bash #logging echo echo "#############################################" date echo "#############################################" #create snapshot: /usr/sbin/lvcreate -L 5G -s -n rrdsnap /dev/vg1/ramlv #mount snapshot: /bin/mount -o noatime /dev/vg1/rrdsnap /mnt/snapshot #logging df -h /mnt/snapshot echo TIME #backup: mv -f /opt/zenoss/rrd6.tar /opt/zenoss/rrd7.tar mv -f /opt/zenoss/rrd5.tar /opt/zenoss/rrd6.tar mv -f /opt/zenoss/rrd4.tar /opt/zenoss/rrd5.tar mv -f /opt/zenoss/rrd3.tar /opt/zenoss/rrd4.tar mv -f /opt/zenoss/rrd2.tar /opt/zenoss/rrd3.tar mv -f /opt/zenoss/rrd.tar /opt/zenoss/rrd2.tar cd /mnt/snapshot && time /bin/tar zcf /opt/zenoss/rrd.tar perf #logging ls -la /opt/zenoss/rrd.tar /usr/sbin/lvdisplay /dev/vg1/rrdsnap /usr/sbin/lvs #remove snapshot: cd && /bin/umount /mnt/snapshot && /usr/sbin/lvremove -f /dev/vg1/rrdsnap ------------------------------------------------------------- Scott -------------------- m2f -------------------- Read this topic online here: http://forums.zenoss.com/viewtopic.php?p=30664#30664 -------------------- m2f -------------------- _______________________________________________ zenoss-users mailing list [email protected] http://lists.zenoss.org/mailman/listinfo/zenoss-users
