People are always worried about ganglia's network or load overhead, writing into the list and honestly I don't understand where this comes from. Have you measured a lot of overhead caused by ganglia? I did a quick check last month when a similar question came up and this is what I found:
We are monitoring almost 2,000 nodes, split into 10 clusters, with no load problems on the gmetad head node using a tmpfs filesystem for the rrds. I should note that we also increased the gmetad polling interval from the default 15 seconds to 60, which helps a little. To prevent loss of data, every night gmetad is stopped, the rrd directory is copied to disk and gmetad is restarted which only takes a few seconds. On the head node, gmetad is reading an average of 250kB/s of XML data total from all 10 of our clusters with our 60 second update interval, which is only about 2% of its available 100Mb/s connection. We also don't notice any problem with multicast, even with our largest cluster of over 400 nodes, we just set a higher ttl and enabled multicast forwarding on our router. We also have igmp enabled on a switched network, so the multicast is only sent where it is supposed to be sent. This allows gmetad to fail-over to another gmond for that cluster's data if the one it was polling does not respond. A quick check with tcpdump shows about 150 packets/sec average on our largest 400 node cluster, which is a little more than 10kB/s or less than 0.1% of the available bandwidth on a 100 Base-T network. The exact packet rate will vary depending on the usage of the nodes, basically how quickly the metric parameters reach their thresholds. We are just using the basic Linux metrics, no extra metrics have been added yet. Do you really need that extra 0.1% of available bandwidth? ~Jason On Wed, 2005-07-13 at 06:38, regatta wrote: > Guys, > > We have a very old bad experience with ganglia (I think 2.5) and we > installed it in around 26 clusters with 128 nodes and as far as we > know it generate alot of traffic in the network and use alot of CPU > resource in the management node so we had to disable it since we need > the network bandwidth > > Anyway today we have a new oracle cluster with around 48 nodes and we > want to ask you how do you think ganglia 3 ? does it produce alot of > network traffic, does it use the CPU alot ? did anyone use it with > oracle cluster ? > > give me your feedback please > > Thanks > >
