Hi all: I'd just like to throw in my $0.02 here... have a look at Gmetad Python in our development branch (trunk):
https://sourceforge.net/apps/trac/ganglia/browser/trunk/monitor-core/gmetad-python This code provides a plugin interface that abstracts out the data storing code so you could write your own plugins to manipulate the metrics data collected however you want. Some examples are storing in RRD, alerting based on certain metric thresholds, storing in MySQL, etc. So you could definitely write a plugin to talk to your Java server and not having to re-write the entire Gmetad framework. We will be releasing this as part of Ganglia 3.2.x, so please stay tuned for that. In the mean time, you can check out the code from our SVN repository, a few folks have already been playing around with it and initial results are quite promising. Cheers, Bernard On Wed, Nov 3, 2010 at 10:55 AM, Rick Cobb <[email protected]> wrote: > You almost certainly don't need to work as hard as you think you do on the > protocol side. > The architecture of Ganglia is very nicely separated between the monitoring > and what I'll call 'local data collection' daemons (gmond), the grid-level > collection daemon (gmetad), and the web front-end. > So what you want to do is write a Java-based equivalent of gmetad, and then > figure out whatever display strategy you want. > Gmetad *polls* the appropriate gmonds (see the documentation for the > 'data_source' configuration parameter for gmetad.conf) every so often (15s > by default). > The protocol for 3.0 is as simple-as-pie: connect with TCP and read until > the gmond closes (shuts down) the socket. I don't think there's a > difference for 3.1, but I haven't coded to that interface. > The payload is an XML document. At least in 3.0, it contained its own DTD. > You should be able to analyze these with just about any XML toolkit. It's > nicely hierarchic, with GRIDs (often null) containing CLUSTERS, which > contain NODEs, which contain METRICs. > The definitions for the fields (especially the time fields) can be a little > odd, but they're still what they were in the original Ganglia paper, > referenced on the wiki. The important ones to be sure you understand are the > time attributes of every metric: Td, Tmax, and Dmax. > One caveat: in 3.0, the XML document would include both the definitions of > the metrics, and their values. It's my understanding (perhaps incorrect) > that in 3.1, the definitions (meta-data) aren't sent every time. You'll > want to read the gmond.conf documentation to identify when it is. > So what your architecture calls for doesn't require a modification of any > Ganglia code at all. It's a replacement of gmetad & the ganglia front-end. > You can use gmond completely unmodified, and I'd recommend that since it > sounds like your actual requirements are to change the metric recording and > presentation layer, not the collection layer. > You do have to sample your gmonds at least as often as the most-sampled > metric, since gmond's don't record any history, just the current value. > Where you will have to work harder than you might think is in recording the > number of diverse metrics in a usably accurate way at speed. Any one XML > document for even a basic cluster (say 50 machines monitoring a few > application metrics in addition to the standard ones) represents parsing and > recording 5-10000 measurements in whatever statistics store you're using, > every 15s (or faster). If you're also summing things up across other > categories (grids, clusters, applications, etc)., you have to deal with > making those updates consistent. RRDtool, when implemented on a RAMdisk or > RRDcache, does a pretty credible job of keeping the statistics working in > the face of missed samples, samples taken a second or two late, etc. Gmetad, > by focusing on a cluster @ a time, avoids having a lot of lock contention > (except when trying to summarize grid metrics out of cluster metrics). > I hope you can come up with a solution that scales that up more and allows > for faster sampling rates; looking forward to your results. > -- ReC > > On Wed, Nov 3, 2010 at 3:50 AM, Afef MDHAFFAR <[email protected]> > wrote: >> >> Hello, >> >> I am trying to use Ganglia for my research works. >> So the idea consists of modifying Ganglia in order to get a new Ganglia >> tool which sends informations (monitored data) via TCP/IP sockets to a Java >> server instead of saving it in a RRD database. >> Can you help me in that way? I need name of files, methods that have to be >> modified to reach this goal. >> Also, I am using Mac OS and I want to import ganglia into Xcode in order >> to compile and build it. Can you please send me a tutorial for that? >> >> Thank you, >> >> Best regards >> >> -- >> Afef MDHAFFAR >> http://www.redcad.org/members/mdhaffar/ >> >> >> >> >> >> >> -- >> Afef MDHAFFAR >> http://www.redcad.org/members/mdhaffar/ >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Achieve Improved Network Security with IP and DNS Reputation. >> Defend against bad network traffic, including botnets, malware, >> phishing sites, and compromised hosts - saving your company time, >> money, and embarrassment. Learn More! >> http://p.sf.net/sfu/hpdev2dev-nov >> _______________________________________________ >> Ganglia-general mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/ganglia-general >> > > > ------------------------------------------------------------------------------ > Achieve Improved Network Security with IP and DNS Reputation. > Defend against bad network traffic, including botnets, malware, > phishing sites, and compromised hosts - saving your company time, > money, and embarrassment. Learn More! > http://p.sf.net/sfu/hpdev2dev-nov > _______________________________________________ > Ganglia-general mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/ganglia-general > > ------------------------------------------------------------------------------ Achieve Improved Network Security with IP and DNS Reputation. Defend against bad network traffic, including botnets, malware, phishing sites, and compromised hosts - saving your company time, money, and embarrassment. Learn More! http://p.sf.net/sfu/hpdev2dev-nov _______________________________________________ Ganglia-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ganglia-general
