I hear what you're saying about using the API, but collectl is a perl script (I know, it's hard to believe it can be that efficient but it really is) and the effort to call those routines would be a lot of work and it's still not clear to me how much interest there would even be in this. As I said in my base note, at one level you could just put collectl on the same machines as each gmond and only have it send the data to the gmond that you're interested in as it does offer a wealth of data.
Do YOU have any interest in trying it out? ;-) It's literally a matter of installing the collectl rpm, which has no dependencies other than perl, me sending you a fairly short script and then you type a collectl command and it starts sending UDP gmond packets to whatever address:port you tell it you. -mark |-----Original Message----- |From: Brad Nicholes [mailto:bnicho...@novell.com] |Sent: Tuesday, March 31, 2009 7:42 PM |To: Seger, Mark; ganglia-developers@lists.sourceforge.net |Cc: Evan J Felix |Subject: RE: [Ganglia-developers] Building a ganglia interface into |collectl | |The only thing that I would suggest is that you use the APIs to create |and send the packets. The problem with hand crafting the packets is |that if the gmond XDR packet definition ever changes, your interface |will be broken. I don't foresee the XDR packet definition change very |frequently, hopefully not in the near future since it has been crafted |to be expandible, but it could cause you problems if it does. Basically |all you would need to do is either dynamically link the ganglia library |or use dlopen() to look for it. If it succeeds then use dlsym() to |import the functions and start calling the APIs. The downside is that |you would probably have to deploy the ganglia library yourself since you |are trying to use collectl rather than gmond to gather metrics. | |Brad | |>>> On 3/31/2009 at 11:32 AM, in message |<9fce3a46fe7c8045a6207ae4b42e9f9a4794075...@gvw1119exc.americas.hpqcorp. |net>, |"Seger, Mark" <mark.se...@hp.com> wrote: |> This uses 3.1 Here's an example of the output I generate when |debugging is |> turned on: |> |> I have a routine I pass 3 parameters to: name of the variable, its |units |> and the value. I generate 2 packets, the first with a header and the |second |> with the data, which looks like this: |> |> 13:41:45.014 Name: ctxint.ctx Units: switches/sec |Val: |> 562 TTL: 60 sent |> 00 00 00 80 00 00 00 0c 63 61 67 2d 64 6c 35 38 35 2d 30 32 00 00 00 |0a 63 |> 74 78 69 6e 74 2e 69 6e 74 00 00 00 00 00 00 00 00 00 06 64 6f 75 62 |6c 65 00 |> 00 00 00 00 0a 63 74 78 69 6e 74 2e 69 6e 74 00 00 00 00 00 0b 69 6e |74 72 70 |> 74 73 2f 73 65 63 00 00 00 00 03 00 00 00 3c 00 00 00 00 00 00 00 00 |> 00 00 00 85 00 00 00 0c 63 61 67 2d 64 6c 35 38 35 2d 30 32 00 00 00 |0a 63 |> 74 78 69 6e 74 2e 69 6e 74 00 00 00 00 00 00 00 00 00 02 25 73 00 00 |00 00 00 |> 04 31 30 37 32 |> |> Not sure if seeing the binary is of much value without the mappings, |but as |> I said a V3.1 gmond is very happy with what I'm sending. We actually |did |> think of xdr, but that would but additional requirements on collectl |and it |> feels like doing things in binary will help minimize overhead. |> |> -mark |> |> |-----Original Message----- |> |From: Brad Nicholes [mailto:bnicho...@novell.com] |> |Sent: Tuesday, March 31, 2009 1:13 PM |> |To: Seger, Mark; ganglia-developers@lists.sourceforge.net |> |Cc: Evan J Felix |> |Subject: Re: [Ganglia-developers] Building a ganglia interface into |> |collectl |> | |> |>>> On 3/31/2009 at 9:56 AM, in message <49d23d46.2090...@hp.com>, |Mark |> |Seger |> |<mark.se...@hp.com> wrote: |> |> |> |> This then leads to my question, which is what is the best way to |send |> |> data to ganglia. I want to keep my messages very dense and so we |> |chose |> |> to simply send out binary data in the same format gmond expects. |In |> |the |> |> case of pnnl, where they have a monitoring hierarchy, we've |completely |> |> replaced all the monitoring gmonds with a dozen that act only as |> |> aggregators. There are about 190 nodes running collectl sending |UPD |> |> messages to each aggregator gmonds and it seems to run just fine. |> |Does |> |> this make sense? Is there anything to watch out for? |> |> |> |> If anyone else is interested in trying this out while we're shaking |> |out |> |> the code, I'd be happy to share some pre-release code with a few |> |people. |> |> |> | |> |Which version of Ganglia are you targeting (3.0.x or 3.1.x). Ganglia |> |uses XDR to pack and unpack the metric packets. However the actual |> |format changed significantly between 3.0.x and 3.1.x. You can see |the |> |XDR packet layout in the file lib/protocol.x for 3.0.x or |> |lib/gm_protocol.x for 3.1.x. The 3.1.x version is a bit more complex |> |than the 3.0.x version. The 3.0.x version is a very simple XDR |packet |> |that basically contains a metric ID and a value. Gmond 3.0.x can get |> |away with just sending an ID in the packet because every 3.0.x gmond |> |hardcodes the metric metadata. Gmond 3.1.x made this more flexible |by |> |splitting the packets in to metadata and value packets. Probably the |> |easiest way to communicate directly with gmond is to use the message |> |creation and sending APIs that are part of the ganglia library. Take |a |> |look at the gmetric utility code for an example of how to use these |> |APIs. Gmetric is basically doing what you want to do but as a |> |standalone utility. For Ganglia 3.0.x you will have to include |> |lib/ganglia.h, for Ganglia 3.1.x you will include include/ganglia.h. |> |The libraries for Ganglia 3.1.x version have been made a little more |> |developer friendly by putting the public headers in the include/ |> |directory and converting the library to be a .so rather than static. |> | |> |Brad |> | | ------------------------------------------------------------------------------ _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
