[Ganglia-general] Is there a way to Display deltas for metrics
Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
I use gmetric from a script scheduled to run every minute via cron. Check 'man gmetric' for syntax, and there are example scripts here: http://www.perzl.org/ganglia/devicespecific.html Hope that helps, Khrist -Original Message- From: Silver, Jonathan [mailto:jonathan.sil...@unify.com] Sent: Monday, August 04, 2014 10:18 AM To: Ganglia Subject: [Ganglia-general] Is there a way to Display deltas for metrics Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
Thanks, but gmetric (I thought) was for the collection of metrics into ganglia. The metrics are already being collected but they are the sum since day 1. I guess that I could create a new metric by running a local script, get the latest 2 values from rrds, compute the difference and set that as the new value for the new metric (added using gmetric), but that is a lot of overhead. Did I miss-understand something? -Original Message- From: Khrist Hansen [mailto:khrist.han...@gmail.com] Sent: Monday, August 04, 2014 11:39 AM To: Silver, Jonathan; 'Ganglia' Subject: RE: [Ganglia-general] Is there a way to Display deltas for metrics I use gmetric from a script scheduled to run every minute via cron. Check 'man gmetric' for syntax, and there are example scripts here: http://www.perzl.org/ganglia/devicespecific.html Hope that helps, Khrist -Original Message- From: Silver, Jonathan [mailto:jonathan.sil...@unify.com] Sent: Monday, August 04, 2014 10:18 AM To: Ganglia Subject: [Ganglia-general] Is there a way to Display deltas for metrics Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
Sounds like you have the RRD files set as a GAUGE when they should be a COUNTER. You could try using rrdtool to fix this on the fly. If you are using gmetric to send metrics, have a look at the --slope option. This is used when creating RRD files for the first time to set the metric (i.e. GAUGE, COUNTER, etc). On Mon, Aug 4, 2014 at 11:17 AM, Silver, Jonathan jonathan.sil...@unify.com wrote: Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Jesse Becker -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
Correct, gmetric will the send your desired metric value and its attributes to gmetad. If the OS or application only provides the metric as a counter but you want them visualized as a gauge, then it is on you to calculate the delta *and* rate of occurrence over time. The example scripts from Dr. Perzl are a good starting point for both, though I had to tweak the algorithm to fit my needs. -Original Message- From: Silver, Jonathan [mailto:jonathan.sil...@unify.com] Sent: Monday, August 04, 2014 10:44 AM To: Khrist Hansen; 'Ganglia' Subject: RE: [Ganglia-general] Is there a way to Display deltas for metrics Thanks, but gmetric (I thought) was for the collection of metrics into ganglia. The metrics are already being collected but they are the sum since day 1. I guess that I could create a new metric by running a local script, get the latest 2 values from rrds, compute the difference and set that as the new value for the new metric (added using gmetric), but that is a lot of overhead. Did I miss-understand something? -Original Message- From: Khrist Hansen [mailto:khrist.han...@gmail.com] Sent: Monday, August 04, 2014 11:39 AM To: Silver, Jonathan; 'Ganglia' Subject: RE: [Ganglia-general] Is there a way to Display deltas for metrics I use gmetric from a script scheduled to run every minute via cron. Check 'man gmetric' for syntax, and there are example scripts here: http://www.perzl.org/ganglia/devicespecific.html Hope that helps, Khrist -Original Message- From: Silver, Jonathan [mailto:jonathan.sil...@unify.com] Sent: Monday, August 04, 2014 10:18 AM To: Ganglia Subject: [Ganglia-general] Is there a way to Display deltas for metrics Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
You are correct - the rrds files do have type GAUGE. Some of these metrics are actually coming from jmxtrans - which has no capability to set the type of value. I cannot figure out the command within rrdtool to change from a gauge to a counter - how do I do this? -Original Message- From: Jesse Becker [mailto:haw...@gmail.com] Sent: Monday, August 04, 2014 11:47 AM To: Silver, Jonathan Cc: Ganglia Subject: Re: [Ganglia-general] Is there a way to Display deltas for metrics Sounds like you have the RRD files set as a GAUGE when they should be a COUNTER. You could try using rrdtool to fix this on the fly. If you are using gmetric to send metrics, have a look at the --slope option. This is used when creating RRD files for the first time to set the metric (i.e. GAUGE, COUNTER, etc). On Mon, Aug 4, 2014 at 11:17 AM, Silver, Jonathan jonathan.sil...@unify.com wrote: Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg. clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Jesse Becker -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
I think something like this should work: rrdtool tune my_metrics.rrd --data-source-type name_of_DS:COUNTER That's 100% untested, and you should make backups of your files, of course. You'll need to figure out the names of the DS entries, which you can get from 'rrdtool info my_metrics.rrd' On Mon, Aug 4, 2014 at 12:06 PM, Silver, Jonathan jonathan.sil...@unify.com wrote: You are correct - the rrds files do have type GAUGE. Some of these metrics are actually coming from jmxtrans - which has no capability to set the type of value. I cannot figure out the command within rrdtool to change from a gauge to a counter - how do I do this? -Original Message- From: Jesse Becker [mailto:haw...@gmail.com] Sent: Monday, August 04, 2014 11:47 AM To: Silver, Jonathan Cc: Ganglia Subject: Re: [Ganglia-general] Is there a way to Display deltas for metrics Sounds like you have the RRD files set as a GAUGE when they should be a COUNTER. You could try using rrdtool to fix this on the fly. If you are using gmetric to send metrics, have a look at the --slope option. This is used when creating RRD files for the first time to set the metric (i.e. GAUGE, COUNTER, etc). On Mon, Aug 4, 2014 at 11:17 AM, Silver, Jonathan jonathan.sil...@unify.com wrote: Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg. clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Jesse Becker -- Jesse Becker -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
Rrdtune worked. - thanks. How does --slope effect the metric type (Gauge ve Counter). What do I need to set slope to to get counter? Thanks again, jon -Original Message- From: Jesse Becker [mailto:haw...@gmail.com] Sent: Monday, August 04, 2014 11:47 AM To: Silver, Jonathan Cc: Ganglia Subject: Re: [Ganglia-general] Is there a way to Display deltas for metrics Sounds like you have the RRD files set as a GAUGE when they should be a COUNTER. You could try using rrdtool to fix this on the fly. If you are using gmetric to send metrics, have a look at the --slope option. This is used when creating RRD files for the first time to set the metric (i.e. GAUGE, COUNTER, etc). On Mon, Aug 4, 2014 at 11:17 AM, Silver, Jonathan jonathan.sil...@unify.com wrote: Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg. clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Jesse Becker -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Is there a way to Display deltas for metrics
Here's a link (with more links) to what's what: http://codeblog.majakorpi.net/post/16281432462/ganglia-xml-slope-attribute (instead of just having me cut-n-paste). On Mon, Aug 4, 2014 at 1:55 PM, Silver, Jonathan jonathan.sil...@unify.com wrote: Rrdtune worked. - thanks. How does --slope effect the metric type (Gauge ve Counter). What do I need to set slope to to get counter? Thanks again, jon -Original Message- From: Jesse Becker [mailto:haw...@gmail.com] Sent: Monday, August 04, 2014 11:47 AM To: Silver, Jonathan Cc: Ganglia Subject: Re: [Ganglia-general] Is there a way to Display deltas for metrics Sounds like you have the RRD files set as a GAUGE when they should be a COUNTER. You could try using rrdtool to fix this on the fly. If you are using gmetric to send metrics, have a look at the --slope option. This is used when creating RRD files for the first time to set the metric (i.e. GAUGE, COUNTER, etc). On Mon, Aug 4, 2014 at 11:17 AM, Silver, Jonathan jonathan.sil...@unify.com wrote: Some of the metrics that we are collecting are totals since the system rebooted. That is not very interesting and it becomes a huge number that shows (percentage-wise) little change over time. Is there some way to change ganglia to display the change of this metric instead of it value without changing the collector itself? Thanks, Jonathan -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg. clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Jesse Becker -- Jesse Becker -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] Several links on ganglia.info do not work
The ones I've seen so far on ganglia.info: On the demo page, the bleeding edge features demo link ( http://fjrkr5ab.joyent.us/ganglia-2.0/) gives an error. On the support page, both the link to the ganglia wiki page ( http://wiki.ganglia.info/) and the link to the ganglia forums ( http://forums.ganglia.info/) are simply rerouted to the ganglia sourceforge page. -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] Ganglia install cannot find libconfuse cfg_parse
libconfuse was installed via rpm and is present on my system. rpm -qa | grep libconfuse libconfuse-2.7-4.el6.x86_64 rpm -ql libconfuse gives me /usr/lib64/libconfuse.so.0 /usr/lib64/libconfuse.so.0.0.0 /usr/share/doc/libconfuse-2.7 /usr/share/doc/libconfuse-2.7/AUTHORS /usr/share/doc/libconfuse-2.7/NEWS /usr/share/doc/libconfuse-2.7/README /usr/share/doc/libconfuse-2.7/html /usr/share/doc/libconfuse-2.7/html/annotated.html /usr/share/doc/libconfuse-2.7/html/classes.html /usr/share/doc/libconfuse-2.7/html/confuse_8h-source.html /usr/share/doc/libconfuse-2.7/html/confuse_8h.html /usr/share/doc/libconfuse-2.7/html/doxygen.png /usr/share/doc/libconfuse-2.7/html/examples.html /usr/share/doc/libconfuse-2.7/html/files.html /usr/share/doc/libconfuse-2.7/html/ftpconf_8c-example.html /usr/share/doc/libconfuse-2.7/html/functions.html /usr/share/doc/libconfuse-2.7/html/functions_vars.html /usr/share/doc/libconfuse-2.7/html/globals.html /usr/share/doc/libconfuse-2.7/html/globals_defs.html /usr/share/doc/libconfuse-2.7/html/globals_enum.html /usr/share/doc/libconfuse-2.7/html/globals_eval.html /usr/share/doc/libconfuse-2.7/html/globals_func.html /usr/share/doc/libconfuse-2.7/html/globals_type.html /usr/share/doc/libconfuse-2.7/html/index.html /usr/share/doc/libconfuse-2.7/html/reread_8c-example.html /usr/share/doc/libconfuse-2.7/html/simple_8c-example.html /usr/share/doc/libconfuse-2.7/html/structcfg__defvalue__t.html /usr/share/doc/libconfuse-2.7/html/structcfg__opt__t.html /usr/share/doc/libconfuse-2.7/html/structcfg__t.html /usr/share/doc/libconfuse-2.7/html/tab_b.gif /usr/share/doc/libconfuse-2.7/html/tab_l.gif /usr/share/doc/libconfuse-2.7/html/tab_r.gif /usr/share/doc/libconfuse-2.7/html/tabs.css /usr/share/doc/libconfuse-2.7/html/tutorial.css /usr/share/doc/libconfuse-2.7/html/unioncfg__value__t.html /usr/share/locale/fr/LC_MESSAGES/confuse.mo /usr/share/locale/sv/LC_MESSAGES/confuse.mo /usr/share/man/man3/cfg_defvalue_t.3.gz /usr/share/man/man3/cfg_opt_t.3.gz /usr/share/man/man3/cfg_t.3.gz /usr/share/man/man3/cfg_value_t.3.gz /usr/share/man/man3/confuse.h.3.gz I've tried setting the --with-libconfuse flag as indicated in the documentation to both /usr/share/man/man3 and /usr/share/doc/libconfuse-2.7, with no luck . It still can't find what it is looking for : cfg_parse? I don't cfg_parse does not appear in my list above. Wondering if I got a malformed libconfuse install or if I just need to point ganglia configure somewhere else. The output of ./configure --with-libconfuse=/usr/share/doc/libconfuse-2.7 is Checking for confuse Added -I/usr/share/doc/libconfuse-2.7/include to CFLAGS Added -L/usr/share/doc/libconfuse-2.7/lib64 to LDFLAGS checking for cfg_parse in -lconfuse... no Trying harder including gettext checking for cfg_parse in -lconfuse... no Trying harder including iconv checking for cfg_parse in -lconfuse... no libconfuse not found -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia install cannot find libconfuse cfg_parse
Hi Jonathan, I've seen this on Fedora and I think Red Hat. You need to create the link manually: # ln -s libconfuse.so.0 /usr/lib64/libconfuse.so I'd expect the RPM install script to do this. Filipe -- Filipe Bonjour In theory there is no difference between theory and practice. In practice there is. -- Yogi Berra -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Extract Ganglia data for processing in R and python
Hi Sergio (and Ganglia community), Thanks so much for the prompt and helpful response. This is just what I was looking for. One unexpected surprise was that my simple four-data-node AWS/EC2 MapReduce cluster has almost 1,400 rrd files in /var/lib/ganglia/rrds. From the path names, I'm guessing there is a rrd file per node and metric. I'm not sure what the __SummaryInfo__ dirs contain. Is there a wiki/FAQ page somewhere that describes the __SummaryInfo__ contents? Since I'm running benchmarks, I need to verify that all the nodes are doing roughly the same amount of work (CPU, disk, memory, network, etc). so need to analyze the per-node rrd files but maybe the Summary files could be useful??? Any pointers or tips would be appreciated. Doug Subject: Re: [Ganglia-general] Extract Ganglia data for processing in R and python From: sergio.ballestr...@gmail.com Date: Sat, 12 Jul 2014 09:28:55 +0200 CC: ganglia-general@lists.sourceforge.net To: dvjbi...@hotmail.com Hello Doug,I'm not sure if it's the kind of extract stats that you have in mind, but you can simply use rrdtool to dump the rrd files created by Ganglia to an XML format.Then you can delete the rrd and start anew. Cheers, Sergio On 12 Jul 2014, at 00:39, Doug Johnson dvjbi...@hotmail.com wrote:I'm sure this question has been asked answered before but I've been unable to find anything useful after a four-hour search of wikis, mail archives and the web. I'm running lots of benchmarks on various AWS/EC2 MapReduce instances and need to extract and process Ganglia data individually for each benchmark. So basically: clear stats, run benchmark, extract stats and save to results directory. I certainly can't be the first person to do this ;-) Any pointers or references would be most appreciated. Thanks, Doug -- ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Extract Ganglia data for processing in R and python
Hello Doug, On 5 Aug 2014, at 02:52, Doug Johnson dvjbi...@hotmail.com wrote: Hi Sergio (and Ganglia community), Thanks so much for the prompt and helpful response. This is just what I was looking for. One unexpected surprise was that my simple four-data-node AWS/EC2 MapReduce cluster has almost 1,400 rrd files in /var/lib/ganglia/rrds. From the path names, I'm guessing there is a rrd file per node and metric. correct: it's /var/lib/ganglia/rrds/cluster/hostname/metric.rrd The number of metrics per host depends on the configuration on the client. The basic metrics are those collected by gmond, plus the plugins you have enabled (see gmond -m), plus anything that your application(s) may be sending directly to the server e.g. using gmetrics. Sure that ~300 metrics per host sounds like a lot... I usually have between 30 and ~100. I'm not sure what the __SummaryInfo__ dirs contain. Is there a wiki/FAQ page somewhere that describes the __SummaryInfo__ contents? I don't know if it's documented somewhere, but I had figured out that since the web interfaces shows summaries/totals per-cluster and overall, well, that's the rrds that contain those total sums over hosts, for each metric. Since I'm running benchmarks, I need to verify that all the nodes are doing roughly the same amount of work (CPU, disk, memory, network, etc). so need to analyze the per-node rrd files but maybe the Summary files could be useful??? Any pointers or tips would be appreciated. if you have the ganglia-web 3.5 or later, look at the Compare Hosts tab, or use the Aggregate Graphs to compose a View with the metrics you are interested in. You may not need to do the manual work of extracting data from RRDs after all, unless you need to do a more complex data analysis? Cheers, Sergio Doug Subject: Re: [Ganglia-general] Extract Ganglia data for processing in R and python From: sergio.ballestr...@gmail.com Date: Sat, 12 Jul 2014 09:28:55 +0200 CC: ganglia-general@lists.sourceforge.net To: dvjbi...@hotmail.com Hello Doug, I'm not sure if it's the kind of extract stats that you have in mind, but you can simply use rrdtool to dump the rrd files created by Ganglia to an XML format. Then you can delete the rrd and start anew. Cheers, Sergio On 12 Jul 2014, at 00:39, Doug Johnson dvjbi...@hotmail.com wrote: I'm sure this question has been asked answered before but I've been unable to find anything useful after a four-hour search of wikis, mail archives and the web. I'm running lots of benchmarks on various AWS/EC2 MapReduce instances and need to extract and process Ganglia data individually for each benchmark. So basically: clear stats, run benchmark, extract stats and save to results directory. I certainly can't be the first person to do this ;-) Any pointers or references would be most appreciated. Thanks, Doug -- ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia install cannot find libconfuse cfg_parse
Hi Jonathan, Thank you, Filipe. That allowed the configure script to complete without needing to specify any special parameters. I am not sure why the RPM script didn't handle it correctly. Neither am I, but I've seen this once or twice from Red Hat packages. I now run into an issue when running 'make' also with libconfuse: libgmond.c:15:21: error: confuse.h: No such file or directory The header file is in the devel package, so you need to yum install libconfuse-devel. I would expect the configure script to pick that up, actually... Good luck, Filipe -- Filipe Bonjour In theory there is no difference between theory and practice. In practice there is. -- Yogi Berra -- Infragistics Professional Build stunning WinForms apps today! Reboot your WinForms applications with our WinForms controls. Build a bridge from your legacy apps to the future. http://pubads.g.doubleclick.net/gampad/clk?id=153845071iu=/4140/ostg.clktrk ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
