Re: [Ganglia-developers] How to submit bug fixes/ new features for ganglia-web?
Hi Michael: Thanks for the email. Your best bet is to fork the code in GitHub, commit your changes and submit a pull request against master -- someone will have a look and potentially integrate it upstream. Cheers, Bernard On Wed, Jun 20, 2012 at 10:58 PM, Michael Shearer zonked.mon...@gmail.com wrote: Hi, I have been playing around with the ganglia - nagios integration scripts, and uncovered a small issue in some of the php files, where the wrong variable is used for the output where a host is not found ($metric_name instead of $host). I have corrected this locally, but how do I best get this fixed upstream (ie, in github)? Also, I have used check_multiple_metrics.php as the basis to write a custom version, as we needed to add the warning functionality, and desired not to alert about metrics that returned no data. If this is something others would find useful, how do I submit it? Side note: A big thanks to everyone who has worked on this project, it is very valuable and I hope to be able to give something back. Cheers, Michael. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia libvirt and other virtualization technologies
Hi Semy: The more elegant solution is to use gmetad-python and write plugins to manipulate the metric data collected however way you want. For more information, please see the README: https://github.com/ganglia/monitor-core/blob/master/gmetad-python/README And Peter is right, ganglia-general is the more appropriate list for your questions. This list should be reserved for code development discussions, not so much user questions. Thanks, Bernard On Wed, Jun 20, 2012 at 12:40 PM, Simon G. semy...@gmail.com wrote: I think I've found answer for my question. I've just read few files from source code of ganglia-web and as i can see they are reading RRD to draw everything so i think I have to write my own wrapper around RRD to do what i need. Anyway thx for informations. Cheers, On Wed, Jun 20, 2012 at 8:57 PM, Peter Phaal peter.ph...@inmon.com wrote: There are people on this list who should be able to answer questions relating to gmetad (although you might have better luck on the ganglia-general list). My expertise is limited to sFlow. On Jun 20, 2012, at 11:39 AM, Simon G. wrote: In next 2-5 days I'll try ganglia + sFlow again, so I'll definitely let you know about my problems. If you can't answer my previous question to whom should I send it? Cheers, On Wed, Jun 20, 2012 at 8:24 PM, Peter Phaal peter.ph...@inmon.com wrote: I can't help you with your questions about gmetad, but I would be very interested in hearing where you ran into trouble during the sFlow/Ganglia configuration. I am currently working on documentation and would like to make sure that I fill any gaps. On Jun 20, 2012, at 11:09 AM, Simon G. wrote: I've already seen sFlow and ganglia about month ago, and I had some problems during configuration, but maybe I should try it again. So maybe I have different question now. Is there any way to query gmetad about historical data? I'd like to retrieve some data in my external piece of software (python module) to analyze those data and do some actions if proper conditions are fulfilled. I don't want store data twice (in my module for example in my Mysql database and in RRD files created by gmetad), so my true question is: can i query gmetad to get those data, or should i just read gmetad's RRD file and do not connect with gmetad itself or maybe should i get data through tcp from gmetad and store it in my own way and own database? I don't know much about RRD and I'd like to avoid reading RRD files myself, but also I don't want to store them twice. It would be perfect if I could get them through ganglia (from gmetad after proper request), because mostly, I'm interested in historical data. Btw. what differences are between gmetad and gmetad-python? Beside language of course. Both implementations have the same features? On Tue, Jun 19, 2012 at 12:04 AM, Peter Phaal peter.ph...@inmon.com wrote: Simon, Ganglia 3.2+ can collect and display sFlow metrics: http://blog.sflow.com/2012/01/using-ganglia-to-monitor-virtual.html You can monitor the virtual machines by installing Host sFlow agents (http://host-sflow.sourceforge.net/ ) on the hypervisors. The sFlow agents export libvirt metrics using the sFlow protocol. Currently, Host sFlow supports KVM, Xen and Hyper-V. Cheers, Peter On Jun 18, 2012, at 2:50 PM, Simon G. wrote: Hello, Is there any way to use Ganglia to monitor virtual machines? Right now I'd like to get data from libvirt's api (CPU usage, IO, Network etc), but I don't want to reinvent the wheel and I'd like to use already mature tools to do that, for example Ganglia. I know collectd has plugins which support libvirt, but from what I've read, ganglia could be better solution. 1st question is: how can I monitor virtual machines in Ganglia. Especially using libvirt, but later also Xen, VMware, Hyper-V. Is it possible? Can ganglia monitor virtual machines? What can I monitor without additional agent on guest and what can I monitor with such agent? 2nd question: I've just found out gmetad-python, but before I'll try to use it I'd like to ask about possibilities to use python scripts to collect data from gmond. What is the easiest way to do that? I'm interested in python, because I'd like to try create simple Openstack and Ganglia proof of concept. I was going to use existing simple monitoring around libvirt from openstack and implement storing data in RRD files, but as I've mentioned above, I don't want to reinvent the wheel and I think a better way is to integrate already existing solutions. The question is how to integrate those two projects. My first step is to run gmond on each host, collect data from it via python script, save those data into RRD file and use this data in Openstack's environment. If gmetad-python can do that, it's great. Then second step will be to get that data from gmetad-python (via python script) and use it in Openstack's environment. Cheers, -- Simon
Re: [Ganglia-developers] gmond steadily memory usage increase
Well at least since version 3.3.1. Before that, we have been changing setup's so I don't know. Cheers, - Ramon. On 19-6-2012 18:05, Bernard Li wrote: Hi Ramon: Which version of Ganglia do you start seeing the issue? Thanks, Bernard On Tuesday, June 19, 2012, Ramon Bastiaans wrote: Hi Bernard, This collector daemon gathers metrics from 512 machines. Every node has ~100 metrics, this results in ~50.000 metrics. I do realize this is a somewhat large amount of metrics for 1 gmond and that this can require a lot of memory. Nevertheless I would expect the memory usage to level out at a certain point. We only use modpython on this machine. When I disable the modpython module and move all *pyconf*, the memory usage still climbs over time. I made a little script to print the amount of HOST/METRICs in memory every minute and the memory consumption of gmond. It seems whever there is a (temporary, small) decrease in METRICs in the gmond's XML, the next cycle the memory usage increases. This sounds like something is not free'd / cleaned. See the attached log. Cheers, - Ramon. On 18-6-2012 17:22, Bernard Li wrote: Hi Ramon: That's not good... are you by chance running any metric modules? How many nodes report to this collector? What if you turn off all except for one gmond reporting to it, does it still leak memory? Thanks, Bernard On Mon, Jun 18, 2012 at 7:43 AM, Ramon Bastiaans ramon.bastia...@sara.nl wrote: This also happens with version 3.4.0. - Ramon. On 18-6-2012 16:40, Ramon Bastiaans wrote: Hi, Is anyone still experiencing memory increase over time in gmond? At one of our systems the collector gmond reaches up to 2GB memory if we keep it running long enough, see attached graph. I'm no expert but when I run valgrind on gmond for a few minutes, amongst other things, it says: ==17668== LEAK SUMMARY: ==17668==definitely lost: 22,339 bytes in 50 blocks ==17668==indirectly lost: 2,206,722 bytes in 659 blocks ==17668== possibly lost: 0 bytes in 0 blocks ==17668==still reachable: 154,605 bytes in 2,745 blocks ==17668== suppressed: 0 bytes in 0 blocks ==17668== Reachable blocks (those to which a pointer was found) are not shown. ==17668== To see them, rerun with: --leak-check=full --show-reachable=yes I'm not sure how accurate or correct this is, but is anyone aware of memory usage issues? Kind regards, - Ramon. -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 smime.p7s Description: S/MIME Cryptographic Signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats.
Re: [Ganglia-developers] gmond steadily memory usage increase
I think this may be this issue: * http://bugzilla.ganglia.info/cgi-bin/bugzilla/show_bug.cgi?id=329 On 21-6-2012 11:38, Ramon Bastiaans wrote: Well at least since version 3.3.1. Before that, we have been changing setup's so I don't know. Cheers, - Ramon. On 19-6-2012 18:05, Bernard Li wrote: Hi Ramon: Which version of Ganglia do you start seeing the issue? Thanks, Bernard On Tuesday, June 19, 2012, Ramon Bastiaans wrote: Hi Bernard, This collector daemon gathers metrics from 512 machines. Every node has ~100 metrics, this results in ~50.000 metrics. I do realize this is a somewhat large amount of metrics for 1 gmond and that this can require a lot of memory. Nevertheless I would expect the memory usage to level out at a certain point. We only use modpython on this machine. When I disable the modpython module and move all *pyconf*, the memory usage still climbs over time. I made a little script to print the amount of HOST/METRICs in memory every minute and the memory consumption of gmond. It seems whever there is a (temporary, small) decrease in METRICs in the gmond's XML, the next cycle the memory usage increases. This sounds like something is not free'd / cleaned. See the attached log. Cheers, - Ramon. On 18-6-2012 17:22, Bernard Li wrote: Hi Ramon: That's not good... are you by chance running any metric modules? How many nodes report to this collector? What if you turn off all except for one gmond reporting to it, does it still leak memory? Thanks, Bernard On Mon, Jun 18, 2012 at 7:43 AM, Ramon Bastiaans ramon.bastia...@sara.nl wrote: This also happens with version 3.4.0. - Ramon. On 18-6-2012 16:40, Ramon Bastiaans wrote: Hi, Is anyone still experiencing memory increase over time in gmond? At one of our systems the collector gmond reaches up to 2GB memory if we keep it running long enough, see attached graph. I'm no expert but when I run valgrind on gmond for a few minutes, amongst other things, it says: ==17668== LEAK SUMMARY: ==17668==definitely lost: 22,339 bytes in 50 blocks ==17668==indirectly lost: 2,206,722 bytes in 659 blocks ==17668== possibly lost: 0 bytes in 0 blocks ==17668==still reachable: 154,605 bytes in 2,745 blocks ==17668== suppressed: 0 bytes in 0 blocks ==17668== Reachable blocks (those to which a pointer was found) are not shown. ==17668== To see them, rerun with: --leak-check=full --show-reachable=yes I'm not sure how accurate or correct this is, but is anyone aware of memory usage issues? Kind regards, - Ramon. -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 -- ing. R. Bastiaans, B.ICT * Senior Systems Programmer * Operations, Support and Development SARA Science Park 140 PO Box 94613 1098 XG Amsterdam NL 1090 GP Amsterdam NL P.+31 (0)20 592 3000 F.+31 (0)20 668 3167 -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint
