Re: [Ganglia-developers] Ganglia Power Consumption
On 11/21/2016 12:39 AM, Hilmi Egemen Ciritoğlu wrote:
Hi Adrian,
Hi!
Thank you for your answer. I'm new ganglia world. So I have one question
I would be happy if you can answer.
I guess this: $UPS_OUT_LOAD1 is variable. Is it predefined because you
didn't mention any thing about it ?
yes, that is something that i take from the ups and pdu through SNMP (i
have Symmetra PX, but you can find specific OIDs from (any) provider)
see
https://github.com/adriansev/ISSMON/blob/master/get_ups_data
https://github.com/adriansev/ISSMON/blob/master/get_PDUtable
or overall https://github.com/adriansev/ISSMON
Also I didn't figured out why this variable stands for $ups_name_low ?
i just changed the SNMP name of the device to all low letters
in principle the content of the repository is self-explanatory, so if
you have questions first you have to read documentation on SNMP (how to
get information from devices and servers, what is an OID, etc..) and
then how to push metrics to ganglia (gmetric).
On the ganglia subject, it could be that is not very clear but i run
separate gmonds for each type of services which is not easy to do
as i had to rewrite the sysinit script to be more flexible w.r.t
configuration location. (on my monitoring machine i run 5 aggregator
gmonds)
Adrian
Regards,
Egemen
2016-11-20 20:45 GMT+00:00 Adrian Sevcenco <adrian.sevce...@cern.ch
<mailto:adrian.sevce...@cern.ch>>:
On 11/20/2016 08:29 PM, Hilmi Egemen Ciritoğlu wrote:
Hi all,
Hi!
Do you have any idea how can I also collect power consumption ?
Do you
have any plugin for this situation ? Any help would be greatly
appreciated :)
As far i am aware, all power related metrics are available only
through snmp (or other means.). So you would need to use gmetric to
push into ganglia the metrics that you want ... something like :
$SEND -S $SPOOF -g "Output_PH1" -n "ups_ph1_load" -v
$UPS_OUT_LOAD1 -t "float" -u "VA" -s "both" -D "Output Power -
Phase1"-T "Output Power - PH1"$LIFETIME
where:
CONF="/etc/ganglia_ups/gmond.conf"
SEND="/usr/bin/gmetric -c $CONF"
LIFETIME="--tmax=900 --dmax=0"
SPOOF=$IP":"$ups_name_low
HTH,
Adrian
--
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
<mailto:Ganglia-developers@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
<https://lists.sourceforge.net/lists/listinfo/ganglia-developers>
--
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
--
--
Adrian Sevcenco, Ph.D. |
Institute of Space Science - ISS, Romania|
adrian.sevcenco at {cern.ch,spacescience.ro} |
--
smime.p7s
Description: S/MIME Cryptographic Signature
--
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia Power Consumption
On 11/20/2016 08:29 PM, Hilmi Egemen Ciritoğlu wrote: Hi all, Hi! Do you have any idea how can I also collect power consumption ? Do you have any plugin for this situation ? Any help would be greatly appreciated :) As far i am aware, all power related metrics are available only through snmp (or other means.). So you would need to use gmetric to push into ganglia the metrics that you want ... something like : $SEND -S $SPOOF -g "Output_PH1" -n "ups_ph1_load" -v $UPS_OUT_LOAD1 -t "float" -u "VA" -s "both" -D "Output Power - Phase1"-T "Output Power - PH1"$LIFETIME where: CONF="/etc/ganglia_ups/gmond.conf" SEND="/usr/bin/gmetric -c $CONF" LIFETIME="--tmax=900 --dmax=0" SPOOF=$IP":"$ups_name_low HTH, Adrian smime.p7s Description: S/MIME Cryptographic Signature -- ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
[Ganglia-developers] request :: cpu module :: add cpu information
Hi! Is there a possibility to add to cpu reporting module information the info given by : cat /proc/cpuinfo | tail -n +2 | head -n +5 (i am assuming that there is no possibility of having different types of cpus in an node (even in the far future)) namely : vendor_id, cpu family,model, model name and stepping this would allow some grouping and metric analysis based on cpu model.. Thank you! Adrian smime.p7s Description: S/MIME Cryptographic Signature -- Learn Graph Databases - Download FREE O'Reilly Book Graph Databases is the definitive new guide to graph databases and their applications. Written by three acclaimed leaders in the field, this first edition is now available. Download your free book today! http://p.sf.net/sfu/13534_NeoTech___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] GSoC application started, more help needed
Hi! Thanks for answering! On 02/10/2014 08:27 PM, Daniel Pocock wrote: On 07/02/14 21:19, Adrian Sevcenco wrote: On 02/07/2014 09:46 PM, Daniel Pocock wrote: Please feel free to add potential project ideas here: https://github.com/ganglia/monitor-core/wiki/GSoC-2014-project-ideas Hi! There were several discussions on the list regarding what i will mention and i will reiterate the basic points in order to have some kind of definitive closure regarding these and see if there are worthy of doing (either in GSoC or not) I will refer only to gmond framework: 1. Adding a string to globals similar to hostname named something like host_uuid; it can contain either a fixed (overridden) uuid or some automatic approach can be chosen (later) (with sensible defaults like empty); this could pave the way for have uuid--metrics association instead of hostname--metrics I already have something like that on a branch, it is in the wiki somewhere Great! 2. make cluster name be (also) a pool metric of the host; this could pave the way to have gmond aggregators (gmonds that gather data from devices in close network proximity but in different logical partitions (clusters)); something like this i think would be useful in clouds or distributed computing associations like the grid. I imagine/hope that these addons will have no impact on gmetad and are completely backward compatible. So, what the experts think? Thank you for taking this into consideration, Adrian The only danger with (1) is that it involves changing the core agent. Finding students with good C skills and supervising their work on the agent itself is a little more demanding than supervising a student who makes a plugin or some piece of work to complement Ganglia. yeah, this is true, but the example code for (1) is already in ganglia.. one just need to stitch the pieces.. also the code for auto can be added later when the metric is already accepted in the framework. Could you comment more on (2)? I frequently see requests from people who want overlapping cluster aggregation. For example, somebody may want to be able to see aggregate reports for any of the following sets: a) grouped by OS (Linux, Solaris, Windows) b) grouped by vendor (Intel, AMD, ARM, ...) c) grouped by role (production servers, test servers, development servers) Well, the idea of (2) is to shift the task of grouping by a category from gmond to gmetad (or other in house developed monitoring frontend). It has nothing to do with the a), b) and c) like scenarios but with capacity to transmit data of _different_ clusters through a single aggregation gmond (channel). For the sake of simplicity i will imagine an cloud example even if my need and experience is with grid computing: lets say that you make several instances of computing and storage with a provider in one area and some other instances in other area (and maybe other provider). You want to have 2 clusters :computing and storage. so you set up a gmond aggregator on each site that gathers _both_ storage and computing information and send data through ipsec to your gmond aggregator from your home/institutional monitoring node. At that point a gmetad (or another custom build frontend) read the data and write the corresponding data in each corresponding cluster for all machines. (and ignoring the cluster tag of the aggregator gmond that surrounds all data) IMHO the a),b),c) scenarios are in the responsibility presentation framework (ganglia web) and i think this could be done by some json views... For me would be very useful.. is this something to help others as well? Thanks! Adrian smime.p7s Description: S/MIME Cryptographic Signature -- Androi apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151iu=/4140/ostg.clktrk___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] GSoC application started, more help needed
On 02/07/2014 09:46 PM, Daniel Pocock wrote: Please feel free to add potential project ideas here: https://github.com/ganglia/monitor-core/wiki/GSoC-2014-project-ideas Hi! There were several discussions on the list regarding what i will mention and i will reiterate the basic points in order to have some kind of definitive closure regarding these and see if there are worthy of doing (either in GSoC or not) I will refer only to gmond framework: 1. Adding a string to globals similar to hostname named something like host_uuid; it can contain either a fixed (overridden) uuid or some automatic approach can be chosen (later) (with sensible defaults like empty); this could pave the way for have uuid--metrics association instead of hostname--metrics 2. make cluster name be (also) a pool metric of the host; this could pave the way to have gmond aggregators (gmonds that gather data from devices in close network proximity but in different logical partitions (clusters)); something like this i think would be useful in clouds or distributed computing associations like the grid. I imagine/hope that these addons will have no impact on gmetad and are completely backward compatible. So, what the experts think? Thank you for taking this into consideration, Adrian smime.p7s Description: S/MIME Cryptographic Signature -- Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121051231iu=/4140/ostg.clktrk___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Gmetad bottlenecks
On 12/06/2013 10:51 PM, Devon H. O'Dell wrote: 2013/12/6 Vladimir Vuksan vli...@veus.hr: Hello everyone, Hi! For few weeks now we have had performance issues due to growth of our monitoring setup. One of my colleagues Devon O'Dell volunteered to help and below is an e-mail of his findings. Hi! I joined the ML, so I'm around to answer questions. Nice to 'meet' you guys! Thank you for your work! I have also some questions/ideas also but i am still struggling with the internal gmond structures so it may take a while until i can contribute also myself (plus i am not a programmer by profession) So: You said that you are using a gmond to collect data from every machine. The problem with the current implementation of gmond is that: 1. cannot be used for aggregation only (no metrics from localhost) 2. the cluster tagging is done at xml reporting level not at host level. It would be nice to have possibility to have gmond aggregators that just pass along a collection of metrics from multiple machines. Also if the cluster tagging would be made at gmond reporting level it would be possible to aggregate in an gmond metrics from different clusters and gmetad would just write each metrics bundle in the corresponding cluster space. Moreover (it was discussed on the list without a clear conclusion) it would be great if there can be introduced in gmond an UUID ID (without regard of method of generation: from hardware or random generated) that would be the actual key for identifying a machine. It would be enough to have in gmond.conf in host section something like: uuid = some_uuid and move override_hostname from globals to host in a form of an list override_hostname_list=list_of names that would be reported to gmetad as a list of aliases (alongside the reverse DNS result) This will have the effect that the host be be search also by any of former or present hostnames (resolved of not by DNS) Ganglia performance, but most of the low hanging fruit is now gone; at some me point it will require: * writing a version of librrd (this probably also means changing the rrd file format), We (ALICE experiment from CERN) use an tool named MonaLisa (http://monalisa.caltech.edu) written in java that can take in many hundredths of thousands of metrics and written them in postgres database. One obvious advantage would be that there is no need of summarizing at recording stage and also that you have access to the precise metrics without losing information because of averaging. Wouldn't be possible to adapt the gmetad to write the data in a postgres database? One side effect would be that gweb can easily be on other server (for security and load separation purposes) and make reports from the database (also with the averaging mechanism implemented at reporting level) * replacing the hash table in Ganglia with one that performs better, * changing the data serialization format from XML to one that is easier faster to parse, i could just speaking nonsense as i dont understand exactly where is the hash table is used (at the metrics collection step by gmond or gmetad?) but couldn't be used for all communication the same xdr format (and maybe the communication can be improved by using zeromq?) (also with some standalone cli tool that would read and process the output of an gmond). This would remove the need of an xml output, and with the cli tool also would be the possibility of text human inspection of the metrics. (eventually with the conversion to xml done by cli tool) * using a different data structure than a hash table for metrics hierarchies (probably a tree with metrics stored at each level in contiguous memory and an index describing each metric at each level) postgres tables? * refactoring gmetad and gmond into a single process that shares memory i dont think is a good idea as there are processes with different functionality in mind. that would make an process very heavy even if you dont start the gmetad part. (and basically what ganglia is excelling is as a simple, light weight and robust agent based monitoring tool) I would want to help if its possible but i would need also some mentoring. Thank you! Adrian smime.p7s Description: S/MIME Cryptographic Signature -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
[Ganglia-developers] gmond :: CLUSTER tag ( cluster {name}) clarification
Hi! I would need a clarification about the gmond cluster name: it is not clear to me if: 1. the cluster name is a tag for the machine in order to be classified by that tag (so its a per host tag and an aggregator gmond pass that tag (packed with the metrics of that host) to gmetad which order the hosts based on tag) 2. it is a name that is given to group of metrics that are aggregated by that gmond. Thanks! Adrian smime.p7s Description: S/MIME Cryptographic Signature -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
[Ganglia-developers] system UUID (dmidecode)
Hi! Regarding the identification of hosts based on motherboard UUID i
thought to post some info to spare developers of ganglia of some wasted
seconds :)
So : on rhel systems you must do :
yum install python-dmidecode (so gmond should have dependency on this
package that is found in distros repo)
and the small python script (taken from the example of developers of
python-demidecode) would be like this (removed all comments for space
reasons) :
import dmidecode
import sys, os
from pprint import pprint
def print_warnings():
Simple function, dumping out warnings with a prefix if warnings
are found and clearing warning buffer
warn = dmidecode.get_warnings()
if warn:
print ### WARNING: %s % warn
dmidecode.clear_warnings()
root_user = (os.getuid() == 0 and True or False)
if not root_user:
print NOT RUNNING AS ROOT
dmixml = dmidecode.dmidecodeXML()
dmixml.SetResultType(dmidecode.DMIXML_DOC)
xmldoc = dmixml.QuerySection('all')
dmixp = xmldoc.xpathNewContext()
data = dmixp.xpathEval('/dmidecode/SystemInfo/SystemUUID')
for d in data:
print %s % (d.get_content())
del dmixp
del xmldoc
HTH,
Adrian
smime.p7s
Description: S/MIME Cryptographic Signature
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] gmond :: CLUSTER tag ( cluster {name}) clarification
On 08/01/2013 11:37 AM, Adrian Sevcenco wrote:
Hi! I would need a clarification about the gmond cluster name:
it is not clear to me if:
i will answer myself :)
the reported metrics (xml format) by a gmond are wrapped by a CLUSTER
tag ... so there is no way to make a network of gmonds that gathers
data toward one (or more) central gmond which will pass data to a gmetad.
So, my question now is :
Is it possible/useful/value adding to have the cluster name as a
envelope for a metric (or metric collection)?
With the end results that a gmetad take a large flow of reports and
split the metrics by cluster name and writes the metric down in the
coresponding place?
Also : because a gmond can be used to inject metric from snmp devices
(and a datacenter have a lots of those - so it could be something
useful) can be done something like localhost_mute = no (default)/yes ?
Thank you!
Adrian
P.S. sorry for cross-posting but i try to maximize the cross-section for
a larger audience.
1. the cluster name is a tag for the machine in order to be classified
by that tag (so its a per host tag and an aggregator gmond pass that tag
(packed with the metrics of that host) to gmetad which order the hosts
based on tag)
2. it is a name that is given to group of metrics that are aggregated by
that gmond.
Thanks!
Adrian
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
--
--
Adrian Sevcenco, Ph.D. |
Institute of Space Science - ISS, Romania|
adrian.sevcenco at {cern.ch,spacescience.ro} |
--
smime.p7s
Description: S/MIME Cryptographic Signature
--
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
