Re: [Ganglia-general] Adding hosts to cluster in gmetad

2011-12-16 Thread Seth Graham
On Dec 16, 2011, at 10:28 AM, Maciek Lasyk wrote: I've been trying to make a basic ganglia configuration: one gmetad getting data from 2 clusters (11 sources and 1 source) via unicast. Unfortunately with attached configuration I see only the first host from data_source It appears you're

Re: [Ganglia-general] O'Reilly eBook on Ganglia

2011-12-12 Thread Seth Graham
I'd be glad to have a reference for all the settings and variables that are being used in the various .json files. As far as I can tell, the only documentation is the php code itself. Some more verbosity about making custom reports (the stuff that lives in graph.d) would be nice too. Reports

Re: [Ganglia-general] Scaling Ganglia

2011-11-04 Thread Seth Graham
On Nov 3, 2011, at 10:49 PM, Eytan Daniyalzade wrote: I am running a cluster with around 80 nodes, and ganglia-server is running on EC2 with 8G. Loading the main page or a host view on ganglia takes fairly long, ~20sec. It looks like this is taking as long as the view is making sequential

Re: [Ganglia-general] Ganglia Cluster aggregated graphs

2011-10-13 Thread Seth Graham
On Oct 13, 2011, at 11:52 AM, Aidan Wong wrote: This is my first post on this Ganglia list =). I'm using the new Ganglia web 2.1.8 . Has anyone been able to create a graph that aggregates one common metric for several hosts. Try looking at the aggregate graphs tab on the web interface.

[Ganglia-general] Making y axis consistent?

2011-08-12 Thread Seth Graham
Has anyone come up with a clever hack for getting rrd graphs produced by ganglia to use the same axis clamp? At this point, my main issue is with the new Views page, where I've configured a view showing the 15 minute load average for 6 hosts. Every single graph has a different Y axis scale,

Re: [Ganglia-general] Making y axis consistent?

2011-08-12 Thread Seth Graham
, at 2:46 PM, Seth Graham wrote: Has anyone come up with a clever hack for getting rrd graphs produced by ganglia to use the same axis clamp? At this point, my main issue is with the new Views page, where I've configured a view showing the 15 minute load average for 6 hosts. Every

Re: [Ganglia-general] [Ganglia-developers] Announcing Ganglia Web 2.0RC1

2011-06-23 Thread Seth Graham
On Jun 22, 2011, at 4:15 PM, Alex Dean wrote: That requires that the view name match the cluster name, right? Yes, it requires the view and the cluster to match. Could you post your changes somewhere so we could see what you did? I attached a gzipped diff of the gweb-2.0 release against

Re: [Ganglia-general] [Ganglia-developers] Announcing Ganglia Web 2.0RC1

2011-06-22 Thread Seth Graham
On Jun 9, 2011, at 12:10 PM, Alex Dean wrote: I started off intending to allow per-view edit access, just like we allow per-cluster edit access for optional graphs. The complication is that each resource (a view or a cluster) in the ACL is only identified by a simple string. Thus you

Re: [Ganglia-general] [Ganglia-developers] Announcing Ganglia Web 2.0RC1

2011-06-09 Thread Seth Graham
On Jun 8, 2011, at 8:25 PM, Alex Dean wrote: Hi Seth. I'm just back from a week off the grid, and trying to get caught up on a mountain of electronic stuff. Here's my quick response. Please let me know if more explanation is required. Nope, the explanation makes sense. The only thing I

Re: [Ganglia-general] Announcing Ganglia Web 2.0RC1

2011-06-07 Thread Seth Graham
I'm having some issues getting the user roles working as expected. The wiki instructs something like: $acl-addRole( $username, GangliaAcl::GUEST ); $acl-allow( $username, $cluster, GangliaAcl::EDIT ); Which does not result in the little blue + sign to be drawn next to graphs. From line 71 in

Re: [Ganglia-general] default auth settings

2011-04-22 Thread Seth Graham
On Apr 22, 2011, at 9:43 AM, Alex Dean wrote: I'd like to get some feedback on how we should configure gweb's default access permissions. #1. $conf['auth_system']=false; will disable authorization, so no logins are required and the system behaves like the current ganglia web frontend.

Re: [Ganglia-general] Need help configuring clusters to use separate multicast IP

2011-03-23 Thread Seth Graham
That might work, but I don't think anyone sets up their ganglia so that a single gmond is trying aggregate all clusters. That's what the gmetad daemon is for. Also note that even though you have a separate multicast address for each cluster, the port still has to be unique. The port is what

Re: [Ganglia-general] Need help configuring clusters to use separate multicast IP

2011-03-23 Thread Seth Graham
On Mar 23, 2011, at 10:12 AM, Ron Cavallo wrote: I see. So I need a separate IP AND A SEPARATE PORT. Got it. Also, I use a single gmond in each cluster to aggregate the single cluster. I configure the gmetad to talk to only gmond from each cluster. Is that wrong? No, your configuration is

Re: [Ganglia-general] Need help configuring clusters to use separate multicast IP

2011-03-23 Thread Seth Graham
On Mar 23, 2011, at 10:34 AM, Ron Cavallo wrote: Ahhh wait! I do!! On the AGGREGATION Server, I have both a gmetad.conf and a gmond.conf (I also monitor the server itself). I configured RECEIVE channels in the gmond.conf on the aggregation server for every cluster, specifying the IP that

Re: [Ganglia-general] Ganglia: Nodes showing up in wrong clusters in web frontend

2011-03-22 Thread Seth Graham
On Mar 22, 2011, at 10:53 AM, Ron Cavallo wrote: I see other examples where I have to go hunting around for cluster members that aren't reporting into the proper cluster. Any ideas? Double check the ports in use in the gmond.conf on the machines that are misbehaving. Also note that

Re: [Ganglia-general] Ganglia -- modify the source code

2011-03-11 Thread Seth Graham
On Mar 11, 2011, at 1:51 PM, Afef MDHAFFAR wrote: Hi all, I am trying to modify the source code of Ganglia in order to make ganglia able to send monitored data via network connection to another component. I noticed that it sums the metric values of all nodes composing the cluster (eg.

Re: [Ganglia-general] Ganglia -- modify the source code

2011-03-11 Thread Seth Graham
On Mar 11, 2011, at 2:30 PM, Bernard Li wrote: Hi Seth: On Fri, Mar 11, 2011 at 12:26 PM, Seth Graham set...@fnal.gov wrote: You shouldn't need to modify the ganglia source to do this. If you want the per-host value, parse the XML coming from gmond. Every host has an entry in this XML

Re: [Ganglia-general] Noobie questions re: Ganglia

2011-03-02 Thread Seth Graham
On Mar 1, 2011, at 11:26 AM, William Saxton wrote: Hi all (potential) new ganglia user here, with a couple quick questions that I couldn't find the answers to via google. 1) Where can I find how ganglia gathers information from a system? Well, it's an open source project, so you can

Re: [Ganglia-general] Multicast/Unicast Poll

2011-01-13 Thread Seth Graham
On Jan 12, 2011, at 4:22 PM, Bernard Li wrote: Hi Seth: On Wed, Jan 12, 2011 at 1:31 PM, Seth Graham set...@fnal.gov wrote: Migrating to unicast eliminated the firewall issues, means only a select few machines have to keep metrics in memory, and no more cross talk with other groups. I

Re: [Ganglia-general] Issue with gmetad

2011-01-12 Thread Seth Graham
On Jan 12, 2011, at 9:39 AM, John Williams wrote: I have also taken this one step further by installing our server on a brand new Dell R710 with 6x240GB SSD (RAID5). Ganglia is the only thing running on the server. I received the same errors after just a few minutes of running. I have

Re: [Ganglia-general] Multicast/Unicast Poll

2011-01-12 Thread Seth Graham
On Jan 12, 2011, at 3:12 PM, Jesse Becker wrote: In light of the recent discussions over metadata and unicast vs. multicast, we (meaning Bernard) have created a poll on http://ganglia.info/ to try and gauge the use of each. Please let us know if you use multicast, unicast, or both in your

Re: [Ganglia-general] tcp/ip instead of multi-cast ???

2011-01-10 Thread Seth Graham
On Jan 10, 2011, at 2:11 PM, Sayler, Steven (Contractor) wrote: Because of our network, multicast protocol will be a major problem. Is there a way to run ganglia gmond/gmetad via tcp/ip? Yes, look into the udp_send_channel option for gmond.conf. If you specify a host and port gmond will

Re: [Ganglia-general] Ganglia for data collection, not storage/graphing?

2010-12-09 Thread Seth Graham
Yes, because all of the ganglia data is stored in an xml format. You telnet to a gmetad or gmond process and get a dump of everything that daemon knows about. Makes it easy to write additional tools because xml parsers are a dime a dozen. It's fast enough to be used in a web page.. I use it

Re: [Ganglia-general] Archive Ganglia

2010-10-29 Thread Seth Graham
On Oct 29, 2010, at 5:24 AM, nigel.le...@uk.bnpparibas.com wrote: For various convoluted reasons, I would like to copy my rrd files to another server, and view them as a point in time archive. In effect just have the webfrontend running, and no gematd or gmond processes. Any ideas ?

Re: [Ganglia-general] Does Ganglia measure itself?

2010-09-21 Thread Seth Graham
On Sep 21, 2010, at 1:52 PM, Jesse Becker wrote: You can avoid this by using unicast to specifically designated collector gmonds (then having gmetad poll those for overall status). Or by enabling 'deaf' on machines that you don't want collecting data and are stuck on multicast for whatever

Re: [Ganglia-general] Ganglia Web Forum?

2010-08-10 Thread Seth Graham
On Aug 10, 2010, at 2:15 PM, Bernard Li wrote: But I just want to clarify that we are *not* abandoning the mailing-list and IRC. Web forums isn't really my cup of tea either but I just wanted to make sure that forum users have a place to get their questions answered about Ganglia if they

Re: [Ganglia-general] python module strings versus gmetric strings

2010-05-07 Thread Seth Graham
On 5/7/10 9:37 AM, Brad Nicholes wrote: This is the process which packages a metric into a very small packet which can be passed between systems safely. Apologies for barging into this discussion, but I've been working on getting used to the modules features of ganglia this week and this

Re: [Ganglia-general] Extending the format of gmetad.conf

2010-01-07 Thread Seth Graham
Daniel Pocock wrote: - is it important for users to maintain the files manually, or will the focus shift to tools, web interface or config files generated from some other enterprise data source? I've been content with the existing file format for the 7 or so years I've been running

Re: [Ganglia-general] Fw: No graph display on ganglia web page

2009-08-17 Thread Seth Graham
. http://img90.imageshack.us/img90/1203/ganglia1.jpg http://img194.imageshack.us/img194/4039/ganglia2.jpg Thank you very much for your help. Thanach Inactive hide details for Seth Graham set...@fnal.govSeth Graham set...@fnal.gov *Seth Graham set...@fnal.gov

Re: [Ganglia-general] Fw: No graph display on ganglia web page

2009-08-14 Thread Seth Graham
How many machines are you monitoring? Are you getting the page headers at all? php defaults for memory allowance are pretty small. I forget what the default is, but whenever a php script exceeds this limit the script will exit. In the case of ganglia, this usually means either no graphs, or

Re: [Ganglia-general] how to preserve rrd data (as long as I want)

2009-03-04 Thread Seth Graham
jiangyouu wrote: Hi! I want preserve rrd database as long as possible,and review specific day (or hour even minute )'s specific data(like cpu Utilization rate ,network Utilization rate ),but the default value of dear Mr Ganglia is one year. The resolution of stored data is configured when

Re: [Ganglia-general] gmond stops logging data

2008-09-15 Thread Seth Graham
timl wrote: I'm running ganglia version 3.1.0 and periodically gmond seems to stop collecting data. I can see the incoming traffic to the server and the web pages show the hosts as being up, but no data is logged. Originally I started seeing this on 3.0.5 so I upgraded.. but the lastest

Re: [Ganglia-general] ganglia and job monarch

2008-07-23 Thread Seth Graham
Daniel Bourque wrote: Currently, I have this as the revc_channel on the gmond accepting info from all the worker nodes: udp_recv_channel { port = 8666 family = inet4 } I don't see how this channel is associated with a particular cluster . If I add another udp_recv_channel , and

Re: [Ganglia-general] ganglia and job monarch

2008-07-22 Thread Seth Graham
Daniel Bourque wrote: Hi, my setup is as follow, 2 PBS head nodes running torque, moab , ganglia and a group of compute nodes running pbs_mom. Ganglia's gmond is running on the headnodes in mute mode. I'm trying to get rid of the localhost.localdomain node that now shows up in

Re: [Ganglia-general] ganglia and job monarch

2008-07-22 Thread Seth Graham
Daniel Bourque wrote: I don't want ganglia to report on the nodes running pbs_server. I only care about the compute nodes. having non compute nodes in ganglia messes up the usage statistics. The proper way to fix this is have your pbs server submit ganglia information on a different port

Re: [Ganglia-general] gmetad giving high TN values

2008-06-26 Thread Seth Graham
Bernard Li wrote: Hi Kirk: On Wed, Jun 25, 2008 at 1:53 PM, Kirk McDonald [EMAIL PROTECTED] wrote: gmetad runs on a certain host. Also on that host are a number of gmond instances, which are the gmond instances polled by gmetad. Each of these instances is reported to by a separate

Re: [Ganglia-general] Leveraging Ganglia XML output for more then monitoring -- the Thebes Consortium Project

2008-06-10 Thread Seth Graham
Jesse Becker wrote: Bernard Li wrote: While not exactly what you have in mind, but have you taken a look at the JobMonarch project? https://subtrac.sara.nl/oss/jobmonarch/ AFAIK it does also work with SGE. Meh...not really. It's under development, and doesn't work so well with the 6.x

Re: [Ganglia-general] Largest Ganglia installation?

2008-06-06 Thread Seth Graham
Bernard Li wrote: Dear Ganglia community: Was browsing our SourceForge website and found this description of the project: Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at

Re: [Ganglia-general] how to setup ganglia to run in Unicast mode

2008-06-06 Thread Seth Graham
Sai p Seshasayee wrote: Hi Team, I am a new user to ganglia. I have been trying to setup ganglia to run in Unciast mode. Please get back to me regarding the same. Configuration for gmetad is identical for both unicast and multicast. The only difference is the gmond.conf on your machines.

Re: [Ganglia-general] Largest Ganglia installation?

2008-06-06 Thread Seth Graham
Bernard Li wrote: Hi Seth: On Fri, Jun 6, 2008 at 7:45 AM, Seth Graham [EMAIL PROTECTED] wrote: We have ~3000 machines being monitored with ganglia, but the load is split between two separate machines. Probably not the statistic you were after. :) Best machine I had available when

Re: [Ganglia-general] Ganglia 3.0.7-1

2008-05-29 Thread Seth Graham
Owens, David L wrote: In the gmetad.conf file I have data_source called Non_Prod with four servers ie: hostname:8649. These four servers are on the same subnet. I want to add two machines that are on a different subnet. I have tried different ports but will not display under Non_Prod. Any

Re: [Ganglia-general] Setup large clusters

2008-03-12 Thread Seth Graham
Martin Hicks wrote: The configuration of gmetad has been modified to store the rrds in /dev/shm, but this directory gets very large so I'd like to move away from that. Using tmpfs is pretty much your only option. As you discovered, the disk I/O will bring most machines to their knees. Is

Re: [Ganglia-general] XML Parser for Gmetad

2007-07-17 Thread Seth Graham
Buccaneer for Hire. wrote: Hey All, Anyone have an XML parser or pointer for more information for gmetad? I am trying to get a notifier together. THX There's one in ganglia.php in the web frontend. I chopped it up for use in some of my personal scripts and works well.. assuming you're

Re: [Ganglia-general] RRDs in memory

2007-07-13 Thread Seth Graham
Ben Hartshorne wrote: I created a ramdisk when my cluster grew beyond ~50 nodes (I report a lot of extra statistics). I use an actual ramdisk instead of tmpfs (though I chose it out of ignorance when I first set it up, wikipedia[*] says that tmpfs might swap to disk whereas ramfs is just

Re: [Ganglia-general] RRDs in memory

2007-07-11 Thread Seth Graham
Ofer Inbar wrote: gmetad is very write-intensive, because it updates hundreds of RRD files about every minute or two. Has anyone tried running it with the rrd directory on a RAM disk (tmpfs) ? You'd need something to periodically copy the RRDs to a real disk, but that could happen much

Re: [Ganglia-general] A survey of Ganglia users and usage.

2007-04-03 Thread Seth Graham
Buccaneer for Hire. wrote: The simplicity is a major plus as well as the integration w/ Globus. With a little thinking you can extend the reporting easily. The only think I with I had was notification. I have a large cluster and a number a smaller (128 nodes) and it would make it easier

Re: [Ganglia-general] A survey of Ganglia users and usage.

2007-04-02 Thread Seth Graham
[EMAIL PROTECTED] wrote: Perhaps we could create a simple anonymous survey for Ganglia users? Code authors could then be guided quantitively by what the community is really doing - what kind of hosts they monitor - what they use in Ganglia, and what they may need. What do you (all) think?

Re: [Ganglia-general] Gmetad and web frontend on different machines.

2007-03-29 Thread Seth Graham
Martin Knoblauch wrote: Richard, depending on the cluster size, writing the RRDs via NFS might turn out to be a huge bottleneck. Writing them to local disk is sometimes bad enough. Reading them over nfs may be okay though, depends how often users are hitting reload. Cheers Martin ---

Re: [Ganglia-general] ganglia between two networks

2007-03-26 Thread Seth Graham
Jeremy Hansen wrote: I've setup ganglia in the past and typically it's pretty straight forward. Now I have to deal with nodes being in two completely separate networks where it seems udp broadcast are most likely filtered. Is there just a simple config option to have nodes contact a host

Re: [Ganglia-general] Using cluster name to differentiate clusters?

2007-02-26 Thread Seth Graham
Ben Hartshorne wrote: On Mon, Feb 26, 2007 at 01:06:23PM -0600, Seth Graham wrote: Ben Hartshorne wrote: It seems to me that using the name to determine cluster membership would simplify things for the people configuring ganglia. It would, but when you have 3000+ machines all chattering

Re: [Ganglia-general] What are the rrdtool creation parameters for Ganglia Databases?

2007-01-26 Thread Seth Graham
The rrd creation values can be found in gmetad/rrd_helpers.c and gmetad/conf.c Ian Wootten wrote: Hi all, I want to replicate ganglia's storage in Java, using a multicast listener, storing and manipulating using rrd4j. Firstly has anyone done anything similar? I'm struggling knowing

Re: [Ganglia-general] Obtaining Immediate Interval Data From Ganglia

2006-08-10 Thread Seth Graham
Ian Wootten wrote: Hmm, Apologies for the empty reply. Thanks for those suggestions... I'm assuming we're talking kernel modules here, No, we're not. The term 'module' is probably being misapplied here, the stuff being discussed is a module in the sense it extends basic ganglia

Re: [Ganglia-general] Obtaining Immediate Interval Data From Ganglia

2006-08-09 Thread Seth Graham
Ben Hartshorne wrote: On Tue, Aug 08, 2006 at 04:22:41PM +0100, Ian Wootten wrote: I am facing a problem in that I would like short-segment up to date information from ganglia in order to monitor services after invocation. One method I have heard of that achieves something similar; write a

Re: [Ganglia-general] number of source problem in gmetad

2003-09-11 Thread Seth Graham
[EMAIL PROTECTED] init.d]# telnet strauss01 8649 Trying 192.168.1.110... Connected to strauss01. Escape character is '^]'. Connection closed by foreign host. Is anyone got any ideas? When I started getting this, I had to add the server I was connecting from as one of the trusted_hosts