I have a patch file from Yemi as of yesterday. Attached. I'm still working on why even though a host shows up after running gmetric, the corresponding metric isn't.
-----Original Message----- From: Bernard Li [mailto:[EMAIL PROTECTED] Sent: Thursday, August 24, 2006 2:38 PM To: [EMAIL PROTECTED]; Dr. Dave Blunt; ganglia-general@lists.sourceforge.net; [EMAIL PROTECTED] Subject: RE: [Ganglia-general] Ganglia scaling testing? Guys: I haven't checked the archives, but is Yemi's spoofing code available somewhere? Perhaps in bugzilla? I have a friend who wants to try it out... Thanks, Bernard > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > [EMAIL PROTECTED] > Sent: Thursday, August 24, 2006 9:27 > To: [EMAIL PROTECTED]; > ganglia-general@lists.sourceforge.net; [EMAIL PROTECTED] > Subject: Re: [Ganglia-general] Ganglia scaling testing? > > Ahh yes, > > I forgot about Yemi's spoofing code. Hacking that sounds the easiest > way. > > regards, > Richard > > -----Original Message----- > From: Dr. Dave Blunt [mailto:[EMAIL PROTECTED] > Sent: 24 August 2006 16:58 > To: ganglia-general; harper.mann; Grevis, Richard: IT (LDN) > Subject: RE: [Ganglia-general] Ganglia scaling testing? > > > Hey Richard, > > We were wanting to populate a gmond on a separate box that we would > run some alarm code against - no gmetad on the box. I'm trying Yemi's > spoof code right now to get a large number of 'hosts' set up with lots > of metrics. I did look at the packet format in the source but my C is > a bit rusty. I got bogged down in tcpdump/tcpreplay without much > progress. > > Thanks, > > > Dave. > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] > Sent: Thursday, August 24, 2006 4:09 AM > To: [EMAIL PROTECTED]; ganglia-general@lists.sourceforge.net; Dr. > Dave Blunt > Subject: RE: [Ganglia-general] Ganglia scaling testing? > > Harper, > > I think that the RRD disk I/O from gmetad will be the first limit you > reach. > > If you want to load up the gmond process, you could write a program to > send properly formatted gmond packets but with a spoofed and always > changing source address. the headnode gmond only determines the host > from the source address of the packet. I am not sure the of the best > way to do this - either a standalone program that sends packets flat > chat where the contents were snarfed from real gmond packets, or maybe > a hack to gmond itself. > > Simulating on the server/gmetric side is easier - just write a script > that contructs the right XML with thousands of hosts and presents it > on a port, and point gmetad to that port. > > regards, > richard grevis > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Harper Mann > Sent: 23 August 2006 18:08 > To: ganglia-general@lists.sourceforge.net; Dr. Dave Blunt > Subject: [Ganglia-general] Ganglia scaling testing? > > > Hi, > We're supporting a couple of sites with several hundred servers > monitored with Ganglia. So far, ganglia is working well and easily > keeping up with the load. We want to run some scaling tests and > thought it would be good to simulate a large gmond with a couple > thousand servers reporting before we have to support that many. Is > there a way to simulate multiple servers reporting to a gmond? If not, > what do you think might be the best approach for this? We could take > a crack at creating it. Thanks for any help. Regards, > - Harper > Harper Mann > Groundwork Open Source > 510-599-2075 (cell) > > > -------------------------------------------------------------- > ---------- > - > Using Tomcat but need to do more? Need to support web services, > security? Get stuff done quickly with pre-integrated technology to > make your job easier Download IBM WebSphere Application Server > v.1.0.1 based > on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > dat=121642 > _______________________________________________ > Ganglia-general mailing list Ganglia-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-general > -------------------------------------------------------------- > ---------- > For more information about Barclays Capital, please visit our web site > at http://www.barcap.com. > > Internet communications are not secure and therefore the Barclays > Group does not accept legal responsibility for the contents of this > message. > Although the Barclays Group operates anti-virus programmes, it does > not accept responsibility for any damage whatsoever that is caused by > viruses being passed. Any views or opinions presented are solely > those of the author and do not necessarily represent those of the > Barclays Group. Replies to this email may be monitored by the > Barclays Group for operational or business reasons. > -------------------------------------------------------------- > ---------- > > > -------------------------------------------------------------- > ----------- > Using Tomcat but need to do more? Need to support web > services, security? > Get stuff done quickly with pre-integrated technology to make > your job easier > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& > dat=121642 > _______________________________________________ > Ganglia-general mailing list > Ganglia-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-general >
diff -udpr ganglia-3.0.3/gmetric/cmdline.sh ganglia-new/gmetric/cmdline.sh --- ganglia-3.0.3/gmetric/cmdline.sh 2006-05-17 16:33:42.000000000 -0700 +++ ganglia-new/gmetric/cmdline.sh 2006-05-17 10:38:02.000000000 -0700 @@ -6,14 +6,15 @@ purpose "The Ganglia Metric Client (gmet on the list of defined send channels defined in a configuration file" option "conf" c "The configuration file to use for finding send channels" string default="/etc/gmond.conf" no -option "name" n "Name of the metric" string yes -option "value" v "Value of the metric" string yes -option "type" t "Either string|int8|uint8|int16|uint16|int32|uint32|float|double" string yes +option "name" n "Name of the metric" string no +option "value" v "Value of the metric" string no +option "type" t "Either string|int8|uint8|int16|uint16|int32|uint32|float|double" string no option "units" u "Unit of measure for the value e.g. Kilobytes, Celcius" string default="" no option "slope" s "Either zero|positive|negative|both" string default="both" no option "tmax" x "The maximum time in seconds between gmetric calls" int default="60" no option "dmax" d "The lifetime in seconds of this metric" int default="0" no - +option "spoof" S "IP address and name of host/device (colon separated) we are spoofing" string default="" no +option "heartbeat" H "spoof a heartbeat message (use with spoof option)" #option "mcast_channel" c "Multicast channel to send/receive on" string default="239.2.11.71" no #option "mcast_port" p "Multicast port to send/receive on" int default="8649" no diff -udpr ganglia-3.0.3/gmetric/gmetric.c ganglia-new/gmetric/gmetric.c --- ganglia-3.0.3/gmetric/gmetric.c 2006-04-17 07:42:55.000000000 -0700 +++ ganglia-new/gmetric/gmetric.c 2006-05-17 10:21:54.000000000 -0700 @@ -50,10 +50,19 @@ main( int argc, char *argv[] ) fprintf(stderr,"Unable to allocate gmetric structure. Exiting.\n"); exit(1); } - - rval = Ganglia_gmetric_set( gmetric, args_info.name_arg, args_info.value_arg, + // Yemi + if(args_info.spoof_given && args_info.heartbeat_given){ + rval = 0; + }else{ + if( ! (args_info.name_given && args_info.value_given && + args_info.type_given)){ + fprintf(stderr,"Incorrect options supplied, exiting.\n"); + exit(1); + } + rval = Ganglia_gmetric_set( gmetric, args_info.name_arg, args_info.value_arg, args_info.type_arg, args_info.units_arg, !strcmp(args_info.slope_arg,"zero")? 0: 3, args_info.tmax_arg, args_info.dmax_arg); + } /* TODO: make this less ugly later */ switch(rval) { @@ -72,7 +81,13 @@ main( int argc, char *argv[] ) } /* send the message */ - rval = Ganglia_gmetric_send(gmetric, send_channels); + //Yemi + if(!strlen(args_info.spoof_arg)) + { + rval = Ganglia_gmetric_send(gmetric, send_channels); + }else{ + rval = Ganglia_gmetric_send_spoof(gmetric, send_channels,args_info.spoof_arg,args_info.heartbeat_given); + } if(rval) { fprintf(stderr,"There was an error sending to %d of the send channels.\n", rval); diff -udpr ganglia-3.0.3/gmond/gmond.c ganglia-new/gmond/gmond.c --- ganglia-3.0.3/gmond/gmond.c 2006-04-17 07:42:55.000000000 -0700 +++ ganglia-new/gmond/gmond.c 2006-05-16 12:19:10.000000000 -0700 @@ -578,19 +578,29 @@ setup_listen_channels_pollset( void ) } } } - +// Yemi static Ganglia_host * -Ganglia_host_get( char *remoteip, apr_sockaddr_t *sa, Ganglia_message *fullmsg) +Ganglia_host_get( char *remIP, apr_sockaddr_t *sa, Ganglia_message *fullmsg) { apr_status_t status; apr_pool_t *pool; Ganglia_host *hostdata; char *hostname = NULL; - + char *remoteip = remIP; + if(!remoteip || !sa || !fullmsg) { return NULL; } + + if(fullmsg->id == spoof_metric){ + hostname = fullmsg->Ganglia_message_u.spmetric.spheader.spoofName; + remoteip = fullmsg->Ganglia_message_u.spmetric.spheader.spoofIP; + } + if(fullmsg->id == spoof_heartbeat){ + hostname = fullmsg->Ganglia_message_u.spheader.spoofName; + remoteip = fullmsg->Ganglia_message_u.spheader.spoofIP; + } hostdata = (Ganglia_host *)apr_hash_get( hosts, remoteip, APR_HASH_KEY_STRING ); if(!hostdata) @@ -689,6 +699,13 @@ Ganglia_host_get( char *remoteip, apr_so /* Processing is finished */ return NULL; } + if(fullmsg->id == spoof_heartbeat) + { + /* nothing more needs to be done. we handled the timestamps above. */ + debug_msg("Got a spoof heartbeat message %d\n", hostdata->gmond_started); + /* Processing is finished */ + return NULL; + } return hostdata; } @@ -739,6 +756,15 @@ Ganglia_message_find_gmetric( Ganglia_ho message->Ganglia_message_u.gmetric.name, APR_HASH_KEY_STRING); } +// Yemi +static Ganglia_metric * +Ganglia_message_find_spmetric( Ganglia_host *host, Ganglia_message *message) +{ + /* Keyed on the name element of the gmetric sent */ + return (Ganglia_metric *)apr_hash_get( host->gmetrics, + message->Ganglia_message_u.spmetric.gmetric.name, + APR_HASH_KEY_STRING); +} static Ganglia_metric * Ganglia_message_find_metric( Ganglia_host *host, Ganglia_message *message) @@ -753,7 +779,7 @@ static void Ganglia_message_save( Ganglia_host *host, Ganglia_message *message ) { Ganglia_metric *metric = NULL; - + if(!host || !message) return; @@ -762,6 +788,11 @@ Ganglia_message_save( Ganglia_host *host { metric = Ganglia_message_find_gmetric( host, message); } + // Yemi + else if(message->id == spoof_metric) + { + metric = Ganglia_message_find_spmetric( host, message); + } else { metric = Ganglia_message_find_metric( host, message); @@ -785,6 +816,11 @@ Ganglia_message_save( Ganglia_host *host { metric->name = apr_pstrdup( metric->pool, message->Ganglia_message_u.gmetric.name ); } + // Yemi + if(message->id == spoof_metric) + { + metric->name = apr_pstrdup( metric->pool, message->Ganglia_message_u.spmetric.gmetric.name ); + } } else { @@ -793,9 +829,19 @@ Ganglia_message_save( Ganglia_host *host } /* Copy in the data */ - memcpy(&(metric->message), message, sizeof(Ganglia_message)); + // Yemi + if(message->id == spoof_metric){ + // Store data as regular gmetric in hash table!! - if(message->id == metric_user_defined) + metric->message.id = metric_user_defined; + metric->message.Ganglia_message_u.gmetric = message->Ganglia_message_u.spmetric.gmetric; + + + }else{ + memcpy(&(metric->message), message, sizeof(Ganglia_message)); + } + + if(message->id == metric_user_defined || message->id == spoof_metric) { /* Save the gmetric */ apr_hash_set( host->gmetrics, metric->name, APR_HASH_KEY_STRING, metric); diff -udpr ganglia-3.0.3/lib/ganglia.h ganglia-new/lib/ganglia.h --- ganglia-3.0.3/lib/ganglia.h 2006-04-17 07:42:55.000000000 -0700 +++ ganglia-new/lib/ganglia.h 2006-05-16 15:01:27.000000000 -0700 @@ -58,6 +58,8 @@ int Ganglia_udp_send_message(Ganglia_udp Ganglia_gmetric Ganglia_gmetric_create( Ganglia_pool parent_pool ); int Ganglia_gmetric_set( Ganglia_gmetric gmetric, char *name, char *value, char *type, char *units, unsigned int slope, unsigned int tmax, unsigned int dmax); int Ganglia_gmetric_send( Ganglia_gmetric gmetric, Ganglia_udp_send_channels send_channels ); +// Yemi +int Ganglia_gmetric_send_spoof( Ganglia_gmetric gmetric, Ganglia_udp_send_channels send_channels, char* spoof_info,int heartbeat); void Ganglia_gmetric_destroy( Ganglia_gmetric gmetric ); void build_default_gmond_configuration(Ganglia_pool context); diff -udpr ganglia-3.0.3/lib/libgmond.c ganglia-new/lib/libgmond.c --- ganglia-3.0.3/lib/libgmond.c 2006-04-17 07:42:55.000000000 -0700 +++ ganglia-new/lib/libgmond.c 2006-05-16 14:32:58.000000000 -0700 @@ -700,6 +700,59 @@ Ganglia_gmetric_send( Ganglia_gmetric gm return Ganglia_udp_send_message( send_channels, gmetricmsg, len); } +// Yemi +int +Ganglia_gmetric_send_spoof( Ganglia_gmetric gmetric, Ganglia_udp_send_channels send_channels, char* spoof_info, int heartbeat) +{ + int len; + XDR x; + char gmetricmsg[GANGLIA_MAX_MESSAGE_LEN]; + Ganglia_message msg; + char *spoofName; + char *spoofIP; + char *buff; + int spoof_info_len; + int result; + + spoof_info_len = strlen(spoof_info); + buff = malloc(spoof_info_len+1); + strcpy(buff,spoof_info); + spoofIP = buff; + if( !(spoofName = strchr(buff+1,':')) ){ + fprintf(stderr,"Incorrect format for spoof argument. exiting.\n"); + exit(1); + } + *spoofName = 0; + spoofName++; + if(!(*spoofName)){ + fprintf(stderr,"Incorrect format for spoof argument. exiting.\n"); + exit(1); + } + printf(" spoofName: %s spoofIP: %s \n",spoofName,spoofIP); + + if(heartbeat){ + msg.id = spoof_heartbeat; + msg.Ganglia_message_u.spheader.spoofName = spoofName; + msg.Ganglia_message_u.spheader.spoofIP = spoofIP; + }else{ + msg.id = spoof_metric; + msg.Ganglia_message_u.spmetric.spheader.spoofName = spoofName; + msg.Ganglia_message_u.spmetric.spheader.spoofIP = spoofIP; + msg.Ganglia_message_u.spmetric.gmetric = *(gmetric->msg); + } + + // memcpy( &(msg.Ganglia_message_u.gmetric), gmetric->msg, sizeof(Ganglia_gmetric_message)); + + /* Send the message */ + xdrmem_create(&x, gmetricmsg, GANGLIA_MAX_MESSAGE_LEN, XDR_ENCODE); + xdr_Ganglia_message(&x, &msg); + len = xdr_getpos(&x); + result = Ganglia_udp_send_message( send_channels, gmetricmsg, len); + free(buff); + return result; + +} + void Ganglia_gmetric_destroy( Ganglia_gmetric gmetric ) { diff -udpr ganglia-3.0.3/lib/protocol.x ganglia-new/lib/protocol.x --- ganglia-3.0.3/lib/protocol.x 2006-04-17 07:42:55.000000000 -0700 +++ ganglia-new/lib/protocol.x 2006-05-16 12:14:52.000000000 -0700 @@ -24,6 +24,16 @@ struct Ganglia_gmetric_message { unsigned int dmax; }; +/* Yemi */ +struct Ganglia_spoof_header { + string spoofName<>; + string spoofIP<>; +}; +struct Ganglia_spoof_message { + struct Ganglia_spoof_header spheader; + struct Ganglia_gmetric_message gmetric; +}; + /* ** When adding a new core metric you need to make three changes ** in this file: @@ -92,13 +102,21 @@ enum Ganglia_message_formats { metric_mem_rm, metric_mem_avm, metric_mem_vm, - GANGLIA_NUM_25_METRICS /* this should always directly follow the last 25 metric_* */ + GANGLIA_NUM_25_METRICS, /* this should always directly follow the last 25 metric_* */ +/* Yemi */ + spoof_metric = 4096, + spoof_heartbeat = 4097 }; union Ganglia_message switch (Ganglia_message_formats id) { case metric_user_defined: Ganglia_gmetric_message gmetric; - +/* Yemi */ + case spoof_metric: + Ganglia_spoof_message spmetric; + case spoof_heartbeat: + Ganglia_spoof_header spheader; + case metric_cpu_num: /* xdr_u_short */ unsigned short u_short;