Aroop Maliakkal schrieb: > It might be worth checking the disk IO... Disk IO is the main > constraint on monitoring clusters/grids actually. > > On Dec 12, 2007 5:42 PM, Markus Reusch < [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> wrote: > > Hi list(eners), > > we are using Ganglia for about 20-30 AIX-Systems. All data is > collected > on 2 LPARs which run gmetad, rrdtool and the Apache, so it's > accessible > for the users with the supplied PHP web frontend. > Our rrd-database has currently a size of about 400 MB and consists of > about 4800 files. I have no idea if this size or number of files is a > problem at all. > Every day, several users connect to the web frontend and open those > graphs, which results in some rrdtool calls, exhausting all the > available CPU time of the machine. No matter if it's just 4 rrdtool > processes or 20 of them. The machine is already at it's limit when > working with 4 rrdtool calls. When about 20 of them are started, a 2 > digit number of kernel threads is being queued... > When all this occurs, you are not able to type properly in your ssh > session and even some shell scripts being called by cron are having > problems and give delayed output. This is no peak behaviour - it's > constantly like this. > > The LPAR has AIX 5.3 with 1 * 1,7 MHz CPU and 2 GB RAM. VMM is > tuned and > machine is usually not paging at all. Not a strong machine but we > thought sufficient for Ganglia + rrdtool + Apache. > > Other applications on the machine are not noticable, as you can > see (topas): > Name PID CPU% PgSp Owner > rrdtool 458924 26.0 0.5 nobody > rrdtool 966772 25.4 0.4 nobody > rrdtool 1302770 23.0 3.6 root > rrdtool 1183858 22.4 0.4 nobody > gmetad 1110100 8.2 5.3 nobody > topas 1159188 0.6 1.7 root > gmond 352448 0.1 1.7 nobody > java 848106 0.0 16.2 root > sched 12294 0.0 0.4 root > clstrmgr 405650 0.0 22.5 root > > > > A snippet from vmstat to see yourself (currently 4 rrdtools are > running): > kthr memory page faults cpu > ----- ----------- ------------------------ ------------ ----------- > r b avm fre re pi po fr sr cy in sy cs us sy id wa > 5 1 304078 4075 0 0 0 0 0 0 285 22896 1079 90 10 > 0 0 > 6 1 303783 4373 0 0 0 0 0 0 249 2038 787 98 2 0 0 > 6 0 304004 4145 0 0 0 0 0 0 239 17285 1001 92 8 0 0 > > > Here is a sample process of rrdtool taken with ps: > /usr/bin/rrdtool graph --start 1197456160 --end 1197459760 --width > 300 > --height 75 --title weight - DEF:sum=/var/lib/ganglia/rrds/PL1650 > Linie > SRZ/<somehostname>/weight.rrd:sum:AVERAGE > AREA:sum#0000ff:<somehostname> > last hour (now -1.00) > > My questions: > Is it normal behaviour/configuration, that so many rrdtools are > started > when people are doing some requests to the web frontend? > Do you guys spend a "big" box just running Ganglia + rrdtool + > Apache or > maybe just a small Intel/Linux box and it runs smooth? > > Any suggestions/hints are welcome, thank you in forward. > > Greetings > Markus > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Ganglia-general mailing list > [email protected] > <mailto:[email protected]> > https://lists.sourceforge.net/lists/listinfo/ganglia-general > <https://lists.sourceforge.net/lists/listinfo/ganglia-general> > > > > > -- > ~=~=~=~=~=~=~=~=~=~=~=~=~=~=~= > Regards, > Aroop > ~=~=~=~=~=~=~=~=~=~=~=~=~=~=~= Hi Aroop,
thank you for the hint - we don't have a disk issue. I/O wait is about zero and iostat shows only low traffic on the disks both on the percentage of time for interval being used and the bandwith and transfers. They are all low. Meanwhile I found this in the archives: http://sourceforge.net/mailarchive/forum.php?forum_name=ganglia-general&max_rows=25&style=nested&viewmonth=200712 So I am trying to compile the latest stable version of rrdtool 1.2.26. Later Markus -- Gruß Markus Reusch -- Debeka Krankenversicherungsverein a. G. Debeka Lebensversicherungsverein a. G. Debeka Allgemeine Versicherung AG Hauptverwaltung, Abteilung IS/FK Ferdinand-Sauerbruch-Str. 18 56058 Koblenz Telefon (0261) 498 - 2227 Telefax (0261) 498 - 3160 E-Mail [EMAIL PROTECTED] Internet www.debeka.de Vorstand: Uwe Laue (Vorsitzender), Rolf Florian, Roland Weber, Thomas Brahm (st. V.), Dr. Peter Görg (st. V.) Aufsichtsratsvorsitzender: Peter Greisler Debeka Krankenversicherungsverein a. G. - AmtsG Koblenz: HRB 125 Debeka Lebensversicherungsverein a. G. - AmtsG Koblenz: HRB 141 Debeka Allgemeine Versicherung AG - AmtsG Koblenz: HRB 2300 Debeka Pensionskasse AG - AmtsG Koblenz: HRB 6683 Sitz Koblenz am Rhein ------------------------------------------------------------------------- SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ Ganglia-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ganglia-general
