re-sending in 2 parts to get around the 40k moderator approval gate.
On Thu, 25 Feb 2010, Scott Dworkis wrote:
finally had a chance to finish up an rrdcached config yesterday. comparison:attached some 1 week graphs showing resource usage of proc/sys/vm tuning before and rrdcached after last night. i reverted my proc/sys/vm customizations to the defaults and set up rrdcached to what i felt was comparable to the proc/sys/vm settings i used, basically a 2 hour write-back schedule (i.e. -w 7200 for rrdcached as opposed to proc/sys/vm/dirty_expire_centisecs to 720000).the average cpu and load is just a hair higher, the 5 minute max cpu and load are about 50% higher (i.e. rrdcached is a little choppier). disk load seems a little lower and wider... probably equivalent on average. memory that was used by fs cache is now used by rrdcached of course... i think this means the data exists in 2 places, fs cache and rrdcached, which doesn't seem strictly optimal but is probably benign since the os would reclaim fs cache if it ever needed it for other processes... though perhaps there is some overhead there (probably trivial).you might notice the triangle memory leak in gmond... i have a thread going elsewhere for that and will have a look with valgrind.someone may want to update the guide: http://sourceforge.net/apps/trac/ganglia/wiki/rrdcached_integrationi found that following that guide literally didn't work... i needed to specify -l explicity. even though the socket file is created and writable by all, i was getting permissions errors without -l.also running make on rrdtool-1.4.2 surprised me a little when it called sync, which took a long time on my heavily cached ganglia server. i shut down gmetad to try to get make to finish a little faster.in the end the tradeoff with rrdcached is that you isolate the caching policy to just ganglia and not the wider system, which seems like a good/safer thing. like Kostas noted, besides customizing init.d scripts and default ganglia php files to make them rrdcached aware, you also have to configure rrdcached for anything else that intends to access up to date data in the rrds... a little more config work.-scott(i replied to the digest here... hopefully i didn't screw up threading somehow)------------------------------ Message: 5 Date: Fri, 19 Feb 2010 11:08:41 -0800 From: Bernard Li <[email protected]> Subject: Re: [Ganglia-general] experiences scaling to 150k metrics To: Scott Dworkis <[email protected]> Cc: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1 Hi Scott: On Thu, Feb 18, 2010 at 7:41 PM, Scott Dworkis <[email protected]> wrote:i tried the tmpfs thing (actually /dev/ramN), but decided syncing back andforth to disks to preserve data across reboots was suboptimal. ?never triedrrdcached.Yeah that's one of the downside of the tmpfs route. Definitely give rrdcached a try and report back your experiences. Cheers, Bernard ------------------------------
<<attachment: avg_load.png>>
<<attachment: avg_disk_use.png>>
<<attachment: avg_cpu.png>>
------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev
_______________________________________________ Ganglia-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ganglia-general
