We also had this problem, using the trunk version of 1.4.4, but I thought it was due to a separate memory leak issue (now fixed in trunk).
If pthread_create fails then you're out of resources, possibly memory or threads... check your system thread/process limits are not causing issues, and that you have sufficient memory. Keep an eye on how much memory rrdcached is using and see if it starts getting unfeasibly larger than it is after the first hour. I've also found that the MRTG daemon (which is writing to rrdcached) can become confused and require restarting at this point, too. Steve Steve Shipway University of Auckland ITS UNIX Systems Design Lead [email protected] Ph: +64 9 373 7599 ext 86487 ________________________________________ From: [email protected] [[email protected]] on behalf of Ulf Zimmermann [[email protected]] Sent: Sunday, 31 October 2010 6:33 p.m. To: '[email protected]' Subject: [rrd-users] rrdcached issues with larger number of clients via network/pthread I got close to 300 machines running collectd, configured to use unixsocks to rrdcached on a central server. We are running more and more into threads dieing (collectd then starts complaining and fills up /var/messages) and when we try to restart collectd, sometimes it works, sometimes we end up with: Oct 30 22:27:19 log02 rrdcached[16864]: listen_thread_main: pthread_create failed. Oct 30 22:27:34 log02 rrdcached[16864]: listen_thread_main: pthread_create failed. Oct 30 22:28:10 log02 rrdcached[16864]: listen_thread_main: pthread_create failed. And at this point we usual have to restart the rrdcached daemon, which then means having to restart collectd on close to 300 machines. How can this be debugged to find the issue (potential inside of pthreads). The central server is running RedHat EL5 Update 4, the rrdtool/rrdcached is 1.4.4 from rpmforge. Ulf, who is getting more grey hair by the minute with issues like this :-( _______________________________________________ rrd-users mailing list [email protected] https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users _______________________________________________ rrd-users mailing list [email protected] https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
