Re: [collectd] v5upgrade target causing high IO wait, no RRD updates
Not sure about debian packaging ... but 5.4 rpm's on centos package the collectd rrdcached plugin separately. Check it's there. Check the rrdcached daemon and collectd config are referencing the same pipe. One thing I did notice ... new rrd db files are created with the root owner, rrdcached is running as low priv, implying that collectd is creating the rdd dirs, and then rrdcached cannot write to them. Cron re-chown fix is not very elegant but does work. My 5.4 + rrdcached 4.7 is now running pretty smoothly and I'm really happy about all the IO wait going away :) Regards, Andrew Gray. RHCSA, Professional Unix Administration. -Original Message- From: collectd-boun...@verplant.org [mailto:collectd-boun...@verplant.org] On Behalf Of Jesse Reynolds Sent: Tuesday, 8 April 2014 4:52 PM To: collectd@verplant.org Subject: [collectd] v5upgrade target causing high IO wait, no RRD updates Hi all, I've just upgraded one of our collectd servers from 4.10.1 to 5.4.1 with rrdcached (1.4.7). Woo! One thing that surprised me was that if I had the v5upgrade target enabled, then RRDs weren't being updating (or not that I could discern) and top was saying 99% IO wait (compare about 10 to 20% without v5upgrade). So I'm currently running it without the v5upgrade target. Has anyone seen anything like this? Or have any idea what might be going on to cause this? I didn't see this behaviour on my testing prior to the upgrade. The main difference is in the size of the RRD files. I also ran `iostat 5` when testing and the IO wait stayed at about 99% consistently even when there were periods of no disk activity to speak of. This is on ubuntu precise 12.04, 64 bit, no virtualisation. TIA Jesse ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd To report this email as spam, please forward to a...@websense.com * This email message (including any file attachments transmitted with it) is for the sole use of the intended recipient(s) and may contain confidential and legally privileged information. Any unauthorised review, use, alteration, disclosure or distribution of this email (including any attachments) by an unintended recipient is prohibited. If you have received this email in error, please notify the sender by return email and destroy all copies of the original message. Any confidential or legal professional privilege is not waived or lost by any mistaken delivery of the email. SPARQ Solutions accepts no responsibility for the content of any email which is sent by an employee which is of a personal nature. Sender Details: SPARQ Solutions PO Box 15760 City East, Brisbane QLD Australia 4002 +61 7 4931 SPARQ Solutions policy is to not send unsolicited electronic messages. Suspected breaches of this policy can be reported by replying to this message including the original message and the word UNSUBSCRIBE in the subject. * ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] v5upgrade target causing high IO wait, no RRD updates
Hi Andrew, So I've used omnibus to build a package containing rrdtool 1.4.7 and collectd 5.4.1, and it all works fairly nicely with rrdcached if I don't employ the v5upgrade target. I am seeing a lot of rrdcached errors due to it being unable to update RRD files that were created with times in the far distant future and that kind of thing, however. Are you using the v5upgrade target? Or you have no v4 clients, or it's ok to live with the old metric naming schema for df, mysql etc? Jesse On 8 Apr 2014, at 16:41:40, GRAY Andrew G (SPARQ) andrew.gr...@sparq.com.au wrote: Not sure about debian packaging ... but 5.4 rpm's on centos package the collectd rrdcached plugin separately. Check it's there. Check the rrdcached daemon and collectd config are referencing the same pipe. One thing I did notice ... new rrd db files are created with the root owner, rrdcached is running as low priv, implying that collectd is creating the rdd dirs, and then rrdcached cannot write to them. Cron re-chown fix is not very elegant but does work. My 5.4 + rrdcached 4.7 is now running pretty smoothly and I'm really happy about all the IO wait going away :) Regards, Andrew Gray. RHCSA, Professional Unix Administration. -Original Message- From: collectd-boun...@verplant.org [mailto:collectd-boun...@verplant.org] On Behalf Of Jesse Reynolds Sent: Tuesday, 8 April 2014 4:52 PM To: collectd@verplant.org Subject: [collectd] v5upgrade target causing high IO wait, no RRD updates Hi all, I've just upgraded one of our collectd servers from 4.10.1 to 5.4.1 with rrdcached (1.4.7). Woo! One thing that surprised me was that if I had the v5upgrade target enabled, then RRDs weren't being updating (or not that I could discern) and top was saying 99% IO wait (compare about 10 to 20% without v5upgrade). So I'm currently running it without the v5upgrade target. Has anyone seen anything like this? Or have any idea what might be going on to cause this? I didn't see this behaviour on my testing prior to the upgrade. The main difference is in the size of the RRD files. I also ran `iostat 5` when testing and the IO wait stayed at about 99% consistently even when there were periods of no disk activity to speak of. This is on ubuntu precise 12.04, 64 bit, no virtualisation. TIA Jesse ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
