I had a bit of a hack with the rrdcached code yesterday; I added in a PAUSE and RESUME command which temporarily stops and restarts the write threads while still accepting (and queueing) the pending updates, so that nothing is lost during the backups.
Its still rather rough and (1) it pauses EVERYTHING so you have to watch how big the in-memory queue gets, and (2) during my test, there were 10,000+ updates queued over a 20min pause, but for some reason 4 writes were still done. I haven't yet found out why. So, its not stable yet, though it's definitely got possibilities. I can send you a copy of the modified code if you want though I'd certinaly not advise putting it into a production environment yet! Steve Steve Shipway University of Auckland ITS UNIX Systems Design Lead [email protected]<mailto:[email protected]> Ph: +64 9 373 7599 ext 86487 ________________________________ From: Jochen Bern [[email protected]] Sent: Wednesday, 27 October 2010 9:08 p.m. To: [email protected] Cc: Steve Shipway Subject: Re: [rrd-users] Locking *read* (copy/backup) of RRD files? On 10/27/2010 02:32 AM, Steve Shipway wrote: > however I'd use a filesystem snapshot so the time during which the > rrdcached is inoperative would be measured in seconds That is, of course, the authoritative solution to the problem of doing full backups - and would require a full reinstall in my case because a need for snapshot capability wasn't anticipated at OS install time. :-} (Someone else did it, resulting in ext3 on LVM with no unused physical extents ...) However, it's not too difficult to imagine scenarios where one would *still* want consistent duplication on a single-RRD-file level; e.g., thanks to n2rrd in full auto mode, I get to go through the cycle of rrddump;vi;mv;rrdrestore every now and then to remove bogus DSs ... >> As a matter of fact, I'm a bit surprised to see that rrdresize seems to >> be the *only* part of rrdtool which sets a(ny) lock while *reading* an >> RRD file. > I'd need to check, but possibly since the RRD file size is constant > (except when running rrdresize) it means that since reads and writes > are atomic, the reads do not need to be locked as you will always get > consistent data? In 1.4.4 src/rrd_open.c::rrd_open(), I see eight __rrd_read() ; these read out of an mmap() if available, and do an actual read() else. My mmap(2) manpage (OpenSuSE 11.2) warns me that it "is unspecified whether changes made to the file after the mmap() call are visible in the mapped region", not to even dream about any kind of atomicity ... Kind regards, J. Bern -- Jochen Bern, Systemingenieur --- LINworks GmbH <http://www.LINworks.de/<http://www.linworks.de/>> Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Geschäftsführer Metin Dogan, Oliver Michel
_______________________________________________ rrd-users mailing list [email protected] https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
