David Brown wrote: > On Sun, Jan 27, 2008 at 01:47:18PM -0800, kelsey hudson wrote: >> David Brown wrote: >>> I run the cyrus imap server as my mail server. One of the things it >>> maintains is a database is visited message ids, to avoid duplicate >>> message >>> delivery. >>> This database is maintained with Berkeley DB, configured to write a >>> logfile >>> and several __db.001 through __db.006 files. >>> The problem I'm having is that I can't back these files up reliably. >> >> What are you using to do your backups? > > It doesn't matter what I use. If the mtime/ctime of the file doesn't > change, no backup program is going to realize that it needs to back them up > again. > > Right now I'm using GNU tar, because it is the _only_ backup software I can > find that understands how to do incremental backups correctly. > >>> - Cyrus somehow seems to be able to change these files without >>> changing >>> the ctime or mtime. Without the change, my backup doesn't think >>> they've changed. >>> - Even on exit, they don't get touched. A restart of imapd does touch >>> the files. >>> I'm wondering if I should worry about it. Perhaps the log file contains >>> enough information to reconstruct whatever state the db files should >>> be in >>> when they were last touched. It also might not matter, the worst case >>> being a few days of duplicate messages being delivered. >>> What I'm really wondering is how the program can be modifying the files >>> without touching the timestamps. I tried using mmap myself, and I can't >>> change the data without updating the mtime. >> >> MMIO and O_DIRECT should allow this. > > I'll have to study that path through the kernel. Even with O_DIRECT, page > cache flushes should still be going through the file interface. I'm not > sure Linus is real interested in fixing it, since he feels that O_DIRECT > needs to go away, and databases should be using madvise. > > Right now, I suspend the mail server, touch the database files, back it up, > and then restart mail/imap. >
Isn't this a classic database backup problem. You somehow need to make the database consistent for the duration of the backup, or get a snapshot of a consistent state. Databases often have a "quiesce" operation, do they not (which flushes and locks all writes until released from the quiesced state). Maybe what you are doing is the only good way for this particular DB? Regards, ..jim -- [email protected] http://www.kernel-panic.org/cgi-bin/mailman/listinfo/kplug-list
