Re: Qmail's queue directory & Linux
-Original Message- From: Petri Kaukasoina <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]> Date: Wednesday, June 23, 1999 6:24 PM Subject: Re: Qmail's queue directory & Linux >On Wed, Jun 23, 1999 at 10:48:39AM +0300, Petri Kaukasoina wrote: > >I got home and tried it myself. To my surprise, the "chattr +S" trick did >work. If testdir had the "S" attribute, testfile was in testdir after reboot >but if the directory didn't have the attribute, fsck put the file in >lost+found. It worked in Linux versions 2.0.37 and 2.2.10. Either something >has changed in the kernel since I tried it last time or I did some mistake >then. > >So, it seems that it's enough to >chattr +S /var/qmail/queue/{bounce,info/*,intd,local/*,mess/*,remote/*,todo} >and the same for the new and cur subdirectories in every maildir. Ah, that's a relief. It saves me from having to rearrange my partitions. Thanks for checking... Regards, Peter van der Landen
Re: Qmail's queue directory & Linux
On Wed, Jun 23, 1999 at 10:48:39AM +0300, Petri Kaukasoina wrote: > 7. ../testprog > 8. switch off the power > 9. switch on the power > 10. after fsck look for the file "testfile" in the directories testdir and > lost+found (under the mount point of the file system) > > If the chattr trick worked you would have "testfile" in "testdir". Otherwise > you would find it in lost+found. I got home and tried it myself. To my surprise, the "chattr +S" trick did work. If testdir had the "S" attribute, testfile was in testdir after reboot but if the directory didn't have the attribute, fsck put the file in lost+found. It worked in Linux versions 2.0.37 and 2.2.10. Either something has changed in the kernel since I tried it last time or I did some mistake then. So, it seems that it's enough to chattr +S /var/qmail/queue/{bounce,info/*,intd,local/*,mess/*,remote/*,todo} and the same for the new and cur subdirectories in every maildir.
Re: Qmail's queue directory & Linux
Petri Kaukasoina writes: > On Wed, Jun 23, 1999 at 08:04:53AM +0200, Peter van der Landen wrote: > > The Cyrus documentation suggests using chattr -R +S on the queue directories > > as well as the mailboxes. Would that be less effective than using a > > sync-mounted filesystem? > > Dan fsyncs explicitely all the _files_ when needed, so there's no problem > with them. But in Linux the metadata, e.g. the directory entries are not > written synchronously by default. Suppose qmail receives a mail message, > writes it to the queue, fsyncs it and reports mail received. In case of > power failure at this point the file was written ok but its directory entry > was not on disk yet. After bootup fsck puts the message to /lost+found > where you can find it but it is not in the qmail queue. > > The solution that Linus proposed was to fsync the _directory_ too in > addition to the _file_ and that is what my shared library does. Actually it > fsyncs the directories every time when link, unlink, open and rename system > calls are used (they update directory entries), needed or not. Right. That's because it's a stupid idea to fsync metadata. A directory is just another file -- why should it automatically be fsynced just because a different file got fsynced? You want your directories written to disk, you fsync them. No, I don't worship Karels, et al. The Berkeley CSRG made some mistakes. Instead of perpetuating these mistakes, as if they were physical properties handed to us by God (186K miles/sec -- not just a good idea, it's the law), Linus is trying to fix them. Yes, it's another "gratuitious" incompatibility, but it has serious efficiency ramifications. Just like Dan is introducing "gratuitious" incompatibilities. :) Only difference is that, unlike vendor gratuitious incompatibilities, I *like* Dan's incompatibilities. I wish Dan had the time to put together a Linux distribution. -- -russ nelson <[EMAIL PROTECTED]> http://crynwr.com/~nelson Crynwr supports Open Source(tm) Software| PGPok | Government schools are so 521 Pleasant Valley Rd. | +1 315 268 1925 voice | bad that any rank amateur Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | can outdo them. Homeschool!
Re: Qmail's queue directory & Linux
On Wed, Jun 23, 1999 at 09:49:27AM +0300, Petri Kaukasoina wrote: > You can try it yourself to see if it works or not with kernels today: > ... Actually that might not be a valid test because file2 was not fsynced. It's hard to remember what was done a year ago... But I used a program similar to the attached one to create and fsync the test file. So this might be valid now. 1. gcc -o testprog testprog.c 2. mkdir testdir 3. chattr +S testdir 4. cd testdir 5. killall -KILL update 6. sync 7. ../testprog 8. switch off the power 9. switch on the power 10. after fsck look for the file "testfile" in the directories testdir and lost+found (under the mount point of the file system) If the chattr trick worked you would have "testfile" in "testdir". Otherwise you would find it in lost+found. #include #include #include #include #include int main() { int fd; char *message = "testing\n"; if ((fd=open("testfile",O_CREAT|O_RDWR|O_EXCL,0666))==-1) perror("open"); if (write(fd,message,strlen(message))==-1) perror("write"); if (fsync(fd)==-1) perror("fsync"); if (close(fd)==-1) perror("close"); return 0; }
Re: Qmail's queue directory & Linux
On Wed, Jun 23, 1999 at 08:04:53AM +0200, Peter van der Landen wrote: > The Cyrus documentation suggests using chattr -R +S on the queue directories > as well as the mailboxes. Would that be less effective than using a > sync-mounted filesystem? Dan fsyncs explicitely all the _files_ when needed, so there's no problem with them. But in Linux the metadata, e.g. the directory entries are not written synchronously by default. Suppose qmail receives a mail message, writes it to the queue, fsyncs it and reports mail received. In case of power failure at this point the file was written ok but its directory entry was not on disk yet. After bootup fsck puts the message to /lost+found where you can find it but it is not in the qmail queue. The solution that Linus proposed was to fsync the _directory_ too in addition to the _file_ and that is what my shared library does. Actually it fsyncs the directories every time when link, unlink, open and rename system calls are used (they update directory entries), needed or not. Dan's suggestion is to mount all the file systems with option "sync". It makes all writing to that file system synchronous which might be a bit overkill when the problem is only with metadata. chattr +S is a good idea but it doesn't work. I guess it works for _files_: a file with that attribute is written synchronously. But there is no problem in that field in case of qmail because Dan uses fsyncs. (With sendmail, procmail, poppers and imappers it may be different.) But chattr +S does not make writing the directory entries synchronous which is the problem. I tested it a year ago and it didn't work for Linux kernels in use at that time. You can try it yourself to see if it works or not with kernels today: 1. mkdir testdir1 testdir2 2. chattr +S testdir2 3. killall -KILL update 4. sync 5. echo "without -S" > testdir1/file1 6. echo "with -S" > testdir2/file2 7. switch off the power 8. switch on the power 9. after fsck look for the files in the directories /lost+found, testdir1 and testdir2. If the chattr trick worked you would have the file "file2" in the directory testdir2.
Re: Qmail's queue directory & Linux
-Original Message- From: Petri Kaukasoina <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]> Date: Wednesday, June 23, 1999 7:22 AM Subject: Re: Qmail's queue directory & Linux >On Tue, Jun 22, 1999 at 05:22:46PM -0700, Russell Steffen wrote: >> This obviously implies that having the queue on a separate partition or >> disk for Linux systems is a good thing (so that it can be mounted with >> "sync"). > >If you want to do that then you want a sync filesystem for the >Maildirs/mboxes too. That's what the Cyrus documentation suggests. The Cyrus documentation suggests using chattr -R +S on the queue directories as well as the mailboxes. Would that be less effective than using a sync-mounted filesystem? Regards, Peter
Re: Qmail's queue directory & Linux
On Tue, Jun 22, 1999 at 05:22:46PM -0700, Russell Steffen wrote: > This obviously implies that having the queue on a separate partition or > disk for Linux systems is a good thing (so that it can be mounted with > "sync"). If you want to do that then you want a sync filesystem for the Maildirs/mboxes too.
Re: Qmail's queue directory & Linux
On 23 Jun 1999, D. J. Bernstein wrote: > Peter van der Landen writes: > > Should I conclude from this that somehow Linux is extra vulnerable to queue > > corruption > > Yes. See http://pobox.com/~djb/qmail/faq/reliability.html for some > discussion of this issue. > > ---Dan This obviously implies that having the queue on a separate partition or disk for Linux systems is a good thing (so that it can be mounted with "sync"). Are there any guidelines for how big a queue filesystem should be? And how gracefully does qmail handle a full queue fs ? Russ -- Russ Steffen [EMAIL PROTECTED]
Re: Qmail's queue directory & Linux
Peter van der Landen writes: > Should I conclude from this that somehow Linux is extra vulnerable to queue > corruption Yes. See http://pobox.com/~djb/qmail/faq/reliability.html for some discussion of this issue. ---Dan
Re: Qmail's queue directory & Linux
On Sat, Jun 19, 1999 at 12:07:11PM +0200, Peter van der Landen wrote: > >Also set the queue directory of the mail daemon to update > >synchronously. The following example is for sendmail: > > > > /sbin/chattr +S /var/spool/mqueue > > Should I conclude from this that somehow Linux is extra vulnerable to queue > corruption (there's no such instruction for other OS'es). If so, why doesn't > Dan (who I know as a very careful person) recommend this? > > Is it a good idea to apply this +S attribute to the Qmail queue directory? > Would it have a high impact on performance? In Linux, in case of power loss you can lose the mail that you received in the last 30 seconds. The chattr trick was suggested a year ago on this list. I tried it but it didn't help at all in case of directories. I guess the attribute only works for files which are opened, not for directories where files are opened. I have no idea if something has changed after that. But there's a little shared library which adds extra fsync() system calls and which should make qmail on Linux reliable: ftp://elektroni.ee.tut.fi/pub/qmail_linux_metadata_message (the included patch for the kernel is not needed any longer)
Qmail's queue directory & Linux
L.S, The following section is from the Installing & Configuring page of the Cyrus IMAP server: >8.LINUX SYSTEMS ONLY: Set the configuration, user, quota, and partition >directories to update synchronously. >Failure to do this may lead to data corruption and/or loss of mail >after a system crash. > > cd /var/imap > chattr +S . user quota > chattr +S /var/spool/imap > >Also set the queue directory of the mail daemon to update >synchronously. The following example is for sendmail: > > /sbin/chattr +S /var/spool/mqueue Should I conclude from this that somehow Linux is extra vulnerable to queue corruption (there's no such instruction for other OS'es). If so, why doesn't Dan (who I know as a very careful person) recommend this? Is it a good idea to apply this +S attribute to the Qmail queue directory? Would it have a high impact on performance? Best regards, Peter van der Landen