RE: Vdelivermail, Maildir support and NFS 'safety'
Hi -
I had a look at the new code and it looks much closer to the Maildir
algorithm now. I think you could still put a call to fsync() before
the file close() function call to ensure the file has been written to
the drive and not to the local cache. This gives you something like:
/* done with the new file */
if ( is_file == 1) {
if(fsync(write_fd) == -1 || close(write_fd) == -1) {
if ( unlink(local_file) != 0 ) {
printf(unlink failed %s errno = %d\n,
local_file, errno);
return(errno);
}
printf(fsync/close failed %s errno = %d\n,
local_file, errno);
return(errno);
}
} else
close(write_fd);
(around line 529 of vdelivermail.c). Now you can pretty much guarantee
that the tmp file is written to the drive. You may be able to use
fdatasync() if you want to optimise this as that will leave out a
write operation to update the modification time on the inode and just
writes the data out to disk but how much time that will save I dont
know.
Hope this is of use,
Marcus
-Original Message-
From: Ken Jones [mailto:[EMAIL PROTECTED]]
Sent: 05 June 2001 15:56
To: [EMAIL PROTECTED]
Subject: Re: Vdelivermail, Maildir support and NFS 'safety'
Oops, I forgot to put a url. It has been a long day
already.
http://www.inter7.com/vpopmail/vpopmail-4.10.15.tar.gz
Re: Vdelivermail, Maildir support and NFS 'safety'
Marcus Williams writes: Hi - [this message is probably only of interest to people running a development version of vpopmail (post 4.10.3) with their domains on NFS mounted drives] I noticed in a recent version of vpopmail (version 4.10.3), the delivery method (deliver_mail) has stopped delivering to the Maildir/tmp (from now on referred to as /tmp) directory first in order to remove a system call to rename from the code (the /tmp file was then linked to Maildir/new - referred to from now on as /new). I've been looking at the new code (version 4.10.14) and it appears to be going straight to the /new directory now which is incorrect if you are trying to follow the Maildir algorithm (that ensures safe/guaranteed delivery, requires no locking under NFS etc). The whole point of delivering first to /tmp and then linking this file to /new is to stop corrupt messages ever hitting Maildir/new and to ensure reliable delivery under NFS failures/machine crashes etc. Once a message is linked from /tmp to /new (and then stat()'d to make sure its there) the idea is that it guaranteed not to be incomplete/undelivered. After linking it to the /new area, assuming there is no error we can remove the /tmp file. With the development versions (4.10.3+) if the server crashes during delivery (mid file write) this will leave a corrupt message on the server in the /new delivery area, something the Maildir format is designed not to do.. Admittedly Qmail will probably try and redeliver the message as it has not been delivered successfully but without the initial delivery to /tmp a user can receive corrupt messages. There is also a bug for NFS in that fsync() (or fdatasync?) is not called on the file vdelivermail writes to in the delivery area so you do not guarantee it has actually been written. This means the file can be successfully delivered according to vdelivermail but in reality only locally cached and not written out (commited?) to the NFS drive. Vdelivermail will have returned success but if the machine crashes at this point the mail can potentially disappear if the NFS mount doesnt recover properly. Is there any reason (other than optimisation) that these calls have been removed (or in the case of the missing fsync not implemented)? Yes, there is another reason. I like to get about 8 hours of sleep a night :) Ken Jones
RE: Vdelivermail, Maildir support and NFS 'safety'
Hi - Thats fair enough! Okay, so the next question is - does this matter at all for vpopmail? If it does I can put a patch together that does this and submit it to you if you want. If it doesnt then I can put a patch up on a website if anyone wants it which I'll try to keep up to date with vpopmail stable versions (which I tend to follow with our production box). Obviously the first solution is better since it'll get more testing in the development versions. Marcus -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: 05 June 2001 14:04 To: [EMAIL PROTECTED] Subject: Re: Vdelivermail, Maildir support and NFS 'safety' [snip] Is there any reason (other than optimisation) that these calls have been removed (or in the case of the missing fsync not implemented)? Yes, there is another reason. I like to get about 8 hours of sleep a night :) Ken Jones
Re: Vdelivermail, Maildir support and NFS 'safety'
Marcus Williams writes: I think this new version will be more to your liking. Please review the code and see if it looks okay to you. I tested it on our NFS system and it seemed to work. But I have not volume tested it. Ken PS: Thanks for bringing this up. It needed to be updated. Hi - Thats fair enough! Okay, so the next question is - does this matter at all for vpopmail? If it does I can put a patch together that does this and submit it to you if you want. If it doesnt then I can put a patch up on a website if anyone wants it which I'll try to keep up to date with vpopmail stable versions (which I tend to follow with our production box). Obviously the first solution is better since it'll get more testing in the development versions. Marcus
Re: Vdelivermail, Maildir support and NFS 'safety'
Oops, I forgot to put a url. It has been a long day already. http://www.inter7.com/vpopmail/vpopmail-4.10.15.tar.gz Ken Jones writes: I think this new version will be more to your liking. Please review the code and see if it looks okay to you. I tested it on our NFS system and it seemed to work. But I have not volume tested it. Ken PS: Thanks for bringing this up. It needed to be updated. Hi - Thats fair enough! Okay, so the next question is - does this matter at all for vpopmail? If it does I can put a patch together that does this and submit it to you if you want. If it doesnt then I can put a patch up on a website if anyone wants it which I'll try to keep up to date with vpopmail stable versions (which I tend to follow with our production box). Obviously the first solution is better since it'll get more testing in the development versions. Marcus
RE: Vdelivermail, Maildir support and NFS 'safety'
I am just moving to a NFS server. This is something I would definately want. Please let me know what becomes of this. --- Brad Dameron Network Account Executive TSCNet Inc. www.tscnet.com Silverdale, WA. 1-888-8TSCNET -Original Message- From: Marcus Williams [mailto:[EMAIL PROTECTED]] Sent: Tuesday, June 05, 2001 6:56 AM To: [EMAIL PROTECTED] Subject: RE: Vdelivermail, Maildir support and NFS 'safety' Hi - Thats fair enough! Okay, so the next question is - does this matter at all for vpopmail? If it does I can put a patch together that does this and submit it to you if you want. If it doesnt then I can put a patch up on a website if anyone wants it which I'll try to keep up to date with vpopmail stable versions (which I tend to follow with our production box). Obviously the first solution is better since it'll get more testing in the development versions. Marcus -Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED]] Sent: 05 June 2001 14:04 To: [EMAIL PROTECTED] Subject: Re: Vdelivermail, Maildir support and NFS 'safety' [snip] Is there any reason (other than optimisation) that these calls have been removed (or in the case of the missing fsync not implemented)? Yes, there is another reason. I like to get about 8 hours of sleep a night :) Ken Jones
