Re: locking rcvstore?
I think anytime that there is a possibility of multiple processes writing to the same file, locking should be enableable. Even if it's a command line switch (everyone likes to say use procmail, but this is essentially what procmail does: locks file and folders by default, Not when you're using rcvstore, because then it doesn't know what the folder's called, even whether there is a folder (you could be pushing it into a program that redirects it as formail can or that processes the message fully in some other way). File locking is essential. If semantics mean it can't be done on some platforms, tought for those users, do it where you can. Don't forget to test on nfs-mounted filesystems too; I've had a few programs fail on nfs-mounts - StarOffice and ypbind amongst them. -- Cheers John Summerfield Microsoft's most solid OS: http://www.geocities.com/rcwoolley/ Note: mail delivered to me is deemed to be intended for me, for my disposition. == If you don't like being told you're wrong, be right!
Re: locking rcvstore?
Michael Richardson [EMAIL PROTECTED] wrote: Aside from trashing sequences (which I've experienced on occasion, no idea why) I've run into situations where I wind up doing an inc from two difference sources into the same folder. Usually due to impatience on my part. As far as I know, inc does to file locking, at least when writing to the standard unix mailbox. The result was a mess of two processes using the same message numbers! That should not be possible. I haven't looked at the code. But it should be opening the file with O_CREAT, which should fail if the message already exists. Unless there are major code deficiencies, creating new messages should depend on the atomicity of unix file creation. No additional locking should be required. That is supposed to be one of the benefits of one message per file. -NWR
Re: locking rcvstore?
On July 12, 2002 at 10:18, Neil W Rickert wrote: That should not be possible. I haven't looked at the code. But it should be opening the file with O_CREAT, which should fail if the message already exists. I think you mean O_CREAT|O_EXCL. Unless there are major code deficiencies, creating new messages should depend on the atomicity of unix file creation. No additional locking should be required. That is supposed to be one of the benefits of one message per file. So how is message number (filename) determined? If a open() fails to pre-existance, does inc re-read the directory to determine the next message number, or does it just keep incrementing the number to something works? It seems that the only real place that locking is needed is for the .mh_sequences file. I.e. The use of locking can be limited to areas where there is no non-locking method available to prevent corruption. --ewh P.S. Somebodies formatting of the References: field is messed up. Only message-ids should be present.
Re: locking rcvstore?
From: Earl Hood [EMAIL PROTECTED] Date: Fri, 12 Jul 2002 14:59:12 -0500 It seems that the only real place that locking is needed is for the .mh_sequences file. I.e. The use of locking can be limited to areas where there is no non-locking method available to prevent corruption. Speaking of non-locking methods to prevent corruption, one of the other things I'd like to see done (speaking again as someone who's not going to be implementing it), is to find a way to support Maildirs as well as the current MH directories. (http://www.qmail.org/man/man5/maildir.html) Preferably these would be extended compatibly with courier-imap's Maildir++ so the same hierarchy of messages could be seen in both places. (http://www.inter7.com/courierimap/README.maildirquota.html) Note that courier-imap uses . a the separator character. I don't know why he didn't bother to specify that. This might be a useful first step before an imap backend since it would be somewhat easier. Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 716 Congress, Suite 200 Austin, TX 78701 +1 512 374 0500 World War III: The Wrong-Doers Vs. the Evil-Doers. msg00969/pgp0.pgp Description: PGP signature
Re: locking rcvstore?
Chris Garrigues [EMAIL PROTECTED] wrote: I always call rcvstore with this wrapper: #!/bin/bash ... and it would be nice if it weren't needed. I'm not convinced that it is needed. The only risk is that your sequences might be corrupted. I prefer that to the problems caused by stuck locks. I had some serious mail corruption a year or two ago. It arose because of an enormous mess of coincidence: 1) I managed to get two copies of fetchmail running at the same time on the same mail boxes (fetchmail bug) 2) The POP3 daemon (cucipop) at the other end didn't lock mail boxes (non-compliant) 3) My procmail recipes didn't implement locking because I assumed nmh did. 4) nmh didn't lock anything, (and I think mh doesn't either). Lack of locking is a problem that needs to be fixed. I lost a few dozen emails over a couple of days. -- Cheers John Summerfield Microsoft's most solid OS: http://www.geocities.com/rcwoolley/ Note: mail delivered to me is deemed to be intended for me, for my disposition. == If you don't like being told you're wrong, be right!
Re: locking rcvstore?
Aside from trashing sequences (which I've experienced on occasion, no idea why) I've run into situations where I wind up doing an inc from two difference sources into the same folder. Usually due to impatience on my part. The result was a mess of two processes using the same message numbers! My impression is that inc doesn't do any locking on message numbers, but I could be wrong. Btw, my installed from CVS copy of nmh seems to be ignoring my .mh_profile's Alternate-Mailboxes: [EMAIL PROTECTED], mcr@*, [EMAIL PROTECTED],[EMAIL PROTECTED],[EMAIL PROTECTED],[EMAIL PROTECTED] when removing stuff for reply-all. Anyone else have this problem? Is mcr@* still a valid? ] ON HUMILITY: to err is human. To moo, bovine. | firewalls [ ] Michael Richardson, Sandelman Software Works, Ottawa, ON|net architect[ ] [EMAIL PROTECTED] http://www.sandelman.ottawa.on.ca/ |device driver[ ] panic(Just another NetBSD/notebook using, kernel hacking, security guy); [
Re: locking rcvstore?
[In a message on Thu, 11 Jul 2002 14:29:20 EDT, the pithy ruminations of Michael Richardson were:] Aside from trashing sequences (which I've experienced on occasion, no idea why) I've run into situations where I wind up doing an inc from two difference sources into the same folder. Usually due to impatience on my part. The result was a mess of two processes using the same message numbers! My impression is that inc doesn't do any locking on message numbers, but I could be wrong. I think anytime that there is a possibility of multiple processes writing to the same file, locking should be enableable. Even if it's a command line switch (everyone likes to say use procmail, but this is essentially what procmail does: locks file and folders by default, and you can disable or use a different file for locking if you choose). Over the years, we've had a LOT of email being trashed because of lock problems. We end up doing arcane workarounds to fix it. You can have timeout's on lockfiles, and fail. You can have command line/compile time options to disable/change locking. But the sequence file and the next message should be locked when written. Oh, and the mail drop should be locked when it's being inc'ed. Using locking in the shell is only slightly better than no locking at all. . . Sean
Re: locking rcvstore?
From: Randy Primeaux [EMAIL PROTECTED] Date: Tue, 09 Jul 2002 22:19:00 -0700 Here's a procmail recipe I use, including a procmail lockfile before rcvstore is called. PATH is on Debian Linux. PATH=/bin:/usr/bin:/usr/bin/mh:/usr/lib/mh:/etc/nmh:/usr/local/bin LOCKEXT=.lock :0 w: nmh-workers/$LOCKEXT * ^TO_.*nmh-workers | rcvstore +in-nmh-workers Would that be a possible solution for you? No. I wrote the wrapper because I don't use procmail. I had this being called from qmail and now from tmda. Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 716 Congress, Suite 200 Austin, TX 78701 +1 512 374 0500 World War III: The Wrong-Doers Vs. the Evil-Doers. msg00959/pgp0.pgp Description: PGP signature
Re: locking rcvstore?
Chris Garrigues [EMAIL PROTECTED] wrote: I always call rcvstore with this wrapper: #!/bin/bash ... and it would be nice if it weren't needed. I'm not convinced that it is needed. The only risk is that your sequences might be corrupted. I prefer that to the problems caused by stuck locks. -NWR
Re: locking rcvstore?
From: Neil W Rickert [EMAIL PROTECTED] Date: Wed, 10 Jul 2002 10:16:03 -0500 Chris Garrigues [EMAIL PROTECTED] wrote: I always call rcvstore with this wrapper: #!/bin/bash ... and it would be nice if it weren't needed. I'm not convinced that it is needed. The only risk is that your sequences might be corrupted. I prefer that to the problems caused by stuck locks. *might*? I wrapped it because I get enough mail in some mailboxes that it was guaranteed that I would get corrupted sequences on a daily basis. I've never had a problem with stuck locks since I started using that wrapper several years ago. I'm curious why some people have such a strong aversion to locking mailboxes? Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 716 Congress, Suite 200 Austin, TX 78701 +1 512 374 0500 World War III: The Wrong-Doers Vs. the Evil-Doers. msg00961/pgp0.pgp Description: PGP signature
locking rcvstore?
I'm just a lurker here, but I'd like to say that I'm glad to see some work being done to nmh once again. I lurk because my C skills are poor, but that doesn't mean I can't put in a request for those who have better C skills. I always call rcvstore with this wrapper: #!/bin/bash PATH=/usr/lib/nmh:$PATH export PATH lockfile $HOME/Mail/$1/.lock echo $ rcvstore +$ rm -f $HOME/Mail/$1/.lock and it would be nice if it weren't needed. Is anyone interested in adding file locking to rcvstore. The lockfile executable that I call comes with procmail and I suppose it wouldn't be unreasonable to do the locking in a way that was compatible with procmail. procmail is under the GPL artistic license, so I guess you can't just borrow the code due to license pollution, but how hard could lockfile be to implement? Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 716 Congress, Suite 200 Austin, TX 78701 +1 512 374 0500 World War III: The Wrong-Doers Vs. the Evil-Doers. msg00957/pgp0.pgp Description: PGP signature
