On Tue, Aug 10, 2010 at 10:11:35AM +0100, Chris G wrote: > There is a sort of logic to what python does, one *might* be deleting > or adding a message into the middle of the mbox
Unless you're expunging messages, or changing their status, it does not make sense to do that. In other words, if you're only delivering a message, the only sensible behavior is to lock the spool and append the message. I don't think there's any compelling reason to insert a message into the middle of the spool... Mail clients need to deal with sorting messages anyway. > in which case the copy, rename sequence is the only sensible way to > do it. Even in the case of expunging messages, it actually isn't... There's another way that's sensible, and better-performing, but also more complicated to implement and potentially devastatingly destructive if you don't do it right: expunge in place. The idea is to: 1. Copy, in chunks, data from undeleted messages which are later in sequence over messages which are deleted 2. Continue, copying later messages over moved/deleted chunks until all the undeleted message data is sequential 3. Truncate the file. The risk here is that if you have a power outage or similar event while your mail program is writing the file, it guarantees corruption of the mail spool. My argument is that this is still sensible to implement, because: - With typical usage patterns, usually earlier messages in an inbox are old, and the user is keeping them around for some reason. - Messages at the end of the file are new, and require action. Most typically, these are the ones that get deleted. - On modern hardware, with MMIO, performing the operations above will be extremely fast for the common case described above - As a result, the risk of expunging the spool file during a power outage is extremely low... And filesystem corruption is somewhat likely to occur, regardless of implementation, under those conditions. - On a well-run busy mail server, the system should be tied to a UPS which shuts down the mail system when an imminent power failure is detected (i.e. either immediately, or when the battery is getting low). The performance benefits make this worth considering. There's more risk than with the traditional method, but there's always *some* risk of filesystem corruption under any sort of failure mode, and with this method there's still really not very much risk, especially if you're doing all this on a well-run mail server. Were I to implement such a thing, I'd also implement the traditional method side-by-side. I'd provide an API call to allow selecting one of them explicitly, and also make the algorithm selectable by environment variable so that users could override the default behavior of applications using the library. -- Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience.
pgpSw1Qy1Qlxz.pgp
Description: PGP signature
