Re: Mailbox corrupt by disk quota
On Thu, 6 Mar 2003, Chris Miller wrote: > > The bottom line is that I need a pop server that plays nicely with quotas, > corrupted mailboxes cause too many support issues. Customers get upset > even though they created their own problem by turning on options they > don't understand (leave mail on server). It would be nice to see a built > in quota feature, or a locking option in a future release of qpopper that > would solve this problem. > > Writing scripts to police mailboxes might be a nice warning feature for > our customers (something I considered) but it's not a solution for corrupt > mailboxes. If qpopper can't write the file out to a filesystem, it should > leave the file behind in the temp drop. Corrupting files is not what I > would consider "failing gracefully". > you could always use Maildir/ maildrop format, eliminating the need for server mode/.pop files. there might be other factors involved for your setup that void this (your LDA won't support it, etc..), but it's an option, in general, nonetheless. --Tony .-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-. Anthony J. BiaccoNetwork Administrator/Engineer [EMAIL PROTECTED] http://www.asteroid-b612.org "This will prove a brave kingdom to me, where I shall have my music for nothing" .-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-. > > On Thu, 6 Mar 2003, Chuck Yerkes wrote: > > > System quotae were intended to keep users from storing too > > much on the machines in their HOME DIRECTORIES. > > > > That was the intent of quota systems. > > > > So we can use it as a hack to limit mailboxes size. But recall > > that it's a hack, so we have to work around some of the quota intent > > of offering a hard ceiling. Users don't duplicate their home > > directories a lot. > > > > The Right Answer is not to (mis)use the system quotae, but rather, > > put the checking in the delivery agent and let it use the soft > > quota as an advisory - you could get the info from LDAP if you > > wanted. But it's work on your part, at this moment. > > > > Quoting Alan Brown ([EMAIL PROTECTED]): > > > On Thu, 6 Mar 2003, Chuck Yerkes wrote: > > > > > > > However, using the disk system to enforce mail quota's is inherently > > > > a hack, given that there will be, for a moment, two spools. > > > > > > The only way around system quotas is to have the files in 2 different > > > partitions, but that is a _huge_ performance hit. > > > > > > Server mode makes user.pop handling a lot safer, but you need to ensure > > > that there is no direct access to the spool (eg, pine or mail) (Pine can > > > be configured to use pop in /etc/pine.conf or /etc/pine.conf.fixed), or > > > the direct access program. > > > > > > As Chuck says, pop is not designed for a lot of this high-end stuff. > > > > > > AB > > > > > > > -- .-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-. Anthony J. BiaccoNetwork Administrator/Engineer [EMAIL PROTECTED] http://www.asteroid-b612.org "This will prove a brave kingdom to me, where I shall have my music for nothing" .-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-._.-.
Re: Mailbox corrupt by disk quota
Or implement a quota system for mail using the only two bits that TOUCH mail, your LDA (procmail/mail.local) and QPopper. File system quotae were developed for home directories. Quoting Michael Kolos ([EMAIL PROTECTED]): > At 12:22 PM 3/6/2003, you wrote: > >On Thu, 6 Mar 2003, Chuck Yerkes wrote: > > > >> However, using the disk system to enforce mail quota's is inherently > >> a hack, given that there will be, for a moment, two spools. > > > >The only way around system quotas is to have the files in 2 different > >partitions, but that is a _huge_ performance hit. > > > > As bad as the hit is, it's not nearly as bad (or unmanageable) as user > calling every day because they can't get their mail. > With the relatively low-cost of disk space, it may be best to simply give > users an unlimited quota, and run scripts to erase any boxes not checked in > an arbitrarily long amount of time. > Of course this also opens up DoS possibilities of someone's box getting > flooded with mail. > > We run on two different file systems to avoid so many quota issues, and it > is not that bad of a performance hit. > > It seems that there really is no absolute solution with the current > software. Either a DoS opportunity is opened up or users are stuck, or > mail is corrupted. > > We run with the temp dir on a non-quota filesystem, and hard quota only > 100k larger than soft quota on the spool partition and with about 10,000 > users, there are no load problems from qpopper and mailbox corruption as > described in this thread only occurs about once every few months for anyone.
Re: Mailbox corrupt by disk quota
At 12:22 PM 3/6/2003, you wrote: On Thu, 6 Mar 2003, Chuck Yerkes wrote: > However, using the disk system to enforce mail quota's is inherently > a hack, given that there will be, for a moment, two spools. The only way around system quotas is to have the files in 2 different partitions, but that is a _huge_ performance hit. As bad as the hit is, it's not nearly as bad (or unmanageable) as user calling every day because they can't get their mail. With the relatively low-cost of disk space, it may be best to simply give users an unlimited quota, and run scripts to erase any boxes not checked in an arbitrarily long amount of time. Of course this also opens up DoS possibilities of someone's box getting flooded with mail. We run on two different file systems to avoid so many quota issues, and it is not that bad of a performance hit. It seems that there really is no absolute solution with the current software. Either a DoS opportunity is opened up or users are stuck, or mail is corrupted. We run with the temp dir on a non-quota filesystem, and hard quota only 100k larger than soft quota on the spool partition and with about 10,000 users, there are no load problems from qpopper and mailbox corruption as described in this thread only occurs about once every few months for anyone. Michael Kolos Colba.Net Inc.
Re: Mailbox corrupt by disk quota
Chuck, I want to start off by thanking you for attempting to assist me, but I'm afraid I'm going to have to disagree with you on a few points here. Quota systems may not have been intended for mail just like mail systems were never intended for spammers, but things evolve over time and requirements change. I don't know of any ISPs that don't use a quota system to enforce email limits, it's now a necessity. There are other popper programs that have quota systems built in just for this reason. In qpopper 3.x if the write to /var/mail failed, the mail simply got left in the temp directory. Still a problem but it didn't lead to corrupted mailboxes which require a sysadmin to fix instead of a first tier support rep. It seems to me that qpopper should be able to play nicely in the types of environments it will run in. Obviously there is no way for qpopper to know what the quota is if any until it hits that quota. This could easily be accomplished by copying the .user.pop file back to the spool directory under a different filename, then once successful it could rename the file. The solution that occured to me is that qpopper could keep the user's mailspool locked during the entire process so that no new mail could be delivered while the user is downloading mail. Although I haven't read the rfc from front to back, I did get some indication that this was considered legal. The bottom line is that I need a pop server that plays nicely with quotas, corrupted mailboxes cause too many support issues. Customers get upset even though they created their own problem by turning on options they don't understand (leave mail on server). It would be nice to see a built in quota feature, or a locking option in a future release of qpopper that would solve this problem. Writing scripts to police mailboxes might be a nice warning feature for our customers (something I considered) but it's not a solution for corrupt mailboxes. If qpopper can't write the file out to a filesystem, it should leave the file behind in the temp drop. Corrupting files is not what I would consider "failing gracefully". Thanks again for your help. Regards, Chris On Thu, 6 Mar 2003, Chuck Yerkes wrote: > System quotae were intended to keep users from storing too > much on the machines in their HOME DIRECTORIES. > > That was the intent of quota systems. > > So we can use it as a hack to limit mailboxes size. But recall > that it's a hack, so we have to work around some of the quota intent > of offering a hard ceiling. Users don't duplicate their home > directories a lot. > > The Right Answer is not to (mis)use the system quotae, but rather, > put the checking in the delivery agent and let it use the soft > quota as an advisory - you could get the info from LDAP if you > wanted. But it's work on your part, at this moment. > > Quoting Alan Brown ([EMAIL PROTECTED]): > > On Thu, 6 Mar 2003, Chuck Yerkes wrote: > > > > > However, using the disk system to enforce mail quota's is inherently > > > a hack, given that there will be, for a moment, two spools. > > > > The only way around system quotas is to have the files in 2 different > > partitions, but that is a _huge_ performance hit. > > > > Server mode makes user.pop handling a lot safer, but you need to ensure > > that there is no direct access to the spool (eg, pine or mail) (Pine can > > be configured to use pop in /etc/pine.conf or /etc/pine.conf.fixed), or > > the direct access program. > > > > As Chuck says, pop is not designed for a lot of this high-end stuff. > > > > AB > > >
Re: Mailbox corrupt by disk quota
System quotae were intended to keep users from storing too much on the machines in their HOME DIRECTORIES. That was the intent of quota systems. So we can use it as a hack to limit mailboxes size. But recall that it's a hack, so we have to work around some of the quota intent of offering a hard ceiling. Users don't duplicate their home directories a lot. The Right Answer is not to (mis)use the system quotae, but rather, put the checking in the delivery agent and let it use the soft quota as an advisory - you could get the info from LDAP if you wanted. But it's work on your part, at this moment. Quoting Alan Brown ([EMAIL PROTECTED]): > On Thu, 6 Mar 2003, Chuck Yerkes wrote: > > > However, using the disk system to enforce mail quota's is inherently > > a hack, given that there will be, for a moment, two spools. > > The only way around system quotas is to have the files in 2 different > partitions, but that is a _huge_ performance hit. > > Server mode makes user.pop handling a lot safer, but you need to ensure > that there is no direct access to the spool (eg, pine or mail) (Pine can > be configured to use pop in /etc/pine.conf or /etc/pine.conf.fixed), or > the direct access program. > > As Chuck says, pop is not designed for a lot of this high-end stuff. > > AB >
Re: Mailbox corrupt by disk quota
On Thu, 6 Mar 2003, Chuck Yerkes wrote: > However, using the disk system to enforce mail quota's is inherently > a hack, given that there will be, for a moment, two spools. The only way around system quotas is to have the files in 2 different partitions, but that is a _huge_ performance hit. Server mode makes user.pop handling a lot safer, but you need to ensure that there is no direct access to the spool (eg, pine or mail) (Pine can be configured to use pop in /etc/pine.conf or /etc/pine.conf.fixed), or the direct access program. As Chuck says, pop is not designed for a lot of this high-end stuff. AB
Re: Mailbox corrupt by disk quota
However, using the disk system to enforce mail quota's is inherently a hack, given that there will be, for a moment, two spools. The more Right Answer is to have the LDA (procmail, mail.local, whatever) check the softquota and tempfail (75) if it's over and tempfail is $CurrentSize + $NewMessage would exceed hardquota. In a large IMAP server I use (file/message), there are settings for "allow one message over quota" to let in a message that happens to exceed quota. You also probably want a reporting method to alert users when they are over quota. POP, unfo, doesn't allow an "alert message" like IMAP. MOst GUI IMAP clients will pop-up a box with an alert. Alerts I've used include "Server will be down for maint from 9-11PM" (like a bulletin), but also jobs that run through, determine over quota and emit a "You are over quota and will not get new mail until you make some room." Can Eudora and friends handle something like this via POP? It wouldn't be too hard for qpopper to inject a (false) message, like a bulletin, PER USER based on factors like this. Heck, a per-user bulletin dir wouldn't be hard - deliver one message from outside the spool and then mark it read. An external process can create and delete those per-user alerts. The bulletin routines pretty much cover all the hooks for doing that. Quoting Chris Miller ([EMAIL PROTECTED]): > this sounds like a great idea on the surface, but this doesn't > prevent people from accumulating more than 10MB of mail, which will > ultimately cause this problem to happen at a higher increment of disk > space, right? > > On Wed, 5 Mar 2003, Chuck Yerkes wrote: > > Generally the hardquota should be AT LEAST 2x the softquota. > > This lets the box get copied and new messages to arrive. > > > > In these situations, I try to be sure that over-quota errors are > > TEMP FAIL e.g. 4xx. Quotae get fixed and you can keep it in queue > > for a little while. Nothing like bouncing someones' mail by sending > > them some large messages - great DOS. > > > > Quoting Matt Garretson ([EMAIL PROTECTED]): > > > Chris Miller wrote: > > > > what OS are you running? > > > > > > > > > AIX 4.3.3 + qpopper 4.0.4 + procmail 3.22 > > > > > > Thanks for sharing your recovery method. On my system, some > > > of the messages are in the temporary pop drop, and some are > > > still in the corrupted spool file. I do something like this: > > > > > > lockfile /var/spool/mail/user.lock > > > mv /var/spool/mail/user /tmp > > > mail -f /tmp/user (then just quit out of mail with "q") > > > cat /var/tmp/pop/.user.pop /tmp/user > /var/spool/mail/user > > > (at this point i either delete some old messages from the > > > user's spool file, or increase the user's quota) > > > chown user.mail /var/spool/mail/user > > > rm -f /var/spool/mail/user.lock > > > > > > Loading the spool file into "mail" seems to skip over the corrupted > > > beginning of the file, so when i quit out it gets saved without > > > the junk. Maybe next time i will try using tail like you do. > > > > > > -Matt > >
Re: Mailbox corrupt by disk quota
Chuck, this sounds like a great idea on the surface, but this doesn't prevent people from accumulating more than 10MB of mail, which will ultimately cause this problem to happen at a higher increment of disk space, right? Regards, Chris On Wed, 5 Mar 2003, Chuck Yerkes wrote: > Generally the hardquota should be AT LEAST 2x the softquota. > This lets the box get copied and new messages to arrive. > > In these situations, I try to be sure that over-quota errors are > TEMP FAIL e.g. 4xx. Quotae get fixed and you can keep it in queue > for a little while. Nothing like bouncing someones' mail by sending > them some large messages - great DOS. > > Quoting Matt Garretson ([EMAIL PROTECTED]): > > Chris Miller wrote: > > > what OS are you running? > > > > > > AIX 4.3.3 + qpopper 4.0.4 + procmail 3.22 > > > > Thanks for sharing your recovery method. On my system, some > > of the messages are in the temporary pop drop, and some are > > still in the corrupted spool file. I do something like this: > > > > lockfile /var/spool/mail/user.lock > > mv /var/spool/mail/user /tmp > > mail -f /tmp/user (then just quit out of mail with "q") > > cat /var/tmp/pop/.user.pop /tmp/user > /var/spool/mail/user > > (at this point i either delete some old messages from the > > user's spool file, or increase the user's quota) > > chown user.mail /var/spool/mail/user > > rm -f /var/spool/mail/user.lock > > > > Loading the spool file into "mail" seems to skip over the corrupted > > beginning of the file, so when i quit out it gets saved without > > the junk. Maybe next time i will try using tail like you do. > > > > -Matt >
Re: Mailbox corrupt by disk quota
On Wed, 5 Mar 2003, Chuck Yerkes wrote: > Generally the hardquota should be AT LEAST 2x the softquota. >... Nothing like bouncing someones' mail by sending > them some large messages - great DOS. And on that note, the maximum allowed message size shouldn't exceed 25% of the hard quota (ie: half the soft quota) AB
Re: Mailbox corrupt by disk quota
Yeah the one thing I forgot to mention is that I run pine -f /usr/local/var/mailtmp/.user.pop and remove all the messages marked as "read". Regards, Chris Chris Miller NetGate Internet On Wed, 5 Mar 2003, Matt Garretson wrote: > Chris Miller wrote: > > what OS are you running? > > > AIX 4.3.3 + qpopper 4.0.4 + procmail 3.22 > > Thanks for sharing your recovery method. On my system, some > of the messages are in the temporary pop drop, and some are > still in the corrupted spool file. I do something like this: > >lockfile /var/spool/mail/user.lock >mv /var/spool/mail/user /tmp >mail -f /tmp/user (then just quit out of mail with "q") >cat /var/tmp/pop/.user.pop /tmp/user > /var/spool/mail/user > (at this point i either delete some old messages from the > user's spool file, or increase the user's quota) >chown user.mail /var/spool/mail/user >rm -f /var/spool/mail/user.lock > > Loading the spool file into "mail" seems to skip over the corrupted > beginning of the file, so when i quit out it gets saved without > the junk. Maybe next time i will try using tail like you do. > > -Matt >
Re: Mailbox corrupt by disk quota
Generally the hardquota should be AT LEAST 2x the softquota. This lets the box get copied and new messages to arrive. In these situations, I try to be sure that over-quota errors are TEMP FAIL e.g. 4xx. Quotae get fixed and you can keep it in queue for a little while. Nothing like bouncing someones' mail by sending them some large messages - great DOS. Quoting Matt Garretson ([EMAIL PROTECTED]): > Chris Miller wrote: > > what OS are you running? > > > AIX 4.3.3 + qpopper 4.0.4 + procmail 3.22 > > Thanks for sharing your recovery method. On my system, some > of the messages are in the temporary pop drop, and some are > still in the corrupted spool file. I do something like this: > > lockfile /var/spool/mail/user.lock > mv /var/spool/mail/user /tmp > mail -f /tmp/user (then just quit out of mail with "q") > cat /var/tmp/pop/.user.pop /tmp/user > /var/spool/mail/user > (at this point i either delete some old messages from the > user's spool file, or increase the user's quota) > chown user.mail /var/spool/mail/user > rm -f /var/spool/mail/user.lock > > Loading the spool file into "mail" seems to skip over the corrupted > beginning of the file, so when i quit out it gets saved without > the junk. Maybe next time i will try using tail like you do. > > -Matt
Re: Mailbox corrupt by disk quota
Chris Miller wrote: what OS are you running? AIX 4.3.3 + qpopper 4.0.4 + procmail 3.22 Thanks for sharing your recovery method. On my system, some of the messages are in the temporary pop drop, and some are still in the corrupted spool file. I do something like this: lockfile /var/spool/mail/user.lock mv /var/spool/mail/user /tmp mail -f /tmp/user (then just quit out of mail with "q") cat /var/tmp/pop/.user.pop /tmp/user > /var/spool/mail/user (at this point i either delete some old messages from the user's spool file, or increase the user's quota) chown user.mail /var/spool/mail/user rm -f /var/spool/mail/user.lock Loading the spool file into "mail" seems to skip over the corrupted beginning of the file, so when i quit out it gets saved without the junk. Maybe next time i will try using tail like you do. -Matt
Re: Mailbox corrupt by disk quota
Matt, what OS are you running? Yes, I have a process too, and this one doesn't lose mail :-). head -1 /var/mail/user | strings > /var/mail/user.tmp wc -l /var/mail/user (subtract 1) tail -(above value) >> /var/mail/user.tmp cp /var/mail/user.tmp /var/mail/user One thing that is strange is that the quota command does not show the size of the corrupt mailbox, but what the good mailbox size was. This puzzles me because qpopper is not suid so the user really was writing the data when the failure occured. Anyone have any suggestions on fixing this? Glad it's not just me. Regards, Chris On Wed, 5 Mar 2003, Matt Garretson wrote: > Chris Miller wrote: > > I've been having problems with qpopper corrupting mailboxes when a > > user is at their disk quota. Here's the background; > > > Yes, this happens to us on a weekly basis, too. I find that > the mailbox can be pretty much reconstructed using a combination > of the temp pop drop and part of the corrupted spool file. If > you want, i can supply the exact steps i use to do this. There's > a chance that a message or two might get lost in this process, but > i don't sweat it too much anymore. Though it would be nice if > the problem never came up in the first place. > > -Matt >
Re: Mailbox corrupt by disk quota
Chris Miller wrote: I've been having problems with qpopper corrupting mailboxes when a user is at their disk quota. Here's the background; Yes, this happens to us on a weekly basis, too. I find that the mailbox can be pretty much reconstructed using a combination of the temp pop drop and part of the corrupted spool file. If you want, i can supply the exact steps i use to do this. There's a chance that a message or two might get lost in this process, but i don't sweat it too much anymore. Though it would be nice if the problem never came up in the first place. -Matt
