[vchkpw] Re: Quota problem: negative values in Maildir/maildirsize
Hi, I have just confirmed another user in another vpopmail domain with the same maildirsize problem. All mail domains reside on NFS server. The user's Maildir size is around 5 megs, excluding .Trash, as reported by `du` The user's quota is set to '1000' i.e. 10MB. vuserinfo -Q [EMAIL PROTECTED] reports _ 0% _ Courier-imap returns a negative usage value when asked with the getquota ROOT command, for the two users having the problem. The problem started happening again about a week or more after I patched qmail- local with maildir++ patch. I am not using qmail-pop3d. I am using courier-2.0.0 imapd and pop3d, and vpopmail 5.2.1 I believe this problem is a race condition because: 1. The problem happens for a few users, who are also heavy users 2. The problem doesn't occur very often, but also very random. It occured 4 days after I setup the system when using unpatched qmail-local with vpopmail, courier imapd and pop3d. After removing/recreating bad maildirsize files and patching qmail-local, the problem occurs after a week or so. Another intersting link which I think is strongly related: http://www.ultraviolet.org/mail-archives/ldap.2000/1471.html The Important part: I am using qmail-ldap which implements the same quota system as Courier/Sqwebmail, and reportedly, there is a race condition when qmail-pop3d and qmail-local are accessing the maildirsize file at the same time. I guess there's no locking on the file, so it is filled with negative numbers and users' e-mail bounces. I dont think email is bouncing at all though. vuserinfo -Q reports 0% usage, and courier reports negative values, so I believe users could probably get more mail delivered than there quota allows, during the time the maildirsize is corrupted. The maildirsize is not being regenerated automagically, as I would expect. So I manually have to remove those bad maildirsize files, and rerun vuserinfo -Q on the user account to have it recreated. Mr Sam recommended I should try using maildrop as the only delivery agent to see if that fixes the problem. And because the problem is happening on a production system, I will use that as a last option. Any suggestions, ideas, comments? -Tim
[vchkpw] Re: Quota problem: negative values in Maildir/maildirsize
Dear MrSam, Thanks for all your help, and my apologies if this turns out not to be a courier related problem. I patched qmail-local with maildir++ patch from billshupp.org. Then I ran my little shell script which removed all maildirsize files with negated values in them. Everything worked fine since then, until one of our users called today, complaining her quota is no longer reporting valid values (In our case the webmail reports You have unlimited quota or some totally random results. the maildirsize in that particular user's Maildir looks like this: 1000S 4456505 473 -2429946 -22 42431 173151 1 169837 1 1439 1 17711 -13912 -1 -46605 -1 -4832 -1 48321 -1221 -1 1754591 -175459 -1 -15295 -1 -3574 -1 -8113 -1 35871 -3587 -1 29301 -6633 -1 -13075 -1 -1627 -1 12061 12661 292301 16511 295951 3743 1 2737 1 27371 -2737 -1 -3743 -1 -3351 -2 -570268 -3 13331 52949 1 -1893017 -10 -173151 -1 -169837 -1 23971 -2397 -1 -921818 -1 -52949 -1 3553161 -355316 -1 185381 -18538 -1 24711 -2471 -1 44559 1 -45631 -3 445591 -44559 -1 -29714 -1 14491 Simulating a login to the courier-imap server: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost.aidasystems.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE STARTTLS] Courier-IMAP ready. Copyright 1998- 2003 Double Precision, Inc. See COPYING for distribution information. a login [EMAIL PROTECTED] somepass a getquota ROOT * QUOTA ROOT (STORAGE -1389 9766) a OK GETQUOTA Ok. a logout * BYE Courier-IMAP server shutting down a OK LOGOUT completed Running /home/vpopmail/bin/vuserinfo -Q [EMAIL PROTECTED] (the same user who's having the problem) reports: quota: 1000 usage: 0% last auth: Tue Sep 16 15:22:40 2003 last auth ip: imap Although dishusage shows otherwise: # pwd /home/vpopmail/domains/somedomain.com/myuser/Maildir # ls courierimapsubscribed courierimapuiddb cur maildirsize new tmp # du -sh . 8.1M. # du -s . 8284. I am running courier-imap 2.0.0 (imapd and pop3d), Vpopmail 5.3.2-Stable, Qmail-1.03 (qmail-local patched with Bill Shupp's qmaildir++ patch) Although I am also using maildrop (compiled with quota support) for filtering, it's not in use for that particular user's domain at all. Maildrop is only used on one domain, and invoked from .qmail-default directly for only that domain. This problem only started happening after migration to the new cluster which is composed of 2 real servers running the above mentioned software, and two failover NFS server (linux unfortunately) which hold the entire /home/vpopmail/domains which is mounted on the freebsd servers (the real servers) which are running the pop/imap toasters. The only difference between the old mail server and the new real servers is we were running courier 1.6.2 or 1.6.4 and now we are running 2.0.0. Although I am not sure if that's relevant. Please advise... Respectfully, Tim Hasson Quoting Sam Varshavchik [EMAIL PROTECTED]: Tim Hasson writes: Also, nothing other than vdelivermail, maildrop, and qmail-local delivers to those Maildirs. I am 100% sure of that =) Mail delivered by qmail-local is not going to be added to the current quota. Subsequently, when those messages are removed, their sizes will be subtracted from the quota. And that's how you get negative quotas.
Re: [vchkpw] Re: Quota problem: negative values in Maildir/maildirsize
Hi all, As mentioned earlier in my message (see copy at end of this message), all my mail resides on NFS. Could this be related to the maildirsize files getting negatives and (-)ve Storage imap reply when getquota ROOT? or i.e. Is this the same as the maildirsize gets hosed problem? Please see the following: http://marc.theaimsgroup.com/?l=qmailm=106355226521299w=2 http://article.gmane.org/gmane.mail.vpopmail/7918 In regards to the first post/link: Does Maildir++ really violate the Maildir specification? How? Does it also make Maildir unsafe for use on NFS without locking? Why? Any advise, suggestions, and help is greatly appreciated. Respectfully, Tim Hasson Quoting Tim Hasson [EMAIL PROTECTED]: Dear MrSam, Thanks for all your help, and my apologies if this turns out not to be a courier related problem. I patched qmail-local with maildir++ patch from billshupp.org. Then I ran my little shell script which removed all maildirsize files with negated values in them. Everything worked fine since then, until one of our users called today, complaining her quota is no longer reporting valid values (In our case the webmail reports You have unlimited quota or some totally random results. the maildirsize in that particular user's Maildir looks like this: 1000S 4456505 473 -2429946 -22 42431 173151 1 169837 1 1439 1 17711 -13912 -1 -46605 -1 -4832 -1 48321 -1221 -1 1754591 -175459 -1 -15295 -1 -3574 -1 -8113 -1 35871 -3587 -1 29301 -6633 -1 -13075 -1 -1627 -1 12061 12661 292301 16511 295951 3743 1 2737 1 27371 -2737 -1 -3743 -1 -3351 -2 -570268 -3 13331 52949 1 -1893017 -10 -173151 -1 -169837 -1 23971 -2397 -1 -921818 -1 -52949 -1 3553161 -355316 -1 185381 -18538 -1 24711 -2471 -1 44559 1 -45631 -3 445591 -44559 -1 -29714 -1 14491 Simulating a login to the courier-imap server: # telnet localhost 143 Trying 127.0.0.1... Connected to localhost.aidasystems.com. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE STARTTLS] Courier-IMAP ready. Copyright 1998- 2003 Double Precision, Inc. See COPYING for distribution information. a login [EMAIL PROTECTED] somepass a getquota ROOT * QUOTA ROOT (STORAGE -1389 9766) a OK GETQUOTA Ok. a logout * BYE Courier-IMAP server shutting down a OK LOGOUT completed Running /home/vpopmail/bin/vuserinfo -Q [EMAIL PROTECTED] (the same user who's having the problem) reports: quota: 1000 usage: 0% last auth: Tue Sep 16 15:22:40 2003 last auth ip: imap Although dishusage shows otherwise: # pwd /home/vpopmail/domains/somedomain.com/myuser/Maildir # ls courierimapsubscribed courierimapuiddb cur maildirsize new tmp # du -sh . 8.1M. # du -s . 8284. I am running courier-imap 2.0.0 (imapd and pop3d), Vpopmail 5.3.2-Stable, Qmail-1.03 (qmail-local patched with Bill Shupp's qmaildir++ patch) Although I am also using maildrop (compiled with quota support) for filtering, it's not in use for that particular user's domain at all. Maildrop is only used on one domain, and invoked from .qmail-default directly for only that domain. This problem only started happening after migration to the new cluster which is composed of 2 real servers running the above mentioned software, and two failover NFS server (linux unfortunately) which hold the entire /home/vpopmail/domains which is mounted on the freebsd servers (the real servers) which are running the pop/imap toasters. The only difference between the old mail server and the new real servers is we were running courier 1.6.2 or 1.6.4 and now we are running 2.0.0. Although I am not sure if that's relevant. Please advise... Respectfully, Tim Hasson Quoting Sam Varshavchik [EMAIL PROTECTED]: Tim Hasson writes: Also, nothing other than vdelivermail, maildrop, and qmail-local delivers to those Maildirs. I am 100% sure of that =) Mail delivered by qmail-local is not going to be added to the current quota. Subsequently, when those messages are removed, their sizes will be subtracted from the quota. And that's how you get negative quotas.
