Re: Quotas and disk usage

[Michael Menge]

Quoting Eric Luyten <eric.luy...@vub.ac.be>:

On Thu, January 14, 2010 8:26 pm, Wil Cooley wrote:

Adam Tauno Williams wrote:

Some earlier versions of Cyrus had quota calculation issues with big
quotas (2GB? 4GB?) or big mailboxes.  I think there is a blurb in WMOGAG
about that, and what version fixed it.

It happens with 2.2 crossing the 4GiB range, but not related to what he's
seeing.

bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user1 Quota   % Used     Used
Root
5242880       63  3303494 user.user1
bash-3.00$ /opt/cyrus-imap/bin/quota -f user.user2 Quota   % Used     Used
Root
41943040       93 39410366 user.user2
bash-3.00$ du -sh user1 7.6G   user1
bash-3.00$ du -sh user2 6.4G   user2

Do you have delayed expunge enabled?  If so actual disk size may very

are still on the disk.

well exceed quota values - because the expunged-but-not-yet-expired messages

Other possible contributors:
o Squatter indexes (my estimation is that they add 50%) o Directory sizes
themselves o Other cyrus.* metadata

You can use something like this to calculate just the total (sorry, you need
GNU
du, find and xargs):

find `mbpath user.user1` -type f -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot "MiB" }'


You need the awk if there are more files than possible for a single
invocation of "du". And you can calculate the metadata overhead by negating
the "-name" parameter:


find `mbpath user.user1` -type f \! -name \*. -print0 \ |xargs -0 du -cm|awk
'/total/ { tot=tot+$1 } END { print tot "MiB" }'

On our previous Cyrus server (2.2 on Solaris 9 with UFS) I detected several
multiply-linked message files (sitting in different folders/directories).

Those will be counted twice/thrice/... towards the Cyrus quota but not 'du'.
We were unable to find out how some IMAP clients were capable of having the
server create them.
(This being said, we never invested much effort into it :-)

   % find `mbpath user/NNNNNN` -type f \! -links 1

It's not the client but the Server doing this. See man imapd.conf

singleinstancestore: 1
            If  enabled, imapd, lmtpd and nntpd attempt to only write one copy
            of a message per partition and create hard links, resulting  in  a
            potentially large disk savings.

enabled) there is a ("zfs get compressratio") difference of 27 to 28 %


Eric Luyten, Computing Centre VUB/ULB, postmaster.

----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

On our new server (2.3 on Solaris 10 with ZFS and basic filesystem compression

--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912

Wächterstraße 76
72074 Tübingen

Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de

smime.p7s
Description: S/MIME Signatur

----
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html