Re: Meaning of "older than"? [Was: Backing up mailboxes]
On 06/14/2011 12:58 PM, Patrick Boutilier wrote: On 06/14/2011 12:51 PM, Adam Tauno Williams wrote: On Thu, 2011-06-09 at 10:04 -0700, Andrew Morgan wrote: On Thu, 9 Jun 2011, Diego Ventrice wrote: I'm planning a backup strategy for a new server that will handle Hundreds of mailboxes. Largest server for me to admin so far Would you share your experiences backing up cyrus mailboxes ? What tool/s do you use ? how many backups a day do you run ? Enabling delayed delete and delayed expunge will help a lot: delete_mode: delayed expunge_mode: delayed You'll also want to modify your expire line in cyrus.conf to something like this: delprune cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100 -D and -X control how long deleted and expunged items remain on-disk: -D delete-days Remove previously deleted mailboxes older than delete-days (when using the "delayed" delete mode). "Older than" is a confusing statement. According to my notes this means a message is kept for a minimum of X days from delivery, not X number of days *after* it is deleted. I don't know why my notes say this, but I remember testing it and convincing myself it is true. So -X / -D may not do exactly what you are assuming they do. Am I wrong? (and are you certain I'm wrong?) Not sure if expire works that way but ipurge does for sure. If you have a message that has been sitting in your mailbox for one year and you delete it today than ipurge -d30 will purge the mail, I should explain that a bit further. I use a modified ipurge that will allow you to only purge deleted messages while leaving the others alone. In that case ipurge determines what messages to purge based on the message date not the date the message was deleted, which makes sense. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Meaning of "older than"? [Was: Backing up mailboxes]
On 06/14/2011 12:51 PM, Adam Tauno Williams wrote: On Thu, 2011-06-09 at 10:04 -0700, Andrew Morgan wrote: On Thu, 9 Jun 2011, Diego Ventrice wrote: I'm planning a backup strategy for a new server that will handle Hundreds of mailboxes. Largest server for me to admin so far Would you share your experiences backing up cyrus mailboxes ? What tool/s do you use ? how many backups a day do you run ? Enabling delayed delete and delayed expunge will help a lot: delete_mode: delayed expunge_mode: delayed You'll also want to modify your expire line in cyrus.conf to something like this: delprune cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100 -D and -X control how long deleted and expunged items remain on-disk: -D delete-days Remove previously deleted mailboxes older than delete-days (when using the "delayed" delete mode). "Older than" is a confusing statement. According to my notes this means a message is kept for a minimum of X days from delivery, not X number of days *after* it is deleted. I don't know why my notes say this, but I remember testing it and convincing myself it is true. So -X / -D may not do exactly what you are assuming they do. Am I wrong? (and are you certain I'm wrong?) Not sure if expire works that way but ipurge does for sure. If you have a message that has been sitting in your mailbox for one year and you delete it today than ipurge -d30 will purge the mail, Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ <> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
On 06/14/2011 12:47 PM, Michael D. Sofka wrote: Patrick Boutilier wrote: On 06/09/2011 01:48 AM, Diego Ventrice wrote: I'm planning a backup strategy for a new server that will handle Hundreds of mailboxes. Largest server for me to admin so far Would you share your experiences backing up cyrus mailboxes ? What tool/s do you use ? how many backups a day do you run ? We backup all the files on the server using TSM, incrementals nightly. Server using delayed delete and delayed expunge. Also use cyrus replication to a second server. And we backup mailboxes.db to a text file once a day. This is exactly what we do. Deleted and expunged messages are kept for up to 2 days, but I'll likely extend this to avoid trips to `the wayback machine.'' A surprising number of people don't notice they deleted messages, or don't report it for a few days. Forgot to mention that we keep deleted and expunged for 7 days. The more disk space you have the longer you can keep them around for. Mike <> Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Meaning of "older than"? [Was: Backing up mailboxes]
On Thu, 2011-06-09 at 10:04 -0700, Andrew Morgan wrote: > On Thu, 9 Jun 2011, Diego Ventrice wrote: > > I'm planning a backup strategy for a new server that will handle > > Hundreds of mailboxes. Largest server for me to admin so far > > Would you share your experiences backing up cyrus mailboxes ? > > What tool/s do you use ? how many backups a day do you run ? > Enabling delayed delete and delayed expunge will help a lot: > delete_mode: delayed > expunge_mode: delayed > You'll also want to modify your expire line in cyrus.conf to something > like this: >delprune cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100 > -D and -X control how long deleted and expunged items remain on-disk: > > -D delete-days >Remove previously deleted mailboxes older than delete-days (when >using the "delayed" delete mode). "Older than" is a confusing statement. According to my notes this means a message is kept for a minimum of X days from delivery, not X number of days *after* it is deleted. I don't know why my notes say this, but I remember testing it and convincing myself it is true. So -X / -D may not do exactly what you are assuming they do. Am I wrong? (and are you certain I'm wrong?) Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
Patrick Boutilier wrote: > On 06/09/2011 01:48 AM, Diego Ventrice wrote: >> I'm planning a backup strategy for a new server that will handle >> Hundreds of mailboxes. Largest server for me to admin so far >> >> Would you share your experiences backing up cyrus mailboxes ? >> What tool/s do you use ? how many backups a day do you run ? > > We backup all the files on the server using TSM, incrementals nightly. > > Server using delayed delete and delayed expunge. > > Also use cyrus replication to a second server. > > And we backup mailboxes.db to a text file once a day. This is exactly what we do. Deleted and expunged messages are kept for up to 2 days, but I'll likely extend this to avoid trips to `the wayback machine.'' A surprising number of people don't notice they deleted messages, or don't report it for a few days. Mike -- Michael D. Sofka sof...@rpi.edu C&MT Sr. Systems Programmer, Email, HPC, TeX, Epistemology Rensselaer Polytechnic Institute, Troy, NY. http://www.rpi.edu/~sofkam/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
--On Friday, June 10, 2011 10:03 -0500 Blake Hudson wrote: > Delayed > delete/expunge serves this task better than frequent backups, but has > the consequence of needing a lot more production storage - which is > usually more expensive than backup storage. It's the only way to restore ALL mail, since backups miss anything that came and went between backups. The additional storage is only for storing messages expunged in the past N days. A short 7 days meets most needs, and also, then your backups would be complete. Weigh the cost of disk space against the staff cost of mounting backups, finding and copying data, and doing reconstructs. Joseph Brennan Columbia University Information Technology Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
rsync As often as you want (we perform between 1 and 24 snapshots per day, depending on the server). Expect someone to want x message(s) restored that he/she accidentally deleted. If you don't use delayed expunge/delete, you'll probably want to retain a week's worth of daily backups for such events. Delayed delete/expunge serves this task better than frequent backups, but has the consequence of needing a lot more production storage - which is usually more expensive than backup storage. --Blake Original Message ---- Subject: Backing up mailboxes From: Diego Ventrice To: info-cyrus@lists.andrew.cmu.edu Date: Wednesday, June 08, 2011 11:48:38 PM > I'm planning a backup strategy for a new server that will handle > Hundreds of mailboxes. Largest server for me to admin so far > > Would you share your experiences backing up cyrus mailboxes ? > What tool/s do you use ? how many backups a day do you run ? > > > Any tips will be really appreciated. > > Thanks a lot > D. > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
On 06/09/2011 01:48 AM, Diego Ventrice wrote: > I'm planning a backup strategy for a new server that will handle > Hundreds of mailboxes. Largest server for me to admin so far > > Would you share your experiences backing up cyrus mailboxes ? > What tool/s do you use ? how many backups a day do you run ? We backup all the files on the server using TSM, incrementals nightly. Server using delayed delete and delayed expunge. Also use cyrus replication to a second server. And we backup mailboxes.db to a text file once a day. > > > Any tips will be really appreciated. > > Thanks a lot > D. > > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Backing up mailboxes
On Thu, 9 Jun 2011, Diego Ventrice wrote:
> I'm planning a backup strategy for a new server that will handle
> Hundreds of mailboxes. Largest server for me to admin so far
>
> Would you share your experiences backing up cyrus mailboxes ?
> What tool/s do you use ? how many backups a day do you run ?
Enabling delayed delete and delayed expunge will help a lot:
delete_mode: delayed
expunge_mode: delayed
You'll also want to modify your expire line in cyrus.conf to something
like this:
delprune cmd="/usr/local/cyrus/bin/cyr_expire -E 1 -X 7 -D 7" at=0100
-D and -X control how long deleted and expunged items remain on-disk:
-D delete-days
Remove previously deleted mailboxes older than delete-days (when
using the "delayed" delete mode).
-X expunge-days
Expunge previously deleted messages older than expunge-days
(when using the "delayed" expunge mode).
In the example above, messages and mailboxes are deleted from disk after 7
days. This gives you a 7 day window to very easily restore deleted
mailboxes and messages. At our site, I hardly ever have to restore email
from backups.
We use EMC Networker for backups here. Some folks have systems setup to
do snapshots of the filesystem before a backup, but we just backup the
files like any other filesystem. However, the mailboxes database is very
important. We dump it to a flat file every hour, like so:
DATE=`date +%Y%m%d-%H%M`
# Make a plaintext dump of the Cyrus mailboxes database
/usr/local/cyrus/bin/ctl_mboxlist -d > \
/var/spool/cyrus/config/db-dumps/mailboxes.db.dump.${DATE}
# Keep 2 days of dumps
find /var/spool/cyrus/config/db-dumps/ -mtime +1 -exec rm {} \;
Let me know if you have any other questions.
Andy
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Backing up mailboxes
I'm planning a backup strategy for a new server that will handle Hundreds of mailboxes. Largest server for me to admin so far Would you share your experiences backing up cyrus mailboxes ? What tool/s do you use ? how many backups a day do you run ? Any tips will be really appreciated. Thanks a lot D. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
