Re: Best practices for backing up small mailserver to remote location
On 2018.08.09. 3:21, Adi Pircalabu wrote: On 09-08-2018 10:05, Kenneth Porter wrote: On 8/7/2018 5:08 PM, Adi Pircalabu wrote: - Since you're on dynamic IP at home, set up a VPN tunnel using the mailserver as server and HTPC as client. OpenVPN is ubiquitous and widely supported. - rsync your mailboxes using the tunnel connection. This way you can back up your entire server, not only the mailboxes. Instead of openvpn, I use openssh. Use compression in the ssh tunnel, not the rsync connection, as rsync compression tends to be buggy and interrupts the download. I run sshd on a non-standard port to keep my logs relatively free of script kiddy noise from people looking for an ssh connection to crack. Run fail2ban to lock out the remaining script kiddies. Use a client certificate to log in with ssh unprompted, making it easy to download in a cron job. There's more than one way to skin a cat :) Moving the ssh port and adding fail2ban in the mix is another option. Personally tend to use VPN tunnels for dynamic IP clients for various reasons, such as being able to lock clients out by revoking keys. I prefer connecting from backup server side (and this will resolve dynamic ip problem in this case), so backups are not accessible from production servers. Another option is rsnapshot (if you need versions), it's rsync over ssh and depending on hard link "magic" it conserves disk space (only one copy of mail, independently how long history you have). But as said before - it better works for maildir. -- KSB
Re: Best practices for backing up small mailserver to remote location
On 09-08-2018 10:05, Kenneth Porter wrote: On 8/7/2018 5:08 PM, Adi Pircalabu wrote: - Since you're on dynamic IP at home, set up a VPN tunnel using the mailserver as server and HTPC as client. OpenVPN is ubiquitous and widely supported. - rsync your mailboxes using the tunnel connection. This way you can back up your entire server, not only the mailboxes. Instead of openvpn, I use openssh. Use compression in the ssh tunnel, not the rsync connection, as rsync compression tends to be buggy and interrupts the download. I run sshd on a non-standard port to keep my logs relatively free of script kiddy noise from people looking for an ssh connection to crack. Run fail2ban to lock out the remaining script kiddies. Use a client certificate to log in with ssh unprompted, making it easy to download in a cron job. There's more than one way to skin a cat :) Moving the ssh port and adding fail2ban in the mix is another option. Personally tend to use VPN tunnels for dynamic IP clients for various reasons, such as being able to lock clients out by revoking keys. -- Adi Pircalabu
Re: Best practices for backing up small mailserver to remote location
On 8/7/2018 5:08 PM, Adi Pircalabu wrote:
- Since you're on dynamic IP at home, set up a VPN tunnel using the
mailserver as server and HTPC as client. OpenVPN is ubiquitous and
widely supported.
- rsync your mailboxes using the tunnel connection.
This way you can back up your entire server, not only the mailboxes.
Instead of openvpn, I use openssh. Use compression in the ssh tunnel,
not the rsync connection, as rsync compression tends to be buggy and
interrupts the download. I run sshd on a non-standard port to keep my
logs relatively free of script kiddy noise from people looking for an
ssh connection to crack. Run fail2ban to lock out the remaining script
kiddies. Use a client certificate to log in with ssh unprompted, making
it easy to download in a cron job.
Here's an example of scripting the download. Uncomment the DRYRUN line
for testing, then comment for production. Add more rsync commands to
back up different partitions. The --one-file-system prevents rsync from
trying to back up /dev, /proc, and /sys. The --delete option will remove
local files that were deleted on the remote server. Use that set of
options once you're happy that the backup is working right.
#!/bin/sh
#set -e
set -x
#DRYRUN=--dry-run
#RSYNC_OPTIONS="$DRYRUN --one-file-system -avH --delete"
RSYNC_OPTIONS="$DRYRUN --one-file-system -avH"
DEST=/home/rsync/Server1
# Allow one hour so we don't burn up our bandwidth allowance
# from a command error
time timeout 1h \
rsync -e 'ssh -C -p 1234' $RSYNC_OPTIONS example.com:/ ${DEST}/ \
--exclude tmp
# add more rsync commands here for other partitions
Re: Best practices for backing up small mailserver to remote location
On Wed, 8 Aug 2018, daniel_1...@protonmail.com wrote: - rsync may not be the best option depending on the format of mailboxes. If you're using maildir or maildir+ that's fine, but what about mbox or dbox ? It depends on the situation. I can't speak for dbox, but if the mbox file is not updated, then it's no different than maildir. (It might actually be slightly faster as you don't read a lot of metadata for boxes with many messages. This is a consideration for Maildir if the filesystem format handles metadata poorly.) For mbox where most of the updates occur at the end of the file (i.e. the latest messages), then you'll have to incur read I/O at the source to calculate rolling checksums, but only the changed blocks will be transferred. The worst situation is a modification at the beginning of a large mbox (e.g. delete first message), which will trigger a full copy. So for mostly static mboxes, and moderately sized active mailboxes, rsync will work fine, especially owing to its simplicity. Joseph Tam
Re: Best practices for backing up small mailserver to remote location
‐‐‐ Original Message ‐‐‐ On August 8, 2018 1:08 AM, Adi Pircalabu wrote: > - rsync may not be the best option depending on the format of mailboxes. If you're using maildir or maildir+ that's fine, but what about mbox or dbox ?
Re: Best practices for backing up small mailserver to remote location
On 08-08-2018 7:48, Ian Evans wrote: My webserver also houses our mailserver. There's about six users on that mail system and I'm thinking it would be good to back up the mailboxes to my always on HTPC computer at home, which is reachable via a dynamic IP service. I know (or think) I need to use doveadm-backup for this but rather than reinvent the wheel (or use the wrong wheel altogether) I'm wondering if anyone can recommend a good tutorial or wiki entry that shows the best way to loop through the users and send their backups to a remote server. Assuming you're running *nix on your HTPC and can install your own software on it a safe, secure and reliable way of doing it is: - Since you're on dynamic IP at home, set up a VPN tunnel using the mailserver as server and HTPC as client. OpenVPN is ubiquitous and widely supported. - rsync your mailboxes using the tunnel connection. This way you can back up your entire server, not only the mailboxes. You can add doveadm in the mix if you want, or use imapsync and so on and so forth. YMMV -- Adi Pircalabu
Best practices for backing up small mailserver to remote location
My webserver also houses our mailserver. There's about six users on that mail system and I'm thinking it would be good to back up the mailboxes to my always on HTPC computer at home, which is reachable via a dynamic IP service. I know (or think) I need to use doveadm-backup for this but rather than reinvent the wheel (or use the wrong wheel altogether) I'm wondering if anyone can recommend a good tutorial or wiki entry that shows the best way to loop through the users and send their backups to a remote server. Thanks.
