[vchkpw] rsync
Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. -- Regards, Tariq Azad Sr. Network Engineer Micronet Broadband (Pvt) Ltd. 73-E GD- Arcade Fazal-ul-Haq Road Islamabad. UAN :- 111-114-444 www.dsl.net.pk, www.nayatel.com !DSPAM:47a2f43f310541211020619!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Try nfs or whatever backend storage interface you fancy for your storage server and put /home/vpopmail/domains there. !DSPAM:47a32ca0310542069149568!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Sigh, I guess you meant qmail configuration and not the mail store. Don't post when you get back home late from work. You could try putting bits of qmail on a shared filesystem of some sort but /var/qmail/queue will be local to the box. Changes to /var/qmail/control/*, /var/qmail/users/* will then be effective immediately without affecting the queues. !DSPAM:47a32d9a310544057367207!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: #!/usr/bin/perl @weekDays = qw(Sun Mon Tue Wed Thu Fri Sat); ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = localtime(); $dayOfWeek = $dayOfWeek - '1'; $TODAYIS = $weekDays[$dayOfWeek]; $RSYNCP = /usr/local/bin/rsync -vazH --stats --password-file=/etc/rsync.secret --delete --delete-after /home/vpopmail/domains [EMAIL PROTECTED]::vpop-$TODAYIS; system($RSYNCP) ; /etc/rsyncd.conf [vpop-Sun] path = /backups/machine-or-mailstore_name/vpopmail/sun auth users = backupuser secrets file = /etc/rsyncd.secrets max connections = 50 hosts allow = 203.h.i.dd.en uid = 0 gid = 0 list = no read only = no ...just repeat that block for all 7 days changing /sun to what_ever_day Theres probably better ways of doing it, but this way has worked for me for years. !DSPAM:47a3ca62310545201136119!
Re: [vchkpw] rsync
Quey wrote: I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: Theres probably better ways of doing it, but this way has worked for me for years. Another way, not necessarily better, but if you can do it, it is better, is to use MySQL replication as then it is real time backed up all the time. Steve !DSPAM:47a3cbbc310541988152250!
Re: [vchkpw] rsync
Steve wrote: Quey wrote: I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: Theres probably better ways of doing it, but this way has worked for me for years. Another way, not necessarily better, but if you can do it, it is better, is to use MySQL replication as then it is real time backed up all the time. Steve Yup, however backups are done like this for a reason, say its a hosting server, a reseller accidentally deletes a domain by mistake, it's a friday afternoon, he's going away now and comes back monday morning to find a trillion messages on his phones and emails, if you do only replication, its lost, its lost pretty much almost the same time because its been instructed to DELETE from * etc, and the mailstore is certainly gone, my implication of doing all databases hourly x 7 days stems from my concern of trust in programmers making mistakes :) ehich of course they all claim they never do, unitl you realise you lost some pretty important data, especially the type of data you need to bill clients :) Now if you only do a normal rsync backup, it's also lost for good, you can not recover, doing it my way means we can recover, might have lost a couple days but we can recover their mail (so long as they dont go away for over a week anyway hehehe) Of course for normal ISP end-user-bases (dial/dsl/cable/etc) a single rsync is fine, but still backup all DB's for a week !DSPAM:47a3d029310541244745340!
Re: [vchkpw] rsync
Now if you only do a normal rsync backup, it's also lost for good, you can not recover, doing it my way means we can recover, might have lost a couple days but we can recover their mail (so long as they dont go away for over a week anyway hehehe) That is why I rsync to a backup server running zfs and use snapshots. I have been able to restore mails that a user deleted accidentally just before a rsync (rsync runs once an hour during working hours) thanks to snapshots. !DSPAM:47a3fc35310541048310599!
Re: [vchkpw] rsync
Quey wrote: Yup, however backups are done like this for a reason, say its a hosting server, a reseller accidentally deletes a domain by mistake, it's a friday afternoon, he's going away now and comes back monday morning to find a trillion messages on his phones and emails, if you do only replication, its lost, its lost pretty much almost the same time because its been instructed to DELETE from * etc, and the mailstore is certainly gone, my implication of doing all databases hourly x 7 days stems from my concern of trust in programmers making mistakes :) ehich of course they all claim they never do, unitl you realise you lost some pretty important data, especially the type of data you need to bill clients :) Yep, I understand, but, realize too it depends on your replication procedure too, and also if you use a log file for your statements. From a log file, you can roll forward to the point in time just before the deletion and not lose the data. Which you can't do from a backup. BUT, in no way did I mean to suggest a backup is not a good thing! Replication in no way replaces a backup. It supplements it. We do both. So, I agree with you, just adding another enhancemet, replication. It just isn't easy to set up for everyone, and, ideally, you do use SSL to encrypt the data. Steve !DSPAM:47a40c96310542715241207!
Re: [vchkpw] Rsync concerns
I had already tried this, however, I pipe the output of the command to a log file which I check most days to ensure it is still working. For some reason the errors don't go to the log but get sent to root. If I use -q I get no output in the log even. Hi Andrew, Well, it seems to be a silly mistake - i believe your error messages are going to root because your pipe is redirecting only the stdout of the first command, letting the stderr to the console (which is forwarded to root). Try this: command_a 21 | command_b And I read your posting about using a LDAP server in order to have a synchronized backup server. I don't know if you are aware that when you use the vpopmail LDAP backend only the authentication data will be synchronized. You'll still need to use rsync in order to copy the maildirs. (I'm sorry if you already know that). So maybe you wouldn't like to move to LDAP and keep using vpopmail.cdb backend and rsync to synchronize. Regards, bruno.
RE: [vchkpw] Rsync concerns
Hi, Thanks - will try this. Actually my LDAP post was regarding using LDAP as authentication for another program which only uses LDAP. I want to try and do LDAP lookups on the SMTP envelope on another gateway product before forwarding it to Qmail. I am still wanting to use maildirs (of course!). Thanks. Andrew. -Original Message- From: Bruno Negrao [mailto:[EMAIL PROTECTED] Sent: 14 September 2005 17:08 To: vchkpw@inter7.com Subject: Re: [vchkpw] Rsync concerns I had already tried this, however, I pipe the output of the command to a log file which I check most days to ensure it is still working. For some reason the errors don't go to the log but get sent to root. If I use -q I get no output in the log even. Hi Andrew, Well, it seems to be a silly mistake - i believe your error messages are going to root because your pipe is redirecting only the stdout of the first command, letting the stderr to the console (which is forwarded to root). Try this: command_a 21 | command_b And I read your posting about using a LDAP server in order to have a synchronized backup server. I don't know if you are aware that when you use the vpopmail LDAP backend only the authentication data will be synchronized. You'll still need to use rsync in order to copy the maildirs. (I'm sorry if you already know that). So maybe you wouldn't like to move to LDAP and keep using vpopmail.cdb backend and rsync to synchronize. Regards, bruno. ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. allpay.net Limited, Fortis et Fides, Whitestone Business Park, Whitestone, Hereford, HR1 3SE. Telephone: 0870 243 3434, Fax: 0870 243 6041. Website: www.allpay.net Email: [EMAIL PROTECTED] **
RE: [vchkpw] Rsync concerns
-Original Message- From: Ken Jones [mailto:[EMAIL PROTECTED] Sent: 12 September 2005 19:02 To: vchkpw@inter7.com Subject: Re: [vchkpw] Rsync concerns Hi Ken, Andrew Hodgson wrote: Hi, I have two Vpopmail servers - one which is a backup of the other at a remote site. I am currently using Rsync to mirror /home/vpopmail/domains/virtualdomain, however, while this works without errors during the night, during the day we get errors in the Cronlog because the users are collecting mail via POP. I am not too bothered about this, but wonder what other people are doing for simple backups like these?? I run the Rsync each hour. I use rsync too. I just ignore those errors about email files not existing. You could try adding the -q (quiet) option to stop those messages. I had already tried this, however, I pipe the output of the command to a log file which I check most days to ensure it is still working. For some reason the errors don't go to the log but get sent to root. If I use -q I get no output in the log even. Thanks. Andrew. ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. allpay.net Limited, Fortis et Fides, Whitestone Business Park, Whitestone, Hereford, HR1 3SE. Telephone: 0870 243 3434, Fax: 0870 243 6041. Website: www.allpay.net Email: [EMAIL PROTECTED] **
[vchkpw] Rsync concerns
Hi, I have two Vpopmail servers - one which is a backup of the other at a remote site. I am currently using Rsync to mirror /home/vpopmail/domains/virtualdomain, however, while this works without errors during the night, during the day we get errors in the Cronlog because the users are collecting mail via POP. I am not too bothered about this, but wonder what other people are doing for simple backups like these?? I run the Rsync each hour. Thanks. Andrew. ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. allpay.net Limited, Fortis et Fides, Whitestone Business Park, Whitestone, Hereford, HR1 3SE. Telephone: 0870 243 3434, Fax: 0870 243 6041. Website: www.allpay.net Email: [EMAIL PROTECTED] **
Re: [vchkpw] Rsync concerns
Andrew Hodgson wrote: Hi, I have two Vpopmail servers - one which is a backup of the other at a remote site. I am currently using Rsync to mirror /home/vpopmail/domains/virtualdomain, however, while this works without errors during the night, during the day we get errors in the Cronlog because the users are collecting mail via POP. I am not too bothered about this, but wonder what other people are doing for simple backups like these?? I run the Rsync each hour. I use rsync too. I just ignore those errors about email files not existing. You could try adding the -q (quiet) option to stop those messages. Ken Jones
Re: [vchkpw] Rsync concerns
On 2005-09-12, at 0348, Andrew Hodgson wrote: I am currently using Rsync to mirror /home/vpopmail/domains/ virtualdomain, however, while this works without errors during the night, during the day we get errors in the Cronlog because the users are collecting mail via POP. I am not too bothered about this, but wonder what other people are doing for simple backups like these?? I run the Rsync each hour. i'm doing essentially the same thing, rsync within ssh, to keep a mirror of my server here at the house. i also get these errors when people delete things from their mailboxes while the rsync is happening. what i do is manually eyeball the list of errors and make sure that the only errors showing up involve messages which may be deleted while the rsync is happening. it depends on how many users you have, but it usually takes about 20 seconds to let my eye wander down the list. my own rsync job is backing up much more than just mailboxes- i also have to mentally skip over multilog @4... files as well, because sometimes my log rotation script runs while the rsync is happening. i'm thinking about having the rsync cron job pipe rsync's output through a grep -v to remove these common errors from the email i receive, but i haven't actually done it yet- i guess it hasn't become irritating enough for me to really care yet. -- | John M. Simpson - KG4ZOW - Programmer At Large | | http://www.jms1.net/ [EMAIL PROTECTED] | -- | Mac OS X proves that it's easier to make UNIX | | pretty than it is to make Windows secure. | -- PGP.sig Description: This is a digitally signed message part
