[vchkpw] rsync
Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. -- Regards, Tariq Azad Sr. Network Engineer Micronet Broadband (Pvt) Ltd. 73-E GD- Arcade Fazal-ul-Haq Road Islamabad. UAN :- 111-114-444 www.dsl.net.pk, www.nayatel.com !DSPAM:47a2f43f310541211020619!
Re: [vchkpw] OT: Which RBLsmtpd lookups are you using?
Adi Pircalabu wrote: On Fri, 01 Feb 2008 07:33:53 +1000 Quey wrote: dnsbl.sorbs.net bl.spamcop.net Don't use these to reject connections at SMTP level, they give many false-positives. Eventually use them only after queueing, and only to increase the spam score. zen.spamhaus.org and list.dsbl.org, au contraire, are much better choices for rblsmtpd. My 0,02RON each to our own, I dont consider they give many false positives at all, not in this part of the world, but of course it may be different for where you are, however even with the acceptable FP's they *may* give, the massive reduction in spam makes it completely worth it. The more they hit there, the less work MailScanner has to do, it can use system resources just to scan for viruses and phishing and of course whatever spam it detects that get past the RBL's :-) A good thing to do as well which also dramatically reduces spam, is enforce DNS forward and reverse, if someone can't be bothered making sure their mail server is RFC compliant, then I am under no obligation to allow my servers to accept connections from them. !DSPAM:47a30057310546247712797!
Re: [vchkpw] OT: Which RBLsmtpd lookups are you using?
On Fri, 01 Feb 2008 07:33:53 +1000 Quey wrote: dnsbl.sorbs.net bl.spamcop.net Don't use these to reject connections at SMTP level, they give many false-positives. Eventually use them only after queueing, and only to increase the spam score. zen.spamhaus.org and list.dsbl.org, au contraire, are much better choices for rblsmtpd. My 0,02RON -- Adi Pircalabu !DSPAM:47a2fce1310546864318583!
Re: [vchkpw] Seems to be a problem with dir_control table
Hmmm... Just removed --disable-users-big-dir from my config and I got what I need. =D Regards! -- [ ]'s Aledr - Alexandre OpenSource Solutions for SmallBusiness Problems !DSPAM:47a309e6310541020117296!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Try nfs or whatever backend storage interface you fancy for your storage server and put /home/vpopmail/domains there. !DSPAM:47a32ca0310542069149568!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Sigh, I guess you meant qmail configuration and not the mail store. Don't post when you get back home late from work. You could try putting bits of qmail on a shared filesystem of some sort but /var/qmail/queue will be local to the box. Changes to /var/qmail/control/*, /var/qmail/users/* will then be effective immediately without affecting the queues. !DSPAM:47a32d9a310544057367207!
[vchkpw] Re: rsync
Hi, ,- - [ Le vendredi 1 février 2008 vers 15:28 Christopher Chan écrivait: ] - - | Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Try nfs or whatever backend storage interface you fancy for your storage server and put /home/vpopmail/domains there. | `- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Storing mail accounts on any backend storage does not have anything to do with the need of backup... nfs or local, the mails must be backup at least every day... We are doing backup trough rsync, and when the mail accounts take more than 10GB (meaning a lot of mails !), the backup take really a lot of time, we needed to split the backup domain per domain... -- Best regards... _ (_' ,_)téphane Bouvard [antarex AT freenet DOT be] http://www.antarex.be !DSPAM:47a33b5d310541471214031!
Re: [vchkpw] OT: Which RBLsmtpd lookups are you using?
Adi Pircalabu wrote: On Fri, 01 Feb 2008 07:33:53 +1000 Quey wrote: dnsbl.sorbs.net bl.spamcop.net Don't use these to reject connections at SMTP level, they give many false-positives. Eventually use them only after queueing, and only to increase the spam score. zen.spamhaus.org and list.dsbl.org, au contraire, are much better choices for rblsmtpd. My 0,02RON each to our own, I dont consider they give many false positives at all, not in this part of the world, but of course it may be different for where you are, however even with the acceptable FP's they *may* give, the massive reduction in spam makes it completely worth it. The more they hit there, the less work MailScanner has to do, it can use system resources just to scan for viruses and phishing and of course whatever spam it detects that get past the RBL's :-) A good thing to do as well which also dramatically reduces spam, is enforce DNS forward and reverse, if someone can't be bothered making sure their mail server is RFC compliant, then I am under no obligation to allow my servers to accept connections from them. My thanks to everyone who contributed! I'm on zen.spamhaus.org now! Noticed at least some increase in stopped connections at smtpd! A good thing, in my review. I'll research FPs for downside. Thanks! !DSPAM:47a381a0310549759113929!
[vchkpw] Backup of vpopmail account databases and mailstore was Re: [vchkpw] Re: rsync
Stephane Bouvard (ML) wrote: Hi, ,- - [ Le vendredi 1 février 2008 vers 15:28 Christopher Chan écrivait: ] - - | Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. Try nfs or whatever backend storage interface you fancy for your storage server and put /home/vpopmail/domains there. | `- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Storing mail accounts on any backend storage does not have anything to do with the need of backup... nfs or local, the mails must be backup at least every day... Heh, sorry for confusing you with my confused post. I doubt the OP was yapping about backup, as in in case you lose data, at all. Mail accounts != mailboxes...at least I consider mail accounts to be whatever is stuffed into the cdb|mysql|pgsql|whatever databases and contain username, password, home directory and what not information and can be be separately maintained from the mailstore. We are doing backup trough rsync, and when the mail accounts take more than 10GB (meaning a lot of mails !), the backup take really a lot of time, we needed to split the backup domain per domain... Do you use snapshots or separate your mailstore per domain? !DSPAM:47a3bbd7310541326559512!
Re: [vchkpw] Re: rsync
Stephane Bouvard (ML) wrote: We are doing backup trough rsync, and when the mail accounts take more than 10GB (meaning a lot of mails !), the backup take really a lot of time, we needed to split the backup domain per domain... define a long time? and what FS do you use (ext2|3/reiser/etc...)? I use to find backing up 100's of GB's took a few hours on ext3, until we moved it to reiserfs now the lots done in an hour. !DSPAM:47a3c5fb310542049912549!
Re: [vchkpw] rsync
Tariq Azad/OPS wrote: Did any body tried using rsync to replicate between backup and primary qmail servers. Is it recommended for a Qmail hosting multiple domains having more then 7000 users. I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: #!/usr/bin/perl @weekDays = qw(Sun Mon Tue Wed Thu Fri Sat); ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek, $dayOfYear, $daylightSavings) = localtime(); $dayOfWeek = $dayOfWeek - '1'; $TODAYIS = $weekDays[$dayOfWeek]; $RSYNCP = /usr/local/bin/rsync -vazH --stats --password-file=/etc/rsync.secret --delete --delete-after /home/vpopmail/domains [EMAIL PROTECTED]::vpop-$TODAYIS; system($RSYNCP) ; /etc/rsyncd.conf [vpop-Sun] path = /backups/machine-or-mailstore_name/vpopmail/sun auth users = backupuser secrets file = /etc/rsyncd.secrets max connections = 50 hosts allow = 203.h.i.dd.en uid = 0 gid = 0 list = no read only = no ...just repeat that block for all 7 days changing /sun to what_ever_day Theres probably better ways of doing it, but this way has worked for me for years. !DSPAM:47a3ca62310545201136119!
Re: [vchkpw] rsync
Quey wrote: I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: Theres probably better ways of doing it, but this way has worked for me for years. Another way, not necessarily better, but if you can do it, it is better, is to use MySQL replication as then it is real time backed up all the time. Steve !DSPAM:47a3cbbc310541988152250!
Re: [vchkpw] rsync
Steve wrote: Quey wrote: I back up our mysql db hourly with mysqldump, on 24 hour x 7day basis and with rsync I do it on all servers regardless of how many users they have, we also do it on a rolling 7 day basis Pretty simple, you can expend on this greatly like I do in my main scripts by using logging, timeouts and alarms but starting point would be: Theres probably better ways of doing it, but this way has worked for me for years. Another way, not necessarily better, but if you can do it, it is better, is to use MySQL replication as then it is real time backed up all the time. Steve Yup, however backups are done like this for a reason, say its a hosting server, a reseller accidentally deletes a domain by mistake, it's a friday afternoon, he's going away now and comes back monday morning to find a trillion messages on his phones and emails, if you do only replication, its lost, its lost pretty much almost the same time because its been instructed to DELETE from * etc, and the mailstore is certainly gone, my implication of doing all databases hourly x 7 days stems from my concern of trust in programmers making mistakes :) ehich of course they all claim they never do, unitl you realise you lost some pretty important data, especially the type of data you need to bill clients :) Now if you only do a normal rsync backup, it's also lost for good, you can not recover, doing it my way means we can recover, might have lost a couple days but we can recover their mail (so long as they dont go away for over a week anyway hehehe) Of course for normal ISP end-user-bases (dial/dsl/cable/etc) a single rsync is fine, but still backup all DB's for a week !DSPAM:47a3d029310541244745340!
Re: [vchkpw] rsync
Now if you only do a normal rsync backup, it's also lost for good, you can not recover, doing it my way means we can recover, might have lost a couple days but we can recover their mail (so long as they dont go away for over a week anyway hehehe) That is why I rsync to a backup server running zfs and use snapshots. I have been able to restore mails that a user deleted accidentally just before a rsync (rsync runs once an hour during working hours) thanks to snapshots. !DSPAM:47a3fc35310541048310599!
Re: [vchkpw] rsync
Quey wrote: Yup, however backups are done like this for a reason, say its a hosting server, a reseller accidentally deletes a domain by mistake, it's a friday afternoon, he's going away now and comes back monday morning to find a trillion messages on his phones and emails, if you do only replication, its lost, its lost pretty much almost the same time because its been instructed to DELETE from * etc, and the mailstore is certainly gone, my implication of doing all databases hourly x 7 days stems from my concern of trust in programmers making mistakes :) ehich of course they all claim they never do, unitl you realise you lost some pretty important data, especially the type of data you need to bill clients :) Yep, I understand, but, realize too it depends on your replication procedure too, and also if you use a log file for your statements. From a log file, you can roll forward to the point in time just before the deletion and not lose the data. Which you can't do from a backup. BUT, in no way did I mean to suggest a backup is not a good thing! Replication in no way replaces a backup. It supplements it. We do both. So, I agree with you, just adding another enhancemet, replication. It just isn't easy to set up for everyone, and, ideally, you do use SSL to encrypt the data. Steve !DSPAM:47a40c96310542715241207!
