Re: [courier-users] Courier Cluster
Svetozar Mihailov pisze: Hello all, Hello Svetozar, Sorry for the late response, but recently I've not had much free time to watch this mailing list. 2.2 I do not want to use load balancer because if load balancer dies everything stop. I will implement monitoring (ping?) daemon, which will add/remove servers to dns if some of them dies. Is there recomended minimal TTL for dns records in this case? Yes, it's true that only one load balancer is not good idea. Hence, you should have two redundant load balancers or second backup load balancer on the alert. In my opinion a load balancer is a base element of modern and high availability cluster in these days. Below is a short list of the most important advantages of the load balancer: 1. Network traffic load balancing My administrator experience is that DNS based load balancing sucks. It's not a blame of architecture of domain names system. It's because of stupid Windows hosts. They ignore setting of DNS TTL and cache resolved IP addresses, so you can't stop them to connect with your cluster nodes when you want to reboot them, for example. Of course, you can move IP addresses between machines, but I don't like that manual job. If you want to switch the network traffic when you upgrade a distro on one of your nodes or do some tests, then you should use some free load balancer. Your jobs will be invisible for your users! We use Keepalived in our clustered mail system and I can recommend it for you if you prefer Linux. 2. Security What do you do when some evil people make DoS attack on your cluster? Probably you drop or reject their IP address on all nodes of the cluster. Yes, you can do it in that way, but it's not quick and handy. If you have load balancer(s), then you can cut off aggressors on only one (or two) machine(s) and the problem is resolved. I hope it will be helpful for you or another Courier admins. My best regards, Pawel Tecza - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Courier Cluster
Hi, Paweł Tęcza schrieb, Am 16.09.2008 10:04: Svetozar Mihailov pisze: 2.2 I do not want to use load balancer because if load balancer dies everything stop. I will implement monitoring (ping?) daemon, which will add/remove servers to dns if some of them dies. Is there recomended minimal TTL for dns records in this case? Yes, it's true that only one load balancer is not good idea. Hence, you should have two redundant load balancers or second backup load balancer on the alert. you could do poor man's loadbalancing by assigning several IP adresses to the same virtual host name which acts as MX. This way there is some kind of round robin (not a real load balancing) among the mail servers. We do this with a little cluster system for failover of different services, including mail. We run the primary and secondary nameservers on the both nodes which advertise their IP adresses with TTL 60. So when a host goes down the nameserver is also down and the IP adress isn't advertised any more. That's not very sophisticated, but cheap... For what the folks want to pay for it it does a great job. Greets, Manuel - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Courier Cluster
Manuel Schneider pisze: Hi, Paweł Tęcza schrieb, Am 16.09.2008 10:04: Svetozar Mihailov pisze: 2.2 I do not want to use load balancer because if load balancer dies everything stop. I will implement monitoring (ping?) daemon, which will add/remove servers to dns if some of them dies. Is there recomended minimal TTL for dns records in this case? Yes, it's true that only one load balancer is not good idea. Hence, you should have two redundant load balancers or second backup load balancer on the alert. Hello Manuel, you could do poor man's loadbalancing by assigning several IP adresses to the same virtual host name which acts as MX. This way there is some kind of round robin (not a real load balancing) among the mail servers. We were doing it exactly in that way before, but as I wrote in my previous post, it's not good solution for serious mail system because of Windows client hosts which piss on DNS TLL. We don't want to do any breaks for our users, if it's not necessary, so now we had to use real load-balancer to switch a traffic while administrator jobs. We do this with a little cluster system for failover of different services, including mail. We run the primary and secondary nameservers on the both nodes which advertise their IP adresses with TTL 60. So when a host goes down the nameserver is also down and the IP adress isn't advertised any more. That's not very sophisticated, but cheap... For what the folks want to pay for it it does a great job. Our university mail system should be available for users by 24 hours per day and 365 days per year, so we have separated and redundant name servers, load-balancers, front-ends with SMTP/POP3/IMAP/webmail/anti-virus servers and anti-spam nodes and storages. All runs on Linux/OpenVZ/free software based platform. As you can see it's not a little system :) Have a nice day, Pawel - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Courier Cluster
Paweł Tęcza wrote: We were doing it exactly in that way before, but as I wrote in my previous post, it's not good solution for serious mail system because of Windows client hosts which piss on DNS TLL. If you have control of the clients, you can stop/disable the DNS Client service to fix that. Good architecture is probably a better solution though. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
[courier-users] has courier any header filters?
i look in documentation and can't understand: may i filter messages by any message headers? some spam messages has one or two headers repeatly presenting in it. -- _ *** * icq: 161874711* * jabber: [EMAIL PROTECTED] * * irc.starlink.ru,#Gene, Devil_InSide * *Registered linux user #450844* *** - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] has courier any header filters?
On Tue, 16 Sep 2008 16:12:06 +0400, -=Devil_InSide=- [EMAIL PROTECTED] wrote: i look in documentation and can't understand: may i filter messages by any message headers? some spam messages has one or two headers repeatly presenting in it. -- _ *** * icq: 161874711* * jabber: [EMAIL PROTECTED] * * irc.starlink.ru,#Gene, Devil_InSide * *Registered linux user #450844* *** Hello, Yes, you can do it with pythonfilter quite easily. In courier-pythonfilter, you can look on the way Xfilter works. As these headers are often forged, I'm not sure it's a good idea to refuse such mails. HTH. Jerome Blion. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] has courier any header filters?
,--[Jerome Blion, Tue, 16 Sep 2008 14:40:33 +0200] |On Tue, 16 Sep 2008 16:12:06 +0400, -=Devil_InSide=- [EMAIL PROTECTED] |wrote: | i look in documentation and can't understand: may i filter messages by |any | message headers? | some spam messages has one or two headers repeatly presenting in it. | | -- |Hello, | |Yes, you can do it with pythonfilter quite easily. |In courier-pythonfilter, you can look on the way Xfilter works. |As these headers are often forged, I'm not sure it's a good idea to refuse |such mails. | |HTH. |Jerome Blion. | |- | `- hmm python-filter has many ways to control mail. and i yet didn't see any documentation about it. is it anywhere? now, i read about maildrop --- i try to use it for example. -- _ *** * icq: 161874711* * jabber: [EMAIL PROTECTED] * * irc.starlink.ru,#Gene, Devil_InSide * *Registered linux user #450844* *** - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Courier Cluster
Steve Shockley pisze: Paweł Tęcza wrote: We were doing it exactly in that way before, but as I wrote in my previous post, it's not good solution for serious mail system because of Windows client hosts which piss on DNS TLL. If you have control of the clients, you can stop/disable the DNS Client service to fix that. Good architecture is probably a better solution though. Hi Steve, Thanks for the hint! I don't use Windows, because I'm happy Linux user, so I didn't know about such interesting service like DNS Client :) (Un)fortunately, I haven't got any control of work/home stations of our university employees and students, because I'm only mail system administrator here. My best regards, Pawel - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] has courier any header filters?
On Tue, 16 Sep 2008 17:11:29 +0400, -=Devil_InSide=- [EMAIL PROTECTED] wrote: ,--[Jerome Blion, Tue, 16 Sep 2008 14:40:33 +0200] |On Tue, 16 Sep 2008 16:12:06 +0400, -=Devil_InSide=- [EMAIL PROTECTED] |wrote: | i look in documentation and can't understand: may i filter messages by |any | message headers? | some spam messages has one or two headers repeatly presenting in it. | | -- |Hello, | |Yes, you can do it with pythonfilter quite easily. |In courier-pythonfilter, you can look on the way Xfilter works. |As these headers are often forged, I'm not sure it's a good idea to refuse |such mails. | |HTH. |Jerome Blion. | |- | `- hmm python-filter has many ways to control mail. Yes that' why it's powerful. At the following URI, you have an example of what pythonfilter could do with special headers: http://phantom.dragonsdawn.net/~gordon/courier-patches/courier-pythonfilter/latest/filters/deliveredto.py and i yet didn't see any documentation about it. is it anywhere? The documentation is contained into the files. now, i read about maildrop --- i try to use it for example. NEVER EVER drop a mail without warning either the recipient or the sender. It's a bad practice. If you drop a message with maildrop, you will have to send another mail to the recipient. HTH. Jerome Blion. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Courier Cluster
On Tue, 2008-09-16 at 10:04 +0200, Paweł Tęcza wrote: Svetozar Mihailov pisze: Hello all, Hello Svetozar, Sorry for the late response, but recently I've not had much free time to watch this mailing list. 2.2 I do not want to use load balancer because if load balancer dies everything stop. I will implement monitoring (ping?) daemon, which will add/remove servers to dns if some of them dies. Is there recomended minimal TTL for dns records in this case? Yes, it's true that only one load balancer is not good idea. Hence, you should have two redundant load balancers or second backup load balancer on the alert. In my opinion a load balancer is a base element of modern and high availability cluster in these days. Below is a short list of the most important advantages of the load balancer: 1. Network traffic load balancing My administrator experience is that DNS based load balancing sucks. It's not a blame of architecture of domain names system. It's because of stupid Windows hosts. They ignore setting of DNS TTL and cache resolved IP addresses, so you can't stop them to connect with your cluster nodes when you want to reboot them, for example. Of course, you can move IP addresses between machines, but I don't like that manual job. If you want to switch the network traffic when you upgrade a distro on one of your nodes or do some tests, then you should use some free load balancer. Your jobs will be invisible for your users! We use Keepalived in our clustered mail system and I can recommend it for you if you prefer Linux. 2. Security What do you do when some evil people make DoS attack on your cluster? Probably you drop or reject their IP address on all nodes of the cluster. Yes, you can do it in that way, but it's not quick and handy. If you have load balancer(s), then you can cut off aggressors on only one (or two) machine(s) and the problem is resolved. I hope it will be helpful for you or another Courier admins. My best regards, Pawel Tecza Hello Pawel, Thanks for answer. Anyway things was changed a little. Now I have cluster from 6 hardware nodes ( Q6600, 8G ram, CentOS 5.2 ) using iscsi gfs2 shared storage plus quorum disk also on iscsi. So I can shutdown 5 nodes, and cluster will be quorate with single node. Courier runs on 4 xen virtual machines. This 4 virtual machines are Courier cluster. Each virtual machine uses 1Gb ram, so I can run all of them on one hardware node only. Virtual machines are setup as cluster services in failover domains. If some of them dies will be automatically restarted. If hardware node dies, virtual machine will be restarted on another hardware node. I setup xen with live migration so I can easy live relocate running virtual machines to another hardware node before shutdown some of them. Also I have another virtual machine in Courier cluster, which is not listed in dns. I use it for testing before apply changes to public machines. Hardware nodes does not have IP address at all on external interfaces, so no one can ever reach them. Hardware nodes can be accessed only from internal network, which have private IP's. This also apply to mysql database. Admin interface is well protected also in similar way. Of course I can run Courier directly on hardware nodes and configure public IP addresses as cluster resource. But in this case if hardware node dies I lose everything in /var/spool/courier. And users will be not happy, me too. I of course prefer Linux. In fact I do not use anything else. And windows dns cache is not problem, because windows machines do not work long enough without reboot. Everything is up and running now. I make final fine tune to entire system. This is storage speed from virtual machine: [EMAIL PROTECTED] mail]# dd if=/dev/zero of=zt bs=1024 count=100 100+0 records in 100+0 records out 102400 bytes (1.0 GB) copied, 7.80421 seconds, 131 MB/s I just need to finish fight with spamassassin and sync current mail storage ( about 900Gb ) to new storage and go in production. After this I will try to make few changes to courier. Most important for me is to modify auth protocol to include sender IP address. With this change my already modified mysql backend will log spammers ip directly to mysql and after few weeks my local rbl dns will be good as public ones. Same logging I must do for spam/virus filters. Best regards, Svetozar Mihailov - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe:
Re: [courier-users] has courier any header filters?
-=Devil_InSide=- wrote: i look in documentation and can't understand: may i filter messages by any message headers? some spam messages has one or two headers repeatly presenting in it. If you're running SpamAssassin, you can fairly easily define new tests: http://wiki.apache.org/spamassassin/WritingRules I'd also suggest using the Pyzor, Razor2, and DCC applications with SA. Using maildrop would be my next suggestion, only if you aren't using SA at all. - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
