RE: [AMaViS-user] FINAL DECISION: Will our machine handle it
-Original Message- From: Clifton Royston [mailto:[EMAIL PROTECTED] Sent: Tuesday, 26 July 2005 6:49 a.m. To: Stuart Johnston Cc: [EMAIL PROTECTED]; amavis-user@lists.sourceforge.net Subject: Re: [AMaViS-user] FINAL DECISION: Will our machine handle it I thought that rejecting non-existent users at SMTP time was considered a bad idea because now the spammer knows that any messages that are accepted are valid email addresses. I believe this was always largely a myth. While there was evidence of a few spammers who actually winnowed out their lists, most do not. Agreed. This is pretty clear proof that the vast majority of spammers never winnow their address lists. If you accept all addresses, you will just end up with more junk to deal with. Exactly, and it opens you to backscatter attacks. For those interested, first part of my presentation shows how bounce attacks work and why you should block them (it's not the best thing in the PDF, if someone is interested let me know and I can send you the PPT). Spammers today usually don't care if the address is existent or not - they have hunderds, if not thousands, of bot machines which will just spit out spam to any address, including those found on the local machine. In order for a spammer to detect that the address is not in use (ie. it's rejected), a spammer has to: 1) either write a program which will deliver e-mail directly to your server so he can parse replies; or 2) monitor bounces to the e-mail address they put in the From: field. As we can see, 2) doesn't work at all because spammers won't use legitimate From: addresses; besides that no matter what they did, their inbox would probably be flooded in a matter of seconds. 1) can work, but I'm pretty sure they are still in fire forget mode. Cheers, Bojan --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
I thought that rejecting non-existent users at SMTP time was considered a bad idea because now the spammer knows that any messages that are accepted are valid email addresses. Is this no longer considered a best practice? Yes it's no longer considered best practice. The problem is that the accept-then-reject alternative wastes more resources due to accepting the whole message (what if it was an 8MB email?) and then bouncing back later. Also, it is possible to bounce the message back to forged from: addresses, wherease rejecting at SMTP time does not send a message back - it is the connecting MTA's responsibility to inform the sender. --- Chris Covington IT Plus One Health Management 75 Maiden Lane Suite 801 NY, NY 10038 646-312-6269 http://www.plusoneactive.com --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_idt77alloc_id492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it?
On Fri, 2005-07-22 at 00:35 -0400, Matt Juszczak wrote: Hi all, OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. Um - doesn't what you've described here mean that anything delivered to the secondary MX won't be spam or virus checked? Or are you planning on that server doing that as well, on the basis that it shouldn't see too much spam since the other two will pick it up first? If the former then this is doomed to failure - sooner rather than later! If the latter then I'm not sure how well this will work - a *lot* OF spam software I see tries the lowest prio MX *first* (presumably becuase often this belongs to your ISP and won't have the same level of restrictions on it that the primary does). Personally I'd set 1/2 the domains to have one of the 1Us as primary and the other 1U as secondary and the other 1/2 domains the other way round, and have the main server only accept mail from the 2 1Us. Rgds Pete The reason I decided this is for a few reasons: 1) Tonight I upgraded nss_ldap on the mail server and I messed some stuff up bad (it worked on the testing box, btw). It took me 20 minutes to fix it. 2) Mail processing is easy. Spam and antivirus processing is a bit more complicated process. Since I'll have two boxes doing the processing, I can easily take one of the boxes down if something goes wrong (IE, I can take relay1 down at anytime, and relay2 will still function for all mail because of backup MX records). 3) It takes the load off the mail server and uncomplicates things. If something on the mail server breaks, I'll have to figure out whether its the LDA, MTA, amavisd, spamd, or antivirus, or even LDAP. Now, I divide it up a bit to make things easier. Please let me know what all of you think about this final idea. In the end it leaves me with a three server setup but at least things will be a bit more spread out, and I'll have nice backup processing servers. Regards, Matt --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/ --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
Gary V wrote: Stuart wrote: Bojan Zdrnja wrote: I completely agree with Gary. Rejecting e-mail for non existent users *at the front-end* is a MUST. I thought that rejecting non-existent users at SMTP time was considered a bad idea because now the spammer knows that any messages that are accepted are valid email addresses. Is this no longer considered a best practice? I'm just curious to hear opinions -- thanks, Stuart Johnston Here is an interesting thread (that Bojan was involved with) that glowingly illustrates good reason to immediately reject mail to unknown users: http://list.waikato.ac.nz/pipermail/nznog/2005-January/009499.html Any suggestions on how to set this up with Postfix 2.0 (RHEL3) and LDAP? Thanks, Stuart Johnston --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
On Mon, Jul 25, 2005 at 09:12:42AM -0500, Stuart Johnston wrote: Bojan Zdrnja wrote: I completely agree with Gary. Rejecting e-mail for non existent users *at the front-end* is a MUST. I thought that rejecting non-existent users at SMTP time was considered a bad idea because now the spammer knows that any messages that are accepted are valid email addresses. I believe this was always largely a myth. While there was evidence of a few spammers who actually winnowed out their lists, most do not. I know a fellow who is a major anti-spam authority and maintains the mail filtering infrastructure for a very large multinational corporation. They went through a domain name change, as a part of which addresses in their former domain name became inactive. First every address in it was refused (rejected at SMTP time) for a considerable period of time; then all MX records were removed for the entire domain. After several years, he reactivated it as a spamtrap, and almost immediately began receiving tens of thousands of messages per day. So three years of non-deliverability did not cause those addresses to be properly validated or invalidated. Working at an ISP I have also seen the effects of spam runs headed for customer mailservers downstream who maintain this strategy. If they don't accept everything at SMTP time, our backup MX has to do the same. One customer with a MS Exchange server was following this approach; when a major spam run hit them one weekend, their mailserver fell right over, and we ended up with 20K spams queued for non-existent addresses in their domain over the course of a couple hours. This is pretty clear proof that the vast majority of spammers never winnow their address lists. If you accept all addresses, you will just end up with more junk to deal with. -- Clifton -- Clifton Royston -- [EMAIL PROTECTED] Tiki Technologies Lead Programmer/Software Architect My own personal theory is that this is the very dawn of the world. We're hardly more than an eyeblink away from the fall of Troy, and scarcely an interglaciation removed from the Altamira cave painters. We live in extremely interesting ancient times. I like this idea. It encourages us to be earnest and ingenious and brave, as befits ancestral peoples; but keeps us from deciding that because we don't know all the answers, they must be unknowable and thus unprofitable to pursue. -- Teresa Nielsen Hayden, 1995 --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it?
I cast my vote: yeah Gary V Matt wrote: Hi all, OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. The reason I decided this is for a few reasons: 1) Tonight I upgraded nss_ldap on the mail server and I messed some stuff up bad (it worked on the testing box, btw). It took me 20 minutes to fix it. 2) Mail processing is easy. Spam and antivirus processing is a bit more complicated process. Since I'll have two boxes doing the processing, I can easily take one of the boxes down if something goes wrong (IE, I can take relay1 down at anytime, and relay2 will still function for all mail because of backup MX records). 3) It takes the load off the mail server and uncomplicates things. If something on the mail server breaks, I'll have to figure out whether its the LDA, MTA, amavisd, spamd, or antivirus, or even LDAP. Now, I divide it up a bit to make things easier. Please let me know what all of you think about this final idea. In the end it leaves me with a three server setup but at least things will be a bit more spread out, and I'll have nice backup processing servers. Regards, Matt --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
On Fri, 2005-07-22 at 00:35 -0400, Matt Juszczak wrote: Hi all, OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. Um - doesn't what you've described here mean that anything delivered to the secondary MX won't be spam or virus checked? Or are you planning on that server doing that as well, on the basis that it shouldn't see too much spam since the other two will pick it up first? If the former then this is doomed to failure - sooner rather than later! If the latter then I'm not sure how well this will work - a *lot* OF spam software I see tries the lowest prio MX *first* (presumably becuase often this belongs to your ISP and won't have the same level of restrictions on it that the primary does). Personally I'd set 1/2 the domains to have one of the 1Us as primary and the other 1U as secondary and the other 1/2 domains the other way round, and have the main server only accept mail from the 2 1Us. Rgds Pete --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
On Fri, 2005-07-22 at 16:58, Gary V wrote: Pete wrote: Personally I'd set 1/2 the domains to have one of the 1Us as primary and the other 1U as secondary and the other 1/2 domains the other way round, and have the main server only accept mail from the 2 1Us. Rgds Pete I don't mean to speak for Matt here, but I think you have misunderstood, Pete. The way I read it, this IS how it is going to be set up. Both 1U's will filter everything (half and half), then relay to the LDA. Each 1U is set as a backup for the other. Then I would assume that after a couple weeks (to give time for external name servers to clear their cache), the LDA will be reconfigured to only accept mail from the two 1U's. If the LDA is currently only accepting mail from Postini, then it would be configured to accept mail from Postini and the 2 1U's for a couple weeks (or longer if desired), then drop Postini after that. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... We're in agreement, but arguing over semantics I suspect ;) Pete --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
Pete wrote: On Fri, 2005-07-22 at 16:58, Gary V wrote: Pete wrote: Personally I'd set 1/2 the domains to have one of the 1Us as primary and the other 1U as secondary and the other 1/2 domains the other way round, and have the main server only accept mail from the 2 1Us. Rgds Pete I don't mean to speak for Matt here, but I think you have misunderstood, Pete. The way I read it, this IS how it is going to be set up. Both 1U's will filter everything (half and half), then relay to the LDA. Each 1U is set as a backup for the other. Then I would assume that after a couple weeks (to give time for external name servers to clear their cache), the LDA will be reconfigured to only accept mail from the two 1U's. If the LDA is currently only accepting mail from Postini, then it would be configured to accept mail from Postini and the 2 1U's for a couple weeks (or longer if desired), then drop Postini after that. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... Good point, I glossed right over that and made an assumption he was talking about the other 1U, but it appears it refers to the LDA. In that case, all your comments are 100% correct. The LDA will get slammed if it is set up as secondary. Most notably by dictionary attacks. We're in agreement, but arguing over semantics I suspect ;) Pete Yep, except that we are not even arguing 8-} Gary V --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it?
On Fri, Jul 22, 2005 at 12:35:04AM -0400, Matt Juszczak wrote: OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. Excellent plan; this is pretty much optimal. If I'd realized you had two machines to spare, I would have recommended this. The reason I decided this is for a few reasons: ... All good reasons. Please let me know what all of you think about this final idea. In the end it leaves me with a three server setup but at least things will be a bit more spread out, and I'll have nice backup processing servers. The one catch in this suggestion is that the more sophisticated variety of both viruses and spammers will try to go around your spam filter servers to hit your mailserver directly. This can mean getting totally hammered during a major virus outbreak. Several strong suggestions: 1) Don't list your end mailserver as an MX record; use Postfix transports to route directly it from your antispam filter to your mailserver. 2) Once everything is working right, firewall inbound SMTP connections from outside your IP space or restrict them via an access list. 3) Optionally, name your mailserver something other than mail, mta, mx, etc. because those names are part of what they will look for in DNS. -- Clifton -- Clifton Royston -- [EMAIL PROTECTED] Tiki Technologies Lead Programmer/Software Architect My own personal theory is that this is the very dawn of the world. We're hardly more than an eyeblink away from the fall of Troy, and scarcely an interglaciation removed from the Altamira cave painters. We live in extremely interesting ancient times. I like this idea. It encourages us to be earnest and ingenious and brave, as befits ancestral peoples; but keeps us from deciding that because we don't know all the answers, they must be unknowable and thus unprofitable to pursue. -- Teresa Nielsen Hayden, 1995 --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it?
Looks like a good plan. On the two relay servers I would setup postfix to do a verify on the incoming mail addr. using reject_unverified_recipient and also set out-going e-mail to go trough the relay's as well.. Milton On Fri, 2005-07-22 at 09:21 -1000, Clifton Royston wrote: On Fri, Jul 22, 2005 at 12:35:04AM -0400, Matt Juszczak wrote: OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. Excellent plan; this is pretty much optimal. If I'd realized you had two machines to spare, I would have recommended this. The reason I decided this is for a few reasons: ... All good reasons. Please let me know what all of you think about this final idea. In the end it leaves me with a three server setup but at least things will be a bit more spread out, and I'll have nice backup processing servers. The one catch in this suggestion is that the more sophisticated variety of both viruses and spammers will try to go around your spam filter servers to hit your mailserver directly. This can mean getting totally hammered during a major virus outbreak. Several strong suggestions: 1) Don't list your end mailserver as an MX record; use Postfix transports to route directly it from your antispam filter to your mailserver. 2) Once everything is working right, firewall inbound SMTP connections from outside your IP space or restrict them via an access list. 3) Optionally, name your mailserver something other than mail, mta, mx, etc. because those names are part of what they will look for in DNS. -- Clifton --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
Gary wrote: Pete wrote: On Fri, 2005-07-22 at 16:58, Gary V wrote: Pete wrote: Personally I'd set 1/2 the domains to have one of the 1Us as primary and the other 1U as secondary and the other 1/2 domains the other way round, and have the main server only accept mail from the 2 1Us. Rgds Pete I don't mean to speak for Matt here, but I think you have misunderstood, Pete. The way I read it, this IS how it is going to be set up. Both 1U's will filter everything (half and half), then relay to the LDA. Each 1U is set as a backup for the other. Then I would assume that after a couple weeks (to give time for external name servers to clear their cache), the LDA will be reconfigured to only accept mail from the two 1U's. If the LDA is currently only accepting mail from Postini, then it would be configured to accept mail from Postini and the 2 1U's for a couple weeks (or longer if desired), then drop Postini after that. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... Good point, I glossed right over that and made an assumption he was talking about the other 1U, but it appears it refers to the LDA. In that case, all your comments are 100% correct. The LDA will get slammed if it is set up as secondary. Most notably by dictionary attacks. My own setup is an example. I have two MX (gateway) servers, I have all my domains set to use server1 as primary and server2 as secondary. These machines receive an EQUAL number of delivery attempts! 83% of which are addressed to nonexistent users (and are rejected by Postfix). I'm sure you are aware of this Matt, but on your 2 gateway servers, you MUST reject mail to nonexistent users. I don't know if or how you are doing this now, but I've heard that use of a relay_recipients map may be more efficient than LDAP queries, but of course this means that programs have to be written to extract email addresses from LDAP and load them into the map(s), and of course, this would have to automatically happen on a regular basis. Gary V --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... Good point, I glossed right over that and made an assumption he was talking about the other 1U, but it appears it refers to the LDA. In that case, all your comments are 100% correct. The LDA will get slammed if it is set up as secondary. Most notably by dictionary attacks. I meant the remaining server for each situation. In other words, the domains that have relay1 setup as primary MX will have relay2 as secondary. The domains that have relay2 as primary will have the remaining server (relay1) set as secondary. That way its full redundancy if one goes down. The main mail server will ONLY accept incoming messages from the two 1U's Hope that clarifies. -Matt --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
Matt wrote: I'm sure you are aware of this Matt, but on your 2 gateway servers, you MUST reject mail to nonexistent users. I don't know if or how you are doing this now, but I've heard that use of a relay_recipients map may be more efficient than LDAP queries, but of course this means that programs have to be written to extract email addresses from LDAP and load them into the map(s), and of course, this would have to automatically happen on a regular basis. This thread was only referring to the introduction of amavisd into our network. Postfix is very well configured and has very restrictive smtpd_recipient_restrictions as well as helo_checks, sender_checks, recipient_checks, and the like. About 50% of the mail sent to the server is immediately rejected (without accepting it first). I assume that percentage will increase once postini is abolished. This is all excellent, but as you describe it here, your server does not reject mail to nonexistent users. Please correct me if I am mistaken and it won't be mentioned again. Unless you reject mail to nonexistent users at your gateway servers, amavisd-new will have burn time, energy and CPU power processing each and every one of these worthless mails, not to mention filling up your deferred queues. Like I said, 83% of my mail is addressed to nonexistent users. You have to find a way to reject this dictionary attack crap. The head relay servers (relay1 and relay2) will now takeover the exact configuration our existing mail server has. That way they continue to function as our current mail server does. Your current server delivers mail locally, and the gateway servers will relay mail, so at least in that respect, they must be configured differently, but I think this is assumed. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... I meant the remaining server for each situation. In other words, the domains that have relay1 setup as primary MX will have relay2 as secondary. The domains that have relay2 as primary will have the remaining server (relay1) set as secondary. That way its full redundancy if one goes down. The main mail server will ONLY accept incoming messages from the two 1U's Hope that clarifies. It does, Thanks. And like Clifton said, Excellent plan; this is pretty much optimal. Regards, Matt Gary V --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
RE: [AMaViS-user] FINAL DECISION: Will our machine handle it
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Gary V Sent: Saturday, 23 July 2005 8:04 a.m. To: amavis-user@lists.sourceforge.net Subject: Re: [AMaViS-user] FINAL DECISION: Will our machine handle it I'm sure you are aware of this Matt, but on your 2 gateway servers, you MUST reject mail to nonexistent users. I don't know if or how you are doing this now, but I've heard that use of a relay_recipients map may be more efficient than LDAP queries, but of course this means that programs have to be written to extract email addresses from LDAP and load them into the map(s), and of course, this would have to automatically happen on a regular basis. I completely agree with Gary. Rejecting e-mail for non existent users *at the front-end* is a MUST. There are multiple ways to do it. Using a relay_recipients (or virtual_alias_maps, if you have virtual domains) map will be, of course, more efficient because postfix just checks a local hash table so it's very, very fast. LDAP is easier because both servers will contact only one directory, however, you now have a single point of failure if your LDAP server goes down (that's why I decided to go with local host tables on our system here, if you saw my presentation). Now, that all a side, time to lookup a user is close to zero comparing to the time you will spend on AV and anti-spam checks, so I wouldn't worry about this at all. Cheers, Bojan --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
I completely agree with Gary. Rejecting e-mail for non existent users *at the front-end* is a MUST. There are multiple ways to do it. Using a relay_recipients (or virtual_alias_maps, if you have virtual domains) map will be, of course, more efficient because postfix just checks a local hash table so it's very, very fast. LDAP is easier because both servers will contact only one directory, however, you now have a single point of failure if your LDAP server goes down (that's why I decided to go with local host tables on our system here, if you saw my presentation). Now Hiya :) OK I'll clarify :) The new 1U boxes will use the same config as the existing mail server, including rejecting users that dont exist. Our amavisd settings will also be stored in LDAP, so that look up will take place anyway. Also, we have three redundant LDAP servers. One primary write only and two read only, which are speedy. LDAP runs our entire network, and we have hourly backups of the entire data, and spares that stand by :) I think we're covered from LDAP's end. Its honestly the simplest setup I've ever worked with. Once you understand it, of course. regards, Matt --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it
Gary wrote: Matt wrote: I'm sure you are aware of this Matt, but on your 2 gateway servers, you MUST reject mail to nonexistent users. I don't know if or how you are doing this now, but I've heard that use of a relay_recipients map may be more efficient than LDAP queries, but of course this means that programs have to be written to extract email addresses from LDAP and load them into the map(s), and of course, this would have to automatically happen on a regular basis. This thread was only referring to the introduction of amavisd into our network. Postfix is very well configured and has very restrictive smtpd_recipient_restrictions as well as helo_checks, sender_checks, recipient_checks, and the like. About 50% of the mail sent to the server is immediately rejected (without accepting it first). I assume that percentage will increase once postini is abolished. This is all excellent, but as you describe it here, your server does not reject mail to nonexistent users. Please correct me if I am mistaken and it won't be mentioned again. Unless you reject mail to nonexistent users at your gateway servers, amavisd-new will have burn time, energy and CPU power processing each and every one of these worthless mails, not to mention filling up your deferred queues. Like I said, 83% of my mail is addressed to nonexistent users. You have to find a way to reject this dictionary attack crap. The head relay servers (relay1 and relay2) will now takeover the exact configuration our existing mail server has. That way they continue to function as our current mail server does. Your current server delivers mail locally, and the gateway servers will relay mail, so at least in that respect, they must be configured differently, but I think this is assumed. Depends what Matt meant by 'the remaining server' ie the 'other' 1U, or the LDA... I meant the remaining server for each situation. In other words, the domains that have relay1 setup as primary MX will have relay2 as secondary. The domains that have relay2 as primary will have the remaining server (relay1) set as secondary. That way its full redundancy if one goes down. The main mail server will ONLY accept incoming messages from the two 1U's Hope that clarifies. It does, Thanks. And like Clifton said, Excellent plan; this is pretty much optimal. Regards, Matt Gary V Doh! I am red faced here, but I think I understand what is happening. I am so used to configuring gateway servers that I forgot that it is not necessary to configure an LDA to reject mail to nonexistent recipients, it happens by design with no additional settings. I think that Matt is thinking in terms of an LDA, and not in terms of a relay server. At this point, if postini tries to deliver a message to a nonexistent user, your LDA rejects it, and the reject ends up as just another statistic in your 50% of the mail gets rejected. Postini is the one who pays the price for your reject here, so you don't have to bother yourself about it. Now, when you run your own relay servers, here is what will happen. First understand that by default, a relay server knows nothing about who valid recipients are. It knows to only accept mail to your domains, but that's it. So, your relay server receives a message to a nonexistent user in one of your domains. It get scanned by amavisd-new and is passed to your LDA. The LDA rejects it, and so your gateway server composes a nice DSN and tries to send it to the sender. The sender is of course bogus, so the DSN sits in your deferred queue, and many delivery attempts occur over the next 5 days (Postfix default). Multiply this by 20,000 per day, and in about a week or less you will have no gateway server. You have to use a mechanism to reject mail (at the gateway) addressed to nonexistent recipients. Doing so will drop the volume of mail in the deferred queue by 90%, and will save you from scanning this garbage. I'll bet you the postini server measures its queue lifetime in hours, not days. Gary V --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] FINAL DECISION: Will our machine handle it?
Hi all, OK, I think I've made a final decision on what I'd like to do. I think I'm going to setup two of the 1U boxes we have (the 3.06 ghz machines with IDE drives). I'm going to call one relay1 and one relay2. I'm going to setup MX records for the 500+ domains we have. Half of them will have relay1 as their primary and half of them will have relay2 as their primary. The remaining server will be set as secondary MX. These two 1U boxes will be IDENTICAL and have support for ALL domains. Upon processing of spam and antivirus, each box will then relay the mail directly to the mail server. All the mail server will do is receive the processed emails and deliver them. The reason I decided this is for a few reasons: 1) Tonight I upgraded nss_ldap on the mail server and I messed some stuff up bad (it worked on the testing box, btw). It took me 20 minutes to fix it. 2) Mail processing is easy. Spam and antivirus processing is a bit more complicated process. Since I'll have two boxes doing the processing, I can easily take one of the boxes down if something goes wrong (IE, I can take relay1 down at anytime, and relay2 will still function for all mail because of backup MX records). 3) It takes the load off the mail server and uncomplicates things. If something on the mail server breaks, I'll have to figure out whether its the LDA, MTA, amavisd, spamd, or antivirus, or even LDAP. Now, I divide it up a bit to make things easier. Please let me know what all of you think about this final idea. In the end it leaves me with a three server setup but at least things will be a bit more spread out, and I'll have nice backup processing servers. Regards, Matt --- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477alloc_id=16492op=click ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
