Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Pump! From: Natalio Gatti Sent: Monday, May 28, 2012 3:03 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. Here we (I) go. You have two options to change your outbound IP address via iptables in the same box where QMT is running. 1) using src-nat and cron 2) using src-nat and NTH option Option 1) Simple iptables rule, 5-minute rotation via cron tasks. You will need one script for each IP address on your system, created as alias on the same interface. Assuming you have 2 IP (1.1.1.1 and 2.2.2.2) the content of each script shoud be: Script 1: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 Script 2: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 Now you have to create two tasks in your cron, each running every 10 minutes. One running at minutes 5's (5, 15, 25, 35, 45, 55) an the other at 0's (0, 10, 20, 30, 40, 50) of every hour. This scripts will rotate your outbound IP every 5 minutes. If you have more IP's well, you can imagine I have not verified what happens with the connections already established when you change there src-nat. I think that those connections will remain using the assigned IP address. You should verify this, becouse otherwise, you will have problems. Option 2) Iptables has an option to select connections according to his order. That option is called nth and I don't remember if it is compiled by default on CentOs. So you will need to download iptables source code and recompile it (including the kernel) If you have the NTH option, you don't need cron, and let iptables do the job. For example: iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 0 -j SNAT --to-source 1.1.1.1 iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 1 -j SNAT --to-source 2.2.2.2 DISCLAIMER: I have not tested any of the alternatives. I have not verified if the commands have the correct syntax. I'm not responsible if your linux box implode and create a black hole. On Mon, May 28, 2012 at 4:23 PM, F. Mendez fmende...@terra.com wrote: Hi all. Any news about this? From: Alberto López Navarro | HazteOir.org Sent: Wednesday, May 23, 2012 4:36 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Thanks a lot Natalio. I will try this and comment. I was hoping that within the comunity someone were able to help achieve a rotation with a script or modify/patch the qmail-T but seems not. Lets try this. From: Natalio Gatti Sent: Monday, May 28, 2012 3:03 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. Here we (I) go. You have two options to change your outbound IP address via iptables in the same box where QMT is running. 1) using src-nat and cron 2) using src-nat and NTH option Option 1) Simple iptables rule, 5-minute rotation via cron tasks. You will need one script for each IP address on your system, created as alias on the same interface. Assuming you have 2 IP (1.1.1.1 and 2.2.2.2) the content of each script shoud be: Script 1: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 Script 2: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 Now you have to create two tasks in your cron, each running every 10 minutes. One running at minutes 5's (5, 15, 25, 35, 45, 55) an the other at 0's (0, 10, 20, 30, 40, 50) of every hour. This scripts will rotate your outbound IP every 5 minutes. If you have more IP's well, you can imagine I have not verified what happens with the connections already established when you change there src-nat. I think that those connections will remain using the assigned IP address. You should verify this, becouse otherwise, you will have problems. Option 2) Iptables has an option to select connections according to his order. That option is called nth and I don't remember if it is compiled by default on CentOs. So you will need to download iptables source code and recompile it (including the kernel) If you have the NTH option, you don't need cron, and let iptables do the job. For example: iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 0 -j SNAT --to-source 1.1.1.1 iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 1 -j SNAT --to-source 2.2.2.2 DISCLAIMER: I have not tested any of the alternatives. I have not verified if the commands have the correct syntax. I'm not responsible if your linux box implode and create a black hole. On Mon, May 28, 2012 at 4:23 PM, F. Mendez fmende...@terra.com wrote: Hi all. Any news about this? From: Alberto López Navarro | HazteOir.org Sent: Wednesday, May 23, 2012 4:36 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hi all. Any news about this? From: Alberto López Navarro | HazteOir.org Sent: Wednesday, May 23, 2012 4:36 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function as an incoming (MX) host as well. -- -Eric 'shubes' Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Here we (I) go. You have two options to change your outbound IP address via iptables in the same box where QMT is running. 1) using src-nat and cron 2) using src-nat and NTH option *Option 1)* Simple iptables rule, 5-minute rotation via cron tasks. You will need one script for each IP address on your system, created as alias on the same interface. Assuming you have 2 IP (1.1.1.1 and 2.2.2.2) the content of each script shoud be: Script 1: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 Script 2: /sbin/iptables -t nat -D POSTROUTING -p tcp --dport 25 -j SNAT --to-source 1.1.1.1 /sbin/iptables -t nat -A POSTROUTING -p tcp --dport 25 -j SNAT --to-source 2.2.2.2 Now you have to create two tasks in your cron, each running every 10 minutes. One running at minutes 5's (5, 15, 25, 35, 45, 55) an the other at 0's (0, 10, 20, 30, 40, 50) of every hour. This scripts will rotate your outbound IP every 5 minutes. If you have more IP's well, you can imagine I have not verified what happens with the connections already established when you change there src-nat. I think that those connections will remain using the assigned IP address. You should verify this, becouse otherwise, you will have problems. *Option 2)* Iptables has an option to select connections according to his order. That option is called nth and I don't remember if it is compiled by default on CentOs. So you will need to download iptables source code and recompile it (including the kernel) If you have the NTH option, you don't need cron, and let iptables do the job. For example: iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 0 -j SNAT --to-source 1.1.1.1 iptables -t nat -A POSTROUTING -m nth --counter 7 --every 2 --packet 1 -j SNAT --to-source 2.2.2.2 * * *DISCLAIMER*: I have not tested any of the alternatives. I have not verified if the commands have the correct syntax. I'm not responsible if your linux box implode and create a black hole. On Mon, May 28, 2012 at 4:23 PM, F. Mendez fmende...@terra.com wrote: Hi all. Any news about this? *From:* Alberto López Navarro | HazteOir.org a...@hazteoir.org *Sent:* Wednesday, May 23, 2012 4:36 PM *To:* qmailtoaster-list@qmailtoaster.com *Subject:* Re: [qmailtoaster] Re: Help request to comunity on tech issue. I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function as an incoming (MX) host as well. -- -Eric 'shubes' Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@**qmailtoaster.comqmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hi Alberto, have you ever tried that multi domain patch for qmail? Maybe how it rotates domains associated to IPs could be a way to grant rotation. Thanks From: Alberto López Navarro | HazteOir.org Sent: Wednesday, May 23, 2012 4:36 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function as an incoming (MX) host as well. -- -Eric 'shubes' Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. All are balanced to reach no more than 8k emails an hour each. No VM, real boxes working. Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus to this. Regards. -Mensaje original- From: Eric Shubert Sent: Monday, May 21, 2012 6:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' - To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
[qmailtoaster] Re: Help request to comunity on tech issue.
On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function as an incoming (MX) host as well. -- -Eric 'shubes' Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus to this. Regards. -Mensaje original- From: Eric Shubert Sent: Monday, May 21, 2012 6:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
I think the bottleneck must be somewhere else. I'm administering a qmail based mass e-mail system, and we're sending a bulletin to 250.000 members, which takes 6-7 hours, with a single server (a run-of-the-mill Dell PE850). I first had it configured with DKIM but had to turn it off because it was a resource hog. Also, I don't think having a single IP is a problem, I would rather check whether your ISP is capping your bandwidth. Regs, Alberto 2012/5/23 Eric Shubert e...@shubes.net On 05/23/2012 12:31 PM, F. Mendez wrote: Hi Eric, 350 per hour is a very low limit. We work as with the lowest standard at this matter, offering same or less than big hostings like hostgator. Perhaps our language isn't consistent. Are you referring to 350 per hour per domain, or per user? (I'm referring to per user, which I still think is high, unless your clients are doing email marketing). Cluster is: 5 servers, one IP per server, MX priority from 0 to 40 each. That's nice to know, but MX won't have anything to do with outbound messages. All are balanced to reach no more than 8k emails an hour each. Inbound or outbound, or both? I'd be interested to know how you manage to throttle this. No VM, real boxes working. Given your setup, you might configure a round robin for outbound, as I mentioned previously in reply to CJ's post. This isn't ideal performance wise, as each messages would be queued in 2 hosts, but I think it would work adequately. Also, you'll need to be sure that DNS caching doesn't interfere with round robin rotation (I'd test that first before committing to this approach). Otherwise, you might assign multiple addresses to one (or more) hosts, and come up with a way to alternate between addresses. One way would be to modify the qmail-remote program. It might be possible to periodically modify the routing table to achieve the same result, but I'm not sure about that. There are likely other ways as well. Personally, I like the round robin solution because of its simplicity. You would need to have all of the submissions come into one server, and relays go out from the others. I don't think that a host could perform both roles, although a submission or relay server could continue to function as an incoming (MX) host as well. -- -Eric 'shubes' Regards. -Mensaje original- From: Eric Shubert Sent: Tuesday, May 22, 2012 8:14 PM To: qmailtoaster-list@**qmailtoaster.comqmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Délsio: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus to this. Regards. From: Délsio Cabá Sent: Monday, May 21, 2012 10:51 PM To: qmailtoaster-list@qmailtoaster.com Subject: Re: [qmailtoaster] Re: Help request to comunity on tech issue. Hi all, I am also a small ISP but I don't have such problems and I don't use a cluster yet. The easiest solution is normall the best one. If you have a Storage try to implement a Load Balance with multiple mail servers instead of a cluster. This way you will be able to answer smtp/pop3 requests using multiple IP addresses. But before that you should check your bandwidth and delay also. Many problems occur on the transmission side. Regards On 22 May 2012 01:14, Eric Shubert e...@shubes.net wrote: On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus to this. Regards. -Mensaje original- From: Eric Shubert Sent: Monday, May 21, 2012 6:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption
[qmailtoaster] Re: Help request to comunity on tech issue.
Sounds like you've taken great measures to prevent unauthorized use. I still think that 350 per account is high though. That's an average of nearly one every 10 seconds for 60 minutes straight. I think it's safe to say that some of these people are sending to lists. They're your customers though, so I don't doubt that they're generating the volumes you say. How is your cluster presently configured? Are all hosts sending outbound email in a balanced fashion? You've said that you have 5 hosts and 5 IP addresses, but haven't told us much about how things are configured. Are each of these 5 hosts QMTs? On bare iron or virtual? On 05/22/2012 05:23 PM, F. Mendez wrote: Hi Eric. We have modified our control panel so that when clients create a new email, the can't use their own passwords. It is generated with high char random values. We also had put limits to conections and monitor ip conections during smtp/pop tasks. Not more than 1 conection to smtp or pop, and only same IP on both tasks can be accepted. Any other attemp over 5 times blocks the accounts. We also track ip origin on smtp/pop conection and webmail conection. If the regular base is that ip connects from peruvian ranges, and suddenly there is one conection from any other part of the world, then account is blocked and client is asked to fill secret info regarding its account and the 2nd email he/she registered at signup time. Limit to 350 is not high, as our clients are not home users. Over 99% of them are small medium size companys that use alot of emails during day. We already had done a process to determine this and it is a real usage. In same cases it is even not enought. And as I wrote before: Our clients are 99% enterprises. Small, medium size, and thus their needs to send emails is not comparable to regular home users.. Even 350 mails per hour is in some cases not enought. Thought they don’t want to rise their monthly payment or move to dedicateds. So traffic is high. Having multiple servers or having them on cluster is just the same. As each one only have 1 ip, reputation may be affected due to the high volume. Solution is to split as much as possible with diferent ips over each current server on array. We already talked about this with our tech assesor. So please any answer or contributions regarding this thread I would really appreciate that would be focus to this. Regards. -Mensaje original- From: Eric Shubert Sent: Monday, May 21, 2012 6:14 PM To: qmailtoaster-list@qmailtoaster.com Subject: [qmailtoaster] Re: Help request to comunity on tech issue. On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' - To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
[qmailtoaster] Re: Help request to comunity on tech issue.
I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see my situation and can help me with this. We used to work with exim, but since we changed to QMT it was the best desition we ever made on this matters. Now we need to push it to a next level. Thanks a lot. - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Eric: Couldn't you try with fail2ban: - Checking qmail logs http://wiki.qmailtoaster.com/index.php/Fail2Ban - Checking spamdyke logs http://notes.benv.junerules.com/qmail-spamdyke-and-fail2ban/ and also Ernesto Vargas-Azofeifa Senior Web Developer IT Manager Macromedia Certified Cold Fusion Web Developer LAMP stack expert. From: Eric Shubert e...@shubes.net To: qmailtoaster-list@qmailtoaster.com Sent: Monday, May 21, 2012 3:55 PM Subject: [qmailtoaster] Re: Help request to comunity on tech issue. I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see my situation and can help me with this. We used to work with exim, but since we changed to QMT it was the best desition we ever made on this matters. Now we need to push it to a next level. Thanks a lot. - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Thanks. On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com [1] wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see my situation and can help me with this. We used to work with exim, but since we changed to QMT it was the best desition we ever made on this matters. Now we need to push it to a next level. Thanks a lot. - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need professional help with your setup, contact them today! - Please visit qmailtoaster.com for the latest news, updates, and packages. To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com [2] For additional commands, e-mail: qmailtoaster-list-h...@qmailtoaster.com [3]
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Have you tried a DNS round robin solution? On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Thanks. On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see
[qmailtoaster] Re: Help request to comunity on tech issue.
I was going to write that RR would be of no help, then it dawned on me. You could set up a single submission server, then smtproute all outbound messages from it to a DNS round robin set of sending agent machines. Virtual machines would work nicely for this. Goes to show, there's more than one way to do things. :) -- -Eric 'shubes' On 05/21/2012 03:37 PM, Cecil Yother, Jr. wrote: Have you tried a DNS round robin solution? On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Thanks. On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com mailto:fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see my situation and can help me with this. We used to work with exim, but since we changed to QMT it was the best desition we ever made on this matters. Now we need to push it to a next level. Thanks a lot. - Qmailtoaster is sponsored by Vickers Consulting Group (www.vickersconsulting.com) Vickers Consulting Group offers Qmailtoaster support and installations. If you need
[qmailtoaster] Re: Help request to comunity on tech issue.
On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com mailto:fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a random fashion, so even with high demand quality service on mail delivery from client accounts is always achieved. I would like to ask for some guidance and help to this comunity on how can we could implement such solution to rotate in a random or other way the IPs for sending clients mails. I hope you people can see my situation and can help me with this. We used to work with exim, but since we changed to QMT it was the best desition we ever made on this
Re: [qmailtoaster] Re: Help request to comunity on tech issue.
Hi all, I am also a small ISP but I don't have such problems and I don't use a cluster yet. The easiest solution is normall the best one. If you have a Storage try to implement a Load Balance with multiple mail servers instead of a cluster. This way you will be able to answer smtp/pop3 requests using multiple IP addresses. But before that you should check your bandwidth and delay also. Many problems occur on the transmission side. Regards On 22 May 2012 01:14, Eric Shubert e...@shubes.net wrote: On 05/21/2012 03:06 PM, fmende...@terra.com wrote: Hello Eric, thanks for your reply. We do not have spam issues with our customers, what we have is a high volume due to large clients number. With so many clients, the probability of compromised passwords is fairly high. I wouldn't be very quick to dismiss this as a possibility. Do your anti-spam measures have any effect on authenticated smtp sessions? All meassures to void spam sending are taken, but the blocks are being generated for large volume send from just a bunch of IPs (5) which are the number of mta's qmt in our cluster. As all you may know, having 9k clients with at least 4 email accounts per client and a limit of 350 per hour per account, it is still a big traffic generated. 350 per hour per account seems like a high limit to me for typical email use. In any case, how are you enforcing this limit? So I am looking forward to have better service on delivery having in mind that custmer number is growing fast and anti-spam messures do its job preatty good. But of the lack of IP on each mta in cluster, it is affecting delivery. Hope someone around may share a solution. Are all machines in the cluster going out on the the same public IP? If so, I presume you have NAT in effect. If that's the case, you should look into implementing SNAT along with NAT, so the source IP changes according to which machine behind the NAT is the source of the packets. This is something your NAT router needs to do. Thanks. A little more detailed description of your current setup might be helpful for us to know what might be most effective for you. -- -Eric 'shubes' On lun 21/05/12 4:55 PM , Eric Shubert e...@shubes.net sent: I don't know if rotating addresses is the best solution or not. It's certainly not practical for small QMT installations. I think in many (if not all or most) of these cases, the user's password has been compromised. This is especially likely if it's possible to configure a client insecurely (plain text password with no TLS/SSL). I've seen this happen on more than one occasion, on a small domain. Password sniffing does happen. First step is to ensure that clients cannot attempt to authenticate with clear text passwords. This can be enforced with dovecot, but we don't have a way yet to enforce it on the sending/smtp side. I'm hopeful that Sam will get this feature built into spamdyke in the near future. Another good defensive weapon is a script I came across on the spamdyke list today, and hope to make available in some form with QTP in the future. It's a script that periodically checks the logs for accounts which have sent more messages in a given interval than some allowed limit. When it finds such an account, it changes the password, removes messages from that account still in the queue, and notifies the postmaster with an email. I think this is very practical, because passwords do become compromised on occasion, even with full encryption (human action). The script is written in python, and will need a little tweaking for the QMT environment, as it's presently written to scan a spamdyke log (the author wasn't using the submission port at all). I think it'd be better to scan the send log if that's feasible. Anywise, I think this approach is promising. If anyone has any thoughts on this, please chime in. It's in everyone's interest to be protecting our public IP addresses so they don't get blacklisted. Thanks. -- -Eric 'shubes' On 05/21/2012 01:42 PM, fmende...@terra.com mailto:fmende...@terra.com wrote: Hello everyone I am the owner of a growing hosting enterprise in my country (Perú), and we are facing big rise on our client number. As an efect of this we are seeying a rise in mail outbound in our servers. Even thoug we put limits to hourly sending, having more than 9k clients, all delivering through the same cluster, it lacks of efectiveness because each server in cluster uses only one ip for sending tasks. We are now seeying blocking issues because of the many clents generated traffic. We talked to some people at godaddy and hostgator, as we know they use a cluster system that includes on each server a list of IPs that rotates in a
