Re: [vchkpw] Mail system configuration recommendations
On Tuesday 04 October 2005 03:11 pm, Bill Wichers wrote: hoy do you pass the mails from the MX servers to the mailstore server? you mount the unit with nfs, or you do it with smtproutes? smtproutes. I've been thinking about changing it from smtp to qmtp too, since qmtp seems to be made for this kind of thing, but I haven't had the time. qmtp wouldn't save much, however qmqp could, as your front end machines could be completely queueless (diskless even, perhaps). no point in queueing a message more than once if you're just queueing it to send it directly to the backend :) -Jeremy -- Jeremy Kitchen ++ [EMAIL PROTECTED] In the beginning was The Word and The Word was Content-type: text/plain -- The Word of Bob. pgpq6AvgMoTQ6.pgp Description: PGP signature
Re: [vchkpw] Mail system configuration recommendations
On Tuesday 04 October 2005 11:43 pm, Shane Chrisp wrote: I've been using simscan to block messages that score 8 or higher for at least 3 months now, and haven't had a single complaint. The result is we're blocking 57% of the inbound email, which greatly reduces the load all around (fewer calls to vdelivermail, less load from users downloading the spam, etc.) Based on my logs, we could block 50% by dropping at 12+. I can't believe that there would be a legitimate message scoring a 12+ that I would not want to miss. I believe you have these backwards.. dropping at 8+ includes all of the 12+ range :) Tom and Others, How did/do you go about training SA? Just interested to hear what others are doing. We have been blocking at a score of 6.5 and do get the odd False Positive, though our customers dont seem to mind. the key to training a bayesian classifier is using as much legitimate mail as possible. The more spam you use (especially with spammers making up garbage filler paragraphs) the more you'll flag mail as spam. I, personally, just round up all of my mailing lists and sa-learn --ham off that. about 200k hams. I haven't had a false negative in a long time, and never once have I had a false positive. -Jeremy -- Jeremy Kitchen ++ [EMAIL PROTECTED] In the beginning was The Word and The Word was Content-type: text/plain -- The Word of Bob. pgpHnPM5ITMtg.pgp Description: PGP signature
Re: [vchkpw] Mail system configuration recommendations
On Oct 4, 2005, at 7:09 PM, Bill Wichers wrote: The trouble is that we tag, but don't block, most of the spam since our customers don't want us to maybe cost a sale by blocking something that shouldn't be blocked. I've been using simscan to block messages that score 8 or higher for at least 3 months now, and haven't had a single complaint. The result is we're blocking 57% of the inbound email, which greatly reduces the load all around (fewer calls to vdelivermail, less load from users downloading the spam, etc.) Based on my logs, we could block 50% by dropping at 12+. I can't believe that there would be a legitimate message scoring a 12+ that I would not want to miss. Tom and Others, How did/do you go about training SA? Just interested to hear what others are doing. We have been blocking at a score of 6.5 and do get the odd False Positive, though our customers dont seem to mind. cheers Shane
RE: [vchkpw] Mail system configuration recommendations
Bill hoy do you pass the mails from the MX servers to the mailstore server? you mount the unit with nfs, or you do it with smtproutes? note that you can configure in vpopmail a mysql read server and a mysql write server, so it would be pretty easy so set up replication servers in mysql and use that. with round robin dns you could scale out more replication servers if in need. how many users/domains are you handling? Ingo. -Mensaje original- De: Bill Wichers [mailto:[EMAIL PROTECTED] Enviado el: Martes, 04 de Octubre de 2005 17:27 Para: vchkpw@inter7.com Asunto: [vchkpw] Mail system configuration recommendations I'm probably going to have to scale our mail system (not really just a mail server anymore :-) a bunch in the coming months, and am looking to get any advice from others that are running largish mail systems. Right now I have several inbound MX boxes that do spam/virus filtering and accept mail from the outside world. They all deliver to one beefy server with a big RAID that stores the mail (we call it the mailstore server), and handles POP3/IMAP connections from users. There is a seperate webmail server running squirrelmail, and another server with a fast RAID 10 that does all the outbound SMTP and takes messages in from users and bounces from the other servers. The whole thing is qmail/vpopmail based, with spamassassin and clamav on the inbound MXes. Right now we're planning on splitting out the MySQL stuff (right now that runs on the mailstore server) onto 2 or more MySQL-only servers since they handle a lot of queries for vpopmail (MySQL backend), squirrelmail (user prefs), and spamassassin (user prefs). We'd like to be able to scale the system with minimal use of hardware load balancers due to cost. Most of the examples I see out there use a big server running NFS and several smaller servers that handle user queries for POP/IMAP/etc., but it seems like that would have some issues keeping sessions with authentication, and most of the examples are out of date. Our inbound works great -- we can just add more MX servers, but I'm concerned about the POP3 and IMAP connections, and possibly the outbound SMTP if we need more than one server. Does anyone have some expierience to share or maybe a link to some newer example info? -Bill * Waveform Technology Systems Engineer
RE: [vchkpw] Mail system configuration recommendations
hoy do you pass the mails from the MX servers to the mailstore server? you mount the unit with nfs, or you do it with smtproutes? smtproutes. I've been thinking about changing it from smtp to qmtp too, since qmtp seems to be made for this kind of thing, but I haven't had the time. note that you can configure in vpopmail a mysql read server and a mysql write server, so it would be pretty easy so set up replication servers in mysql and use that. with round robin dns you could scale out more replication servers if in need. Yeah, that's what we were thinking on the MySQL side of things... A few replicated MySQL servers with lots of RAM and RAID. We use round robin DNS to split load between our inbound MX servers, but I don't think that would work for authenticated services like IMAP and POP3. That's my big concern with scaling. how many users/domains are you handling? Right now maybe 1500-2000 or so users, and about 1.5 million messages/day. This amount of load is handled pretty well by our current setup, but I expect in the coming year to be well over 10k users and probably 6 times the message volume, maybe more. We're rolling out a new fiber Internet access product that includes email service for buisness, and I expect a lot of new load from that. -Bill * Waveform Technology Systems Engineer
RE: [vchkpw] Mail system configuration recommendations
Bill: I think that you can have a round mysql replication, the master (all are masters and slaves) that initiates the update in the bin log discards the update when it gets back to it, you can give it a try. i'm looking for a qmtp solution myself, please tell me if you find some info on how to configure it. regards, ingo -Mensaje original- De: Bill Wichers [mailto:[EMAIL PROTECTED] Enviado el: Martes, 04 de Octubre de 2005 18:12 Para: vchkpw@inter7.com Asunto: RE: [vchkpw] Mail system configuration recommendations hoy do you pass the mails from the MX servers to the mailstore server? you mount the unit with nfs, or you do it with smtproutes? smtproutes. I've been thinking about changing it from smtp to qmtp too, since qmtp seems to be made for this kind of thing, but I haven't had the time. note that you can configure in vpopmail a mysql read server and a mysql write server, so it would be pretty easy so set up replication servers in mysql and use that. with round robin dns you could scale out more replication servers if in need. Yeah, that's what we were thinking on the MySQL side of things... A few replicated MySQL servers with lots of RAM and RAID. We use round robin DNS to split load between our inbound MX servers, but I don't think that would work for authenticated services like IMAP and POP3. That's my big concern with scaling. how many users/domains are you handling? Right now maybe 1500-2000 or so users, and about 1.5 million messages/day. This amount of load is handled pretty well by our current setup, but I expect in the coming year to be well over 10k users and probably 6 times the message volume, maybe more. We're rolling out a new fiber Internet access product that includes email service for buisness, and I expect a lot of new load from that. -Bill * Waveform Technology Systems Engineer
Re: [vchkpw] Mail system configuration recommendations
Bill Wichers wrote: Right now maybe 1500-2000 or so users, and about 1.5 million messages/day. This amount of load is handled pretty well by our current setup, but I expect in the coming year to be well over 10k users and probably 6 times the message volume, maybe more. We're rolling out a new fiber Internet access product that includes email service for buisness, and I expect a lot of new load from that. Hi, 1.5 million messages a day for 1500-2000 users ? We have 30K+ users and only see about 10K an hour messages. We use one MX server, one spamd server and two real qmail/vpopmail servers (one server handles one 20K+ domain and the other server handles about 100 domains with about 11K users). We are in the process of switching over to a Netapps server with diskless MX, SA, and vpopmail servers, using MX records with the same weight for the incoming smtp servers. Why is your incoming volume so high ? Regards, Rick
Re: [vchkpw] Mail system configuration recommendations
1.5 million messages a day for 1500-2000 users ? We have 30K+ users and only see about 10K an hour messages. We use one MX server, one spamd server and two real qmail/vpopmail servers (one server handles one 20K+ domain and the other server handles about 100 domains with about 11K users). We are in the process of switching over to a Netapps server with diskless MX, SA, and vpopmail servers, using MX records with the same weight for the incoming smtp servers. Why is your incoming volume so high ? Not always 1.5M/day, but usually in the 1M-1.5M/day range. We front-end for a lot of customers that have their own mail servers, but want us to do spam/virus filtering. This is especially important for the people running MS Exchange servers that seem to have issues with virii. We also have a lot of users that get a lot of spam, which I attribute mostly to having a lot of domains that have been in service for 10+ years. I doubt we get more than maybe 50k-100k messages per day that are actually real (non-spam) messages. The trouble is that we tag, but don't block, most of the spam since our customers don't want us to maybe cost a sale by blocking something that shouldn't be blocked. The spam/virus filtering isn't a big issue since it scales easily with more servers and more MX entries in DNS, but the back-end mailstore server with all the maildirs is more difficult to scale. Probably should have mentioned that a lot of the inbound volume just goes from the MX to the outbound box and then delivers to customer mail servers... The outbound SMTP server also queues messages for any unreachable customer mail servers. I'm assuming you plan to use your Netapp box as a storage platform and will have multiple servers mount it and handle the user load that way? -Bill * Waveform Technology Systems Engineer
Re: [vchkpw] Mail system configuration recommendations
On Oct 4, 2005, at 7:09 PM, Bill Wichers wrote: The trouble is that we tag, but don't block, most of the spam since our customers don't want us to maybe cost a sale by blocking something that shouldn't be blocked. I've been using simscan to block messages that score 8 or higher for at least 3 months now, and haven't had a single complaint. The result is we're blocking 57% of the inbound email, which greatly reduces the load all around (fewer calls to vdelivermail, less load from users downloading the spam, etc.) Based on my logs, we could block 50% by dropping at 12+. I can't believe that there would be a legitimate message scoring a 12+ that I would not want to miss. -- Tom Collins - [EMAIL PROTECTED] QmailAdmin: http://qmailadmin.sf.net/ Vpopmail: http://vpopmail.sf.net/ You don't need a laptop to troubleshoot high-speed Internet: sniffter.com
