Re: [Dovecot] Lots of pop3-logins
On Fri, 2009-06-26 at 02:01 -0700, V S Rao wrote: Timo Wrote: You can also just decrease login_process_max_count Wouldn't decreasing the login_process_max_count simply create more problems. Now users will start experiencing timeouts sooner than before, because whatever is causing the login processes to increase (attack, rogue process or whatever) will *always* be trying to login and genuine users will be denied login. So without knowing the root cause of the issue simply decreasing or increasing the login_process_max_count will lead to other problems. Correct me if I am wrong. Depends on the attacker. Dovecot will always drop the oldest connection. So if attacker is authenticating multiple times in a single session, it's pretty much always the oldest connection that gets killed first. If attacker logins once and then disconnects, I think Dovecot still kills those processes sooner than others, because they're waiting a couple of seconds for authentication failed. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Lots of pop3-logins
Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. My idle box has 64 imap-login processes and no, I'm not under a dictionary attack :) I am not sure what your load is (user base, system config etc), but I will give you my typical load here. I run mail server for about 6000 users with a mix of 70% POP3 and 30% IMAP (thro webmail). And here are the typical stats (I run a script in the background collecting this data every 5 secs): pop3-logins:12 pop3-connections:8 IMAP-logins:7 IMAP-connections:11 I have read other opinions in this thread by Timo others. And I am interested in a few things. So if you will indulge me, maybe it will be useful for others who face these kind of issues Timo Wrote: You can also just decrease login_process_max_count Wouldn't decreasing the login_process_max_count simply create more problems. Now users will start experiencing timeouts sooner than before, because whatever is causing the login processes to increase (attack, rogue process or whatever) will *always* be trying to login and genuine users will be denied login. So without knowing the root cause of the issue simply decreasing or increasing the login_process_max_count will lead to other problems. Correct me if I am wrong. Rodman Wrote: I'll go ahead and lower that limit to something that fits my usage better. No, I think leave that value to default and try and identify the root cause and prevent it. Noel Wrote: What would be nice is, an anti brute force option Yes, that would be nice. But consider a situation where the system is not under brute force attack, but for some reason the number of login processes keep on increasing by the hour. This would ultimately lead the system to deny connections to the users. Is there a way to track what is happening? strace would be too complicated for us field guys to work with. Any suggestions? Regards --Rao
Re: [Dovecot] Lots of pop3-logins
Well concerning my problem, I adjusted fail2ban so that it can parse the maillog and ban IP's that have 6 incorrect pop3 logins. I had another attack last night, but fail2ban got him only have 6 attempts and banned his sorry ass. If anyone wants to see the fail2ban config file I am using for Dovecot, let me know... Rodman - Original Message - From: V S Rao viriy...@yahoo.com To: j...@co.sapo.pt Cc: dovecot@dovecot.org Sent: Friday, June 26, 2009 4:01 AM Subject: Re: [Dovecot] Lots of pop3-logins Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. My idle box has 64 imap-login processes and no, I'm not under a dictionary attack :) I am not sure what your load is (user base, system config etc), but I will give you my typical load here. I run mail server for about 6000 users with a mix of 70% POP3 and 30% IMAP (thro webmail). And here are the typical stats (I run a script in the background collecting this data every 5 secs): pop3-logins:12 pop3-connections:8 IMAP-logins:7 IMAP-connections:11 I have read other opinions in this thread by Timo others. And I am interested in a few things. So if you will indulge me, maybe it will be useful for others who face these kind of issues Timo Wrote: You can also just decrease login_process_max_count Wouldn't decreasing the login_process_max_count simply create more problems. Now users will start experiencing timeouts sooner than before, because whatever is causing the login processes to increase (attack, rogue process or whatever) will *always* be trying to login and genuine users will be denied login. So without knowing the root cause of the issue simply decreasing or increasing the login_process_max_count will lead to other problems. Correct me if I am wrong. Rodman Wrote: I'll go ahead and lower that limit to something that fits my usage better. No, I think leave that value to default and try and identify the root cause and prevent it. Noel Wrote: What would be nice is, an anti brute force option Yes, that would be nice. But consider a situation where the system is not under brute force attack, but for some reason the number of login processes keep on increasing by the hour. This would ultimately lead the system to deny connections to the users. Is there a way to track what is happening? strace would be too complicated for us field guys to work with. Any suggestions? Regards --Rao
Re: [Dovecot] Lots of pop3-logins
On 6/26/2009, Rodman Frowert (rod...@thefrowerts.com) wrote: If anyone wants to see the fail2ban config file I am using for Dovecot, let me know... Does it also work for IMAP ligins? I'd like to see it regardless... thanks! -- Best regards, Charles
Re: [Dovecot] Lots of pop3-logins
Charles,
I haven't tested it with IMAP so I'm not sure. I was going to play with
that later. It could also be modified to ban failed SASL SMTP auths as
well. Here is the line in my /etc/fail2ban/filter.d/dovecot.conf file that
makes it work:
failregex = (?: Disconnected|Aborted
login).*rip=(?:::f{4,6}:)?(?Phost\S*),.*
I have to use the Disconnected AND Aborted login to pick up 100% of
failed pop3's. For some reason, some attacks only show Disconnected in
the logs while the others show as Aborted login. If I try to do a failed
pop3 auth myself, I show as Disconnected but the dictionary attack the
other day showed as Aborted login.
Rodman
- Original Message -
From: Charles Marcus cmar...@media-brokers.com
Cc: dovecot@dovecot.org
Sent: Friday, June 26, 2009 8:57 AM
Subject: Re: [Dovecot] Lots of pop3-logins
On 6/26/2009, Rodman Frowert (rod...@thefrowerts.com) wrote:
If anyone wants to see the fail2ban config file I am using for Dovecot,
let me know...
Does it also work for IMAP ligins? I'd like to see it regardless...
thanks!
--
Best regards,
Charles
[Dovecot] Lots of pop3-logins
Hello, Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Thanks! Rodman
Re: [Dovecot] Lots of pop3-logins
On Qui, 2009-06-25 at 09:07 -0500, Rodman Frowert wrote: Hello, Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. -- Jose Celestino SAPO.pt::Systems http://www.sapo.pt - * Progress (n.): The process through which Usenet has evolved from smart people in front of dumb terminals to dumb people in front of smart terminals.
Re: [Dovecot] Lots of pop3-logins
Jose, Thank you for your reply. Makes me feel better everything is working properly and resources aren't being wasted. Thank you! Rodman - Original Message - From: Jose Celestino j...@co.sapo.pt To: Rodman Frowert rod...@thefrowerts.com Cc: dovecot@dovecot.org Sent: Thursday, June 25, 2009 9:34 AM Subject: Re: [Dovecot] Lots of pop3-logins On Qui, 2009-06-25 at 09:07 -0500, Rodman Frowert wrote: Hello, Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. -- Jose Celestino SAPO.pt::Systems http://www.sapo.pt - * Progress (n.): The process through which Usenet has evolved from smart people in front of dumb terminals to dumb people in front of smart terminals.
Re: [Dovecot] Lots of pop3-logins
Hello, Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. There is definitely some issue. Once the load increases the system will start timing out on POP3 connections or other network connections, such as IMAP, SSH etc. Better check out the system logs, utilization etc. for any abnormal values. Regards Rao
Re: [Dovecot] Lots of pop3-logins
On Qui, 2009-06-25 at 10:01 -0700, V S Rao wrote: Hello, Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. -- Jose Celestino SAPO.pt::Systems http://www.sapo.pt - * Progress (n.): The process through which Usenet has evolved from smart people in front of dumb terminals to dumb people in front of smart terminals.
Re: [Dovecot] Lots of pop3-logins
On Jun 25, 2009, at 10:07 AM, Rodman Frowert wrote: Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Take a look at your log file. Is there a dictionary attack taking place? I get this all the time. I want to find these little cracker kiddies and break their fingers. -Dave -- Dave McGuire Port Charlotte, FL
Re: [Dovecot] Lots of pop3-logins
Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. Regards --Rao
Re: [Dovecot] Lots of pop3-logins
You can also just decrease login_process_max_count. If Dovecot reaches the limit, it'll just start killing off old connections that haven't logged in. And yeah, some day I should also make Dovecot kill some of the login processes after many of them have been idling for a while. On Thu, 2009-06-25 at 14:33 -0500, Rodman Frowert wrote: Well, after going through my log files, I was hit with a dictionary based attack. My maillog is full of about 20,000 lines of crap like this: Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=warren, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=williams, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=www, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:05 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=wilson, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:05 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=willy, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:05 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=valerie, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Starts with A and runs all the way to Z. The IP traces back to cable modem subscriber on Cox Communications out of Arizona. I'll shoot them off my standard attack e-mail. In the meantime, I need to modify fail2ban so that it checks the maillog for failed pop3 auth logins and bans IP's so this won't happen again. Rodman - Original Message - From: V S Rao viriy...@yahoo.com To: dovecot@dovecot.org Sent: Thursday, June 25, 2009 1:15 PM Subject: Re: [Dovecot] Lots of pop3-logins Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. Regards --Rao signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Lots of pop3-logins
I'll go ahead and lower that limit to something that fits my usage better. Thanks Timo! You built a hell of a mail server. Rodman - Original Message - From: Timo Sirainen t...@iki.fi To: Rodman Frowert rod...@thefrowerts.com Cc: dovecot@dovecot.org Sent: Thursday, June 25, 2009 2:46 PM Subject: Re: [Dovecot] Lots of pop3-logins
Re: [Dovecot] Lots of pop3-logins
On Jun 25, 2009, at 3:46 PM, Timo Sirainen wrote: You can also just decrease login_process_max_count. If Dovecot reaches the limit, it'll just start killing off old connections that haven't logged in. I don't see this option in my dovecot.conf. Was it added after 1.1.6? -Dave -- Dave McGuire Port Charlotte, FL
Re: [Dovecot] Lots of pop3-logins
On Qui, 2009-06-25 at 11:15 -0700, V S Rao wrote: Doing a ps aux on my Slackware box, I have approx 100 PID's of pop3-login's going on. This is a production mail server, but it is getting VERY low traffic. In fact, only 3 people can pop3 into it. I've check their e-mail clients, and they are not checking mail any more often than every 5 minutes. This is a new installation and I've had the server up and running since Sunday. If it matters, I'm using Postfix for the MTA and using the Dovecot SASL library to AUTH SMTP. Is this a cause for concern? Why does Dovecot need this many processes? Because dovecot preforks the *-login processes to speed-up the login. No need to worry. 100 login sessions for just 3 connections? That is not right, no matter what. No, login_processes_count matters. How? If my understanding is correct, you have extra 3 login processes created to cater to new connections. So with only 3 POP3 users, why should so many login processes be spawned? I can understand 10-15. But 100 definitely indicates either the processes are not dying or something else happening on the system which is causing such high number of login processes. The system definitely needs to be checked for some kind of attack, a rogue process running on the system or something else. If you don't change the defaults that's right. But the *-login processes will never be less than login_processes_count so it does matter. And, as timo pointed out, you can put a upper limit with login_max_processes_count. My idle box has 64 imap-login processes and no, I'm not under a dictionary attack :) -- Jose Celestino SAPO.pt::Systems http://www.sapo.pt - * Progress (n.): The process through which Usenet has evolved from smart people in front of dumb terminals to dumb people in front of smart terminals.
Re: [Dovecot] Lots of pop3-logins
On Thu, 2009-06-25 at 15:46 -0400, Timo Sirainen wrote: You can also just decrease login_process_max_count. If Dovecot reaches the limit, it'll just start killing off old connections that haven't logged in. What would be nice is, an anti brute force option, like xinetd, X-number of connections from Y i.p. in Z seconds (optional setting of course) or maybe a way to extend that to detect if the same i.p is retrying constantly using different usernames on every new connection within X seconds, come to think of it, that way would be much cooler :) Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=warren, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=williams, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2 Jun 21 23:06:04 mail dovecot: pop3-login: Aborted login (auth failed, 1 attempts): user=www, method=PLAIN, rip=68.14.228.186, lip=10.10.11.2
Re: [Dovecot] Lots of pop3-logins
--On Friday, June 26, 2009 8:48 AM +1000 Noel Butler noel.but...@ausics.net wrote: What would be nice is, an anti brute force option, like xinetd, X-number of connections from Y i.p. in Z seconds (optional setting of course) or maybe a way to extend that to detect if the same i.p is retrying constantly using different usernames on every new connection within X seconds, come to think of it, that way would be much cooler :) Some good discussion about fighting dictionary attacks here: http://www.codinghorror.com/blog/archives/001206.html
Re: [Dovecot] Lots of pop3-logins
On Fri, 2009-06-26 at 07:48 +1000, Noel Butler wrote: What would be nice is, an anti brute force option, like xinetd, X-number of connections from Y i.p. in Z seconds (optional setting of course) or maybe a way to extend that to detect if the same i.p is retrying constantly using different usernames on every new connection within X seconds, come to think of it, that way would be much cooler :) v2.0 makes it possible in a lot easier way. Maybe I'll get it implemented there. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Lots of pop3-logins
On Thu, 2009-06-25 at 18:31 -0400, Timo Sirainen wrote: On Fri, 2009-06-26 at 07:48 +1000, Noel Butler wrote: What would be nice is, an anti brute force option, like xinetd, X-number of connections from Y i.p. in Z seconds (optional setting of course) or maybe a way to extend that to detect if the same i.p is retrying constantly using different usernames on every new connection within X seconds, come to think of it, that way would be much cooler :) v2.0 makes it possible in a lot easier way. Maybe I'll get it implemented there. That would be awesome :) Cheers
