Re: [CentOS] which firewall to automatically block bandwidth abusers?
From: Rudi Ahlers r...@softdux.com the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills. Hum, if it is www traffic, maybe put squid as a reverse proxy and use delay pools? JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 18/08/2011 4:13, Craig White wrote: On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote: Hi, I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly). But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people. Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits. Does this make sense? Can this be done with iptables? If so, how? If not, what else could I use for this? A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well. The servers behind the firewall will serve mail, http, ftp, sql and SSH http://tinyurl.com/3n5yn8u Would you mind providing the url without using such url shorteners? Thanks, Regards ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 4:13 AM, Craig White craigwh...@azapple.com wrote: On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote: Hi, I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly). But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people. Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits. Does this make sense? Can this be done with iptables? If so, how? If not, what else could I use for this? A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well. The servers behind the firewall will serve mail, http, ftp, sql and SSH http://tinyurl.com/3n5yn8u Craig We already monitor traffic usage on the switches with cacti via SNMP. But, I need to block traffic abusers automatically. from any IP address, to any IP address. The firewalls we have, and have tested all need a set of IP addresses to throttle, which won't work in this case. A user can login from any IP address on the internet, and either upload or download exsesively and we need to block that IP address as soon as it's reaches a certain (pre-set by us) threshold -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 2011-08-18 at 19:20 +0200, Patrick Lists wrote: Lmgtfy means let me google that for you. Posting such an url is a pretty standard response to people who ask for help without first making an effort to find some answers (by googling, etc.). The hint is: do your homework first and don't expect spoonfeeding. Thanks Patrick. I do do my own research first, usually via Google or my own technical web pages. I usually get good answers most of the time. -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. My research: The firewalls which we've tried (both normal Linux iptables and hardware based firewalls) can do this, as long as I can specify the IP's to block - this is standard for an office-type firewall. BUT, I don't have a range of IP's to specify since these particular servers are on the internet, thus any possible IP on the net could connect to the server. I also need to exclude certain IP's from this rule (i.e. for backup servers which actually need to transfer a lot of traffic). To some degree this would mean traffic accounting, but that just keeps a log of traffic usage. And we already measure traffic use with cacti SNMP. Cacti can send us an email if a certain amount of bandwidth is used up, but it doesn't tell the firewall to block the offending IP address. DDOS protection type firewalls doesn't help much either since they only block incoming attacks, but not really normal uploads. They also don't block outgoing traffic once the condition is met. -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. First question is: (a) how can you get the IP address ? (b) how can you introduce a, or use an existing, system to record and store the data amounts (bandwidth) and IP addresses ? (c) how long will this information be retained before being discarded ? (d) how can you monitor on every change to the data amount ? (e) will it do both IP4 and IP6 ? (f) what mechanism can you use to block the IP address ... IP Tables via simple BASH command ? Its an interesting requirement. -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:09 PM, Always Learning cen...@u61.u22.net wrote: On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. First question is: (a) how can you get the IP address ? I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's, /var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well. We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP. (b) how can you introduce a, or use an existing, system to record and store the data amounts (bandwidth) and IP addresses ? What do you mean? (c) how long will this information be retained before being discarded ? How long will what information be retained? And what for? I don't understand the nature of this question? (d) how can you monitor on every change to the data amount ? Again, I don't understand what you mean? (e) will it do both IP4 and IP6 ? Does it matter? IPV6 is already being used on a wide scale. iptables support both (f) what mechanism can you use to block the IP address ... IP Tables via simple BASH command ? if that will do the trick, yes. Any way to block the IP would be fine. iptables would probably be easiest. Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers Its an interesting requirement. -- With best regards, Paul. England, EU. -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 8/18/2011 2:01 PM, Rudi Ahlers wrote: Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only? -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
If there isn't an existing system, or systems you can use together, your only alternative is to create a system to satisfy your requirement. I was speculating on the essentials. -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 18 Aug 2011, Rudi Ahlers wrote: Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. As one might imagine there is at least one commercial product that seems to fit the bill. http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-the-User-based-approach.pdf I mention this as I thought it was well written and thorough. After reading the pdf seems to me there ought to be something open source based upon perhaps this: http://lartc.org/lartc.html Anyway maybe some food for thought. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:21 PM, Les Mikesell lesmikes...@gmail.com wrote: On 8/18/2011 2:01 PM, Rudi Ahlers wrote: Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only? -- Les Mikesell lesmikes...@gmail.com ___ Les, it's not really about blocking people who paid. the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills. I know the requirements are strange, but I'm really hoping I could find something that could do this for us. Right now they have someone who monitors ntop and block IP's that way around, but it's inefficient and a salary which could have been spent elsewhere. Bandwidth in our country is exuberantly expensive, probably about 20x the price of bandwidth in the USA -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 8/18/2011 2:15 PM, Rudi Ahlers wrote: On Thu, Aug 18, 2011 at 9:09 PM, Always Learningcen...@u61.u22.net wrote: On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. First question is: (a) how can you get the IP address ? I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's, You said 'user' which may or may not map to a consistent, single, IP address. /var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well. Are these web browser clients, locally attached PCs, or what? We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP. What's 'abusive'? If they are using a web app, let the app monitor the connection of a logged in user and handle them appropriately. Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers A separate box won't know what is going on. Suppose you have a remote mail server relaying in or out for a large number of users. The intermediate box will see a lot of smtp traffic to/from one IP, but it will correspond to a lot of users. Likewise for web users behind a company proxy. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:25 PM, Mike m...@microdel.org wrote: On Thu, 18 Aug 2011, Rudi Ahlers wrote: Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. As one might imagine there is at least one commercial product that seems to fit the bill. http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-the-User-based-approach.pdf I mention this as I thought it was well written and thorough. After reading the pdf seems to me there ought to be something open source based upon perhaps this: http://lartc.org/lartc.html Anyway maybe some food for thought. ___ Thanx. We already tried the cyberoams, but they didn't work as expected since they manage bandwidth on a per-user basis, and our users come from the world-wide-web. I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/18/2011 08:45 PM, Rudi Ahlers wrote: And you obviously think I didn't do my homework? Did you see my specific requirement? Or did you just see how and firewall and assumed google ? I was not referring to you Rudi. Merely pointing out the lmgtfy concept which imho seemed lost on Paul. And yes I did look at your requirements but don't have the answer for you. Maybe a combination of iptables and tc perhaps with connection tracking thrown in? Regards, Patrick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:29 PM, Les Mikesell lesmikes...@gmail.com wrote: On 8/18/2011 2:15 PM, Rudi Ahlers wrote: On Thu, Aug 18, 2011 at 9:09 PM, Always Learningcen...@u61.u22.net wrote: On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. First question is: (a) how can you get the IP address ? I don't fully understand your question? How do you get any IP address from any machine that connects to a server on the internet? netstat shows the IP's, You said 'user' which may or may not map to a consistent, single, IP address. well, a 'user' is anyone accessing the server from the internet, so the IP's will change the whole time. /var/log/http/access.log shows the IP's and I'm sure it's listed in other places as well. Are these web browser clients, locally attached PCs, or what? web / SQL / SMTP / POP3 clients, connecting from the internet. We currently use ntop to monitor the server's usage, but there's no way to automatically block an abusive IP. What's 'abusive'? If they are using a web app, let the app monitor the connection of a logged in user and handle them appropriately. yes, but no monitor can block their IP, that I'm aware of. Ideally I would like to get a dedicated firewall, or dedicated Linux / UNIX firewall appliance for this purpose as it needs to monitor and protect a whole bunch of servers A separate box won't know what is going on. Suppose you have a remote mail server relaying in or out for a large number of users. The intermediate box will see a lot of smtp traffic to/from one IP, but it will correspond to a lot of users. Likewise for web users behind a company proxy. For this very reason I need to exclude certain IP's from the limits. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 8/18/2011 2:27 PM, Rudi Ahlers wrote: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingresoutgres) for a specific amount of time. Those requirements don't mesh very well with the real world. That is, people use use a network that they've been provided or paid for aren't necessarily 'abusing' anything, and blocking access at times when the network isn't fully loaded doesn't help anyone. What's the big picture here? Don't you really need QOS to throttle certain things at peak times only? Les, it's not really about blocking people who paid. the servers in question provide a free service and no money is generated from it, but the client still pays for bandwidth so we'd like to cap heavy users a bit to avoid expensive bills. Are you paying for bandwidth by total bits transferred or by peak or 95th percentile rate? I know the requirements are strange, but I'm really hoping I could find something that could do this for us. Right now they have someone who monitors ntop and block IP's that way around, but it's inefficient and a salary which could have been spent elsewhere. You should be able to automate what you are doing with ntop. Or use a netflow collector to centralize the traffic counting and translate your rules into iptables settings. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:38 PM, Mike m...@microdel.org wrote: I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient? How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range. So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit? -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:38 PM, Les Mikesell lesmikes...@gmail.com wrote: Are you paying for bandwidth by total bits transferred or by peak or 95th percentile rate? We pay per MB and the servers are connected to a 100MB/s port. You should be able to automate what you are doing with ntop. Or use a netflow collector to centralize the traffic counting and translate your rules into iptables settings. Really? That would be great. But, I'm not a programmer, so I don't know where to start. And, I need to protect a whole bunch of servers, so ideally this should be done either on a central gateway which connects on the other side of the switch, or a firewall appliance. Any suggestions? -- Les Mikesell lesmikes...@gmail.com -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/18/2011 09:31 PM, Rudi Ahlers wrote: [snip] I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. How about the netfilter quota, fuzzy and iplimit extensions? http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO.html#toc3.4 http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO.html#toc3.5 http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.html#ss3.13 Regards, Patrick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 18 Aug 2011, Rudi Ahlers wrote: On Thu, Aug 18, 2011 at 9:38 PM, Mike m...@microdel.org wrote: I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient? How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range. So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit? I think I understand now and the short answer is that you can't! In other words you're saying that say Steve is using a ton of bandwidth so you want to block him. But Fred and 10 other users that may be at the same IP address are fine and you don't want to block them. I mean you could conceptually at least block the IP/Source port that Steve is coming from right now. But the source port (and perhaps IP) will eventually change and your block is now useless. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/18/11 12:43 PM, Rudi Ahlers wrote: But, I'm not a programmer, so I don't know where to start. hire one. your needs and requirements are vague and unique, no off the shelf solution will do exactly what it is you want. you also need to start thinking of your requirements in more precise terms, what the thresholds of traffic that will trigger and reset these blocks or throttles. you probably want to tie this into QoS so that when your algorithm determines that a specific host is over its threshold, you throttle it rather than block it entirely. messy messy messy. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, Aug 18, 2011 at 9:52 PM, Mike m...@microdel.org wrote: On Thu, 18 Aug 2011, Rudi Ahlers wrote: On Thu, Aug 18, 2011 at 9:38 PM, Mike m...@microdel.org wrote: I have read through that document link on http://lartc.org/lartc.html#AEN1393 and the closest I could get is rate limiting, but that doesn't actually block the IP if it goes over a certain threshold, it just slows everything down. So I'm not sure I fully understand your requirements. Why isn't slowing the user to zero or at least near zero sufficient? How do I slow one user down, without affecting the others? The way I understand rate limiting is that you rate limit a certain protocol / port, or IP / IP range. So, how would I automatically slow down someone (on any IP address, and accessing any protocol) once he hits a certain threshold / limit? I think I understand now and the short answer is that you can't! In other words you're saying that say Steve is using a ton of bandwidth so you want to block him. But Fred and 10 other users that may be at the same IP address are fine and you don't want to block them. I mean you could conceptually at least block the IP/Source port that Steve is coming from right now. But the source port (and perhaps IP) will eventually change and your block is now useless. ___ No, not quite. Steve will have a different IP from Fred. I don't care so much about the users as such, but rather the IP where the connection is from, and to. i.e. I don't need to know what the user's name is, nor match him to a DB like LDAP or something. I purely need to block an abusive IP. BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well. -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/18/11 12:56 PM, Rudi Ahlers wrote: BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well. how would you know this? -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 8/18/2011 4:38 PM, John R Pierce wrote: On 08/18/11 12:56 PM, Rudi Ahlers wrote: BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well. how would you know this? If he is using pop, imap, authenticated smtp, web services with a logged in session, ssh, etc., the applications know the user and may be logging it. But there is nothing central or standard to collate this information, and there are various circumstances that will cause many users to have the same IP source or one user to have several. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 2011-08-18 at 21:33 +0200, Patrick Lists wrote: And yes I did look at your requirements but don't have the answer for you. Maybe a combination of iptables and tc perhaps with connection tracking thrown in? IP tables would be a good place to link-in; perhaps route requests to a specific port or internal IP address and then examine the traffic before routing it to the correct destination. -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 2011-08-18 at 21:27 +0200, Rudi Ahlers wrote: Bandwidth in our country is exuberantly expensive, probably about 20x the price of bandwidth in the USA Een oplossing voor Zuid Afrika ? If your country has good internal Internet connections, host the site in Europe or the USA where bandwidth is a lot cheaper ? -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
Apologies for top posting. I fear you will either have to work with cacti bandwidth alerts, figuring out how to grab the client IP and push it into iptables; find another way to get the client IP out of cacti and into iptables; or look into the QoS capabilities within Linux. On 08/18/2011 03:01 PM, Rudi Ahlers wrote: Let's try again: I need to automatically block any user who abuses bandwidth, either incoming or outgoing. I should be able to set the limits, in either rate/s or usage/s: 1Mb/s or 10GB/h, for example. Then, any users, connecting from anywhere, on any IP should be blocked - either if he uploads or downloads (i.e ingres outgres) for a specific amount of time. My research: The firewalls which we've tried (both normal Linux iptables and hardware based firewalls) can do this, as long as I can specify the IP's to block - this is standard for an office-type firewall. BUT, I don't have a range of IP's to specify since these particular servers are on the internet, thus any possible IP on the net could connect to the server. I also need to exclude certain IP's from this rule (i.e. for backup servers which actually need to transfer a lot of traffic). To some degree this would mean traffic accounting, but that just keeps a log of traffic usage. And we already measure traffic use with cacti SNMP. Cacti can send us an email if a certain amount of bandwidth is used up, but it doesn't tell the firewall to block the offending IP address. DDOS protection type firewalls doesn't help much either since they only block incoming attacks, but not really normal uploads. They also don't block outgoing traffic once the condition is met. -- -- John Jasen (jja...@realityfailure.org) -- No one will sorrow for me when I die, because those who would -- are dead already. -- Lan Mandragoran, The Wheel of Time, New Spring ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Thu, 2011-08-18 at 21:56 +0200, Rudi Ahlers wrote: BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well. How will you know Steve has successfully circumvented your block until until the same Steve, with IP2, eventually exceeds the 'quota' ? And if Steve gets away with that, he can probably try again with IP3 and IP4 etc. - making a mockery of your bandwidth restriction. -- With best regards, Paul. England, EU. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Fri, Aug 19, 2011 at 12:57 AM, Always Learning cen...@u61.u22.net wrote: On Thu, 2011-08-18 at 21:56 +0200, Rudi Ahlers wrote: BUT, if Steve changes his IP to circumvent the block, then his new IP should be blocked as well. How will you know Steve has successfully circumvented your block until until the same Steve, with IP2, eventually exceeds the 'quota' ? And if Steve gets away with that, he can probably try again with IP3 and IP4 etc. - making a mockery of your bandwidth restriction. -- The point it, it doesn't matter who the user is. As soon as an IP, any IP exceeds the limit, it should get blocked. -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/18/11 4:05 PM, Rudi Ahlers wrote: The point it, it doesn't matter who the user is. As soon as an IP, any IP exceeds the limit, it should get blocked. you might take a look at the various fail2ban scripts that are commonly used to block an IP for some period of time after a threshold number of SSH or appache login attempts are made, and you can probably figure out how to implement that same sort of concept to run off whatever per-source-IP traffic statistics you're keeping... of course, if your web and mail and whatever servers are accessed by 100s or 1000s of unique hosts a day, those traffic statistics are going to be quite a lot of overhead to track. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Aug 17, 2011, at 3:50 PM, Rudi Ahlers r...@softdux.com wrote: Hi, I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly). But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people. Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits. Does this make sense? Can this be done with iptables? If so, how? If not, what else could I use for this? A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well. The servers behind the firewall will serve mail, http, ftp, sql and SSH Best approach, throttle, you can cause the throttle to increase as the overage increases until it reaches dial-up speed. With some cleverness you can back the throttle out after a period of idle-ness. -Ross ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On 08/17/11 12:50 PM, Rudi Ahlers wrote: A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well. The servers behind the firewall will serve mail, http, ftp, sql and SSH without requests coming in, no web etc traffic can go out. you want to block your own mail server from sending too much mail to a single host?and block an internet mail server from sending too much mail to you? thats not going to end well. SQL? what are you doing letting a SQL server be publically accessible? SQL servers should only be accessed by application servers over secure connections. I think as it stands, this is a very poorly thought out idea with much room for gotchas and problems. -- john r pierceN 37, W 122 santa cruz ca mid-left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] which firewall to automatically block bandwidth abusers?
On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote: Hi, I'm looking for a firewall (preferably on Linux / UNIX) that could automatically block bandwidth abusers as soon as a connection goes over a certain speed, or limit - i.e. either more than say 3Mb/s or 10GB in a giving period (like weekly / monthly). But, I need it to block the IP to, or where the traffic comes from, or goes to. i.e. a user logs into a web server and upload a LOT of data, then the firewall should block him, but not other people. Or, someone uploads a small bit of data but downloads a lot of data and then get's blocked. But I need to set thresholds And I should be able to exclude certain IP's / domains from the limits. Does this make sense? Can this be done with iptables? If so, how? If not, what else could I use for this? A normal DDOS prevention firewall doesn't really work since it only blocks traffic coming in. But I need to limit traffic going out as well. The servers behind the firewall will serve mail, http, ftp, sql and SSH http://tinyurl.com/3n5yn8u Craig -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
