I've been experimenting with this modsecurity_crs_11_dos_protection.conf ruleset more. An outline of how it seems to work:
- Every non-image request increments the ip.dos_counter counter. - If that counter is > tx.dos_counter_threshold, it increments ip.dos_burst_counter, sets an expiration time on ip.dos_burst_counter, and clears ip.dos_counter - If ip.dos_burst_counter > ip.dos_burst_threshold, it blocks the request until tx.dos_block_timeout seconds elapses The rational behind the burst counter seems to be because mod_security expiration doesn't work based on the FIRST write to a variable, it works based on the most recent write. In my case, I'm using a time slice of 60 seconds, but if a bot requests a page every 10 seconds for hours it will never expire ip_dos_counter, the only time that will happen is if they don't do any requests for 60 seconds. So, this burst counter is designed to try to get around this problem by waiting until the dos_counter is over the threshold (which doesn't really prove anything about how fast requests were being made, only that the user has done a bunch of requests without pausing) and then it increments the burst counter which has an expiration time (in my case, 60 seconds) and clears out the dos_counter. So, if the dos counter gets up to the threshold AGAIN, then it knows that those requests really all have come in the past 60 seconds, so it knows that the user is requesting too quickly. Great in theory, but this is broken. The problem scenario: - A bot (bingbot in my case) has been requesting pages for the past hour. Averaging one or two requests per second. Nothing to worry about. - The dos_counter is over the threshold because it has been submitting requests for a long time without pausing - Sometimes it submits 2-4 requests virtually simultaneously (on the same second in the log) - It submits 2 or more requests simultaneously. For EACH REQUEST, mod_security sees that ip_dos_counter is over the threshold, so each request increments ip_dos_burst_counter. bingbot is now blocked. So this is an example where the bot was blocked even though it was operating at a far slower rate than the DoS settings specify. If those 2 requests would have been spaced apart by a second, it would not have been blocked. But, by submitting requests very close together, a bot can be blocked without really submitting that many requests very quickly. This is happening on my server at least every day. I am new to mod_security so I'm still learning exactly how these rules work, but are there any solutions to this problem? The way this rule is set up, it seems to be operating on the principle that the rule is atomic when it clearly isn't. If dos_counter would be cleared before the burst counter is incremented, this problem wouldn't exist. I'm also unclear on how this is coordinated across the multiple apache processes. I understand this to be a collection, which resides on disk, so I'm not sure when mod_security writes this information to disk. Does anyone have any pointers for where to look, or a different DoS ruleset to try? Or is DoS protection not something that is possible with mod_security, and I need another module? I don't need anything fancy, I just want to be able to limit an IP address to a certain number of requests within a time period. Thanks, Nick On Mon, Apr 29, 2013 at 12:49 PM, Nick <darknovan...@gmail.com> wrote: > Hello, > > I've been setting up mod_security and enabled > the modsecurity_crs_11_dos_protection.conf rule. This is mod_security 2.6.8 > and CRS version 2.2.5. > > I have initialized the settings with: > SecAction \ > "id:'900015', \ > phase:1, \ > t:none, \ > setvar:'tx.dos_burst_time_slice=60', \ > setvar:'tx.dos_counter_threshold=300', \ > setvar:'tx.dos_block_timeout=600', \ > nolog, \ > pass" > > This works and it is blocking some very aggressive bots the way it should. > But there is a problem. I have occasionally been getting lines like this in > the log: > > Warning. Operator GE matched 2 at IP:dos_burst_counter. [file > "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_11_dos_protection.conf"] > [line "44"] [id "981049"] [msg "Potential Denial of Service (DoS) Attack > from 65.55.24.236 - # of Request Bursts: 3"] > > This bot was actually bingbot. I am new to mod_security, but my > understanding of my settings is that it shouldn't block until a bot has > requested 300 pages in 60 seconds. > > When I check the logs I see that IP 65.55.24.236 has requested 313 pages > in 1 hour. In the 60 seconds before the DoS block happening, this IP only > requested 6 pages. This block obviously shouldn't be happening. > > I am grossly misunderstanding something, or what can I do to fix this? > Thanks, > > Nick > >
_______________________________________________ Owasp-modsecurity-core-rule-set mailing list Owasp-modsecurity-core-rule-set@lists.owasp.org https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
