Christian, Where do I set the anomaly limit?
On Wed, May 18, 2016 at 6:51 AM, Chaim Sanders <csand...@trustwave.com> wrote: > In addition to Barry’s fantastic links I’ll direct you to this post > written by Ryan Barnett that details how to add exceptions in an effective > method. There is some tuning required as mentioned before but after about a > week these tend to go away pretty quickly and just rarely rear their head. > Check it out > https://www.trustwave.com/Resources/SpiderLabs-Blog/ModSecurity-Advanced-Topic-of-the-Week--(Updated)-Exception-Handling/. > If you need additional help please do reach out J > > > > *Chaim Sanders * > > Security Researcher > > *Trustwave* | SMART SECURITY ON DEMAND > > www.trustwave.com > > > > *From:* owasp-modsecurity-core-rule-set-boun...@lists.owasp.org [mailto: > owasp-modsecurity-core-rule-set-boun...@lists.owasp.org] *On Behalf Of *T. > Kenneth Lojo (IRRI) > *Sent:* Tuesday, May 17, 2016 3:51 AM > *To:* Barry Pollard <barry_poll...@hotmail.com> > *Cc:* owasp-modsecurity-core-rule-set@lists.owasp.org > *Subject:* Re: [Owasp-modsecurity-core-rule-set] Facebook scrape problem > > > > Thank you for this Barry. We actually have attacks going on on the server > and so far mod_sec has blocked it with the CRS. Our server with wordpress > is being brute forced by a xmlrpc.php attack and our joomla server is being > flooded by a torrent request when we don't have any torrents running. I may > need to do both in parallel. Do the anomaly scoring to allow facebook > scraping the content and probably setup another server and do your > recommendation to better tweak the WAF and then apply to the production. > > > > On Tue, May 17, 2016 at 3:05 PM, Barry Pollard <barry_poll...@hotmail.com> > wrote: > > I would say first thing is to turn blocking off and run in DetectionOnly > mode to help you fine tune your rules. To do that update your SecRuleEngine > config like so: > > > > SecRuleEngine DetectionOnly > > This will of course mean you are not protected but us a necessary step to > getting your set up right. Now leave it run for a while and then check the > logs for every rule that fired (but did not block this time) and categories > them into: > > > > 1) False positive - this request looks to be the sort of request our > application expects and ModSecurity should not be alerting on it. > > 2) Bad request - this request shouldn't be made and ModSecurity was right > to block it. This will include bots and scripts that scan websites even if > they don't cause any trouble. > > > > For all the 1s (and there will be a lot at the beginning) you need to > decide how to tweak the rules to not alert for False positives. This > involves either turning the rule off completely (using the likes of > SecRuleRemoveById), turning it off for particular parameters (using the > likes of SecRuleUpdateActionById) or turning it off for particular URLs > (this is more complicated aged and can require building a new rule to do > this). > > > > Tuning rules is necessary and, as long as you have a good understanding of > what the rules intention is, why it blocked, why that was incorrect thing > for it to do then you should tweak and turn them off for certain scenarios. > Does that reduce the effectiveness of ModSecurity? Potentially but then if > it blocks all sorts of real visitors incorrectly then that's not much use > is it! The OWASP CRS is very generic and all the rules will not be > appropriate for all websites. By default it blocks too much. There is some > work going on to make the default more lenient so you can start off with > some protection and ramp up as you see fit rather than current situation > where you start with too much protection and have to ramp down. > > > > Anyway after you see no more false positives for a while you can > turn SecRuleEngine back to on. > > > > This can take some time. See my story here to prepare you: > http://stackoverflow.com/questions/35149264/how-long-do-you-fine-tune-false-positives-with-mod-security-and-owasp-rules/35162976#35162976 > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jfp1DhFSw&s=5&u=http%3a%2f%2fstackoverflow%2ecom%2fquestions%2f35149264%2fhow-long-do-you-fine-tune-false-positives-with-mod-security-and-owasp-rules%2f35162976%2335162976>. > A WAF like ModSecurity is, unfortunately, not just a turn it on and it > works and you can forget about it solution. It takes a lot of set up to be > useful, and then a bit if minding afterwards. Personally I think it's worth > it but you also see people online saying WAFs are too much effort for this > reason. > > > > Anomaly scoring mode is an interesting one. It basically let's all the > rules run and then only blocks if a certain threshold applies. This means a > number of unimportant rules can fire. e.g. most browsers send a user agent > so no user agent, while it likely won't cause a problem is a flag that this > is probably a bad request. If a few of these flags fire on same request > then this is highly likely to be a bad request and should be blocked. Some > rules (like missing user agent) might have low threshold and so won't block > on their own and some will block with just one rule firing if it's obvious > this request should not processed. > > > > While anomaly scoring is undoubtedly helpful to reduce the number of > incorrect blocks, and lots of people use it and recommend it, I'm not a > particular fan. I find it makes the log files noisy and confusing to see > rules firing and not know if they caused a block or not. I prefer to turn > off the low value rules completely and using the original block at first > bad attempt mode despite the fact this takes extra work to set up initially > and can allow more spam and bad bots through. But each to their own. Will > leave Christian to explain how best to set it up if that's the way you want > to go. > > > > Here's some other posts that might help: > > > http://stackoverflow.com/questions/33676348/extra-sensitive-mod-security-rules-giving-403-forbidden-error > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44mTj0m1ERQ&s=5&u=http%3a%2f%2fstackoverflow%2ecom%2fquestions%2f33676348%2fextra-sensitive-mod-security-rules-giving-403-forbidden-error> > > > http://stackoverflow.com/questions/34478019/keep-modsecurity-enabled-with-symfony-installation-w-cpanel-whm/34484463#34484463 > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44j6x0GVFTg&s=5&u=http%3a%2f%2fstackoverflow%2ecom%2fquestions%2f34478019%2fkeep-modsecurity-enabled-with-symfony-installation-w-cpanel-whm%2f34484463%2334484463> > > > http://stackoverflow.com/questions/33989273/modsecurity-excessive-false-positives/34027786#34027786 > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jGz12xDTw&s=5&u=http%3a%2f%2fstackoverflow%2ecom%2fquestions%2f33989273%2fmodsecurity-excessive-false-positives%2f34027786%2334027786> > > Note this mailing list is awesome and you will get help here but I have > also been answering ModSecurity questions on StackOverflow/ServerFault as > feel they are better to reference again for common questions like yours. > Been meaning to write a friendly, short, beginners containing a lot of the > detail here but have a problem keeping my posts short :-) > > > > Hope that helps and feel free to ask any questions here. We're a friendly > bunch. > > > > Thanks, > > Barry > > > > > > -- > > *T. Kenneth S. Lojo* > Specialist-Online Media Design > > [image: Image removed by sender. IRRI] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jay1mwTSg&s=5&u=http%3a%2f%2firri%2eorg%2f> > > +63 2 580 5600 ext. 2703/2744 > +63 928 209 1191 (mobile) > t.l...@irri.org <g.lav...@irri.org> > www.irri.org > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jPnhj0STA&s=5&u=http%3a%2f%2fwww%2eirri%2eorg> > > [image: Image removed by sender. Facebook] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44mTggTpFSg&s=5&u=http%3a%2f%2fwww%2efacebook%2ecom%2fIRRI%2ericenews> > [image: Image removed by sender. Twitter] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44mC2hjoRGQ&s=5&u=http%3a%2f%2ftwitter%2ecom%2fRiceResearch> > [image: Image removed by sender. Flickr] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jLn3D4WHA&s=5&u=http%3a%2f%2fwww%2eflickr%2ecom%2fphotos%2fricephotos%2fcollections%2f> > [image: Image removed by sender. Youtube] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jC2gD0TRA&s=5&u=http%3a%2f%2fwww%2eyoutube%2ecom%2fuser%2firrivideo%2ffeatured> > [image: Image removed by sender. Scribd] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44mLig21AGA&s=5&u=http%3a%2f%2fwww%2escribd%2ecom%2fIRRI%5fresources> > [image: Image removed by sender. Linkedin] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44j_p1GQQSQ&s=5&u=http%3a%2f%2fwww%2elinkedin%2ecom%2fcompany%2finternational-rice-research-institute> > [image: Image removed by sender. Soundcloud] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jPmhmwSHw&s=5&u=https%3a%2f%2fsoundcloud%2ecom%2firri-radio> > [image: Image removed by sender. Google+] > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jLo1TkWGQ&s=5&u=https%3a%2f%2fplus%2egoogle%2ecom%2f103972671963502739315> > > The International Rice Research Institute > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jXghm0XGA&s=5&u=http%3a%2f%2firri%2eorg> > is > a member of the CGIAR > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44j_j1m4UGQ&s=5&u=http%3a%2f%2fwww%2ecgiar%2eorg%2f> > > > The International Rice Research Institute > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44jXghm0XGA&s=5&u=http%3a%2f%2firri%2eorg> > is > a member of the CGIAR > <http://scanmail.trustwave.com/?c=4062&d=otS611Q20SL86MvZAP2-57BForsTwwP44mOxgW4WSQ&s=5&u=http%3a%2f%2fcgiar%2eorg> > consortium > > ------------------------------ > > This transmission may contain information that is privileged, > confidential, and/or exempt from disclosure under applicable law. If you > are not the intended recipient, you are hereby notified that any > disclosure, copying, distribution, or use of the information contained > herein (including any reliance thereon) is strictly prohibited. If you > received this transmission in error, please immediately contact the sender > and destroy the material in its entirety, whether in electronic or hard > copy format. > -- *T. Kenneth S. Lojo* Specialist-Online Media Design [image: IRRI] <http://irri.org/> +63 2 580 5600 ext. 2703/2744 +63 928 209 1191 (mobile) t.l...@irri.org <g.lav...@irri.org> www.irri.org [image: Facebook] <http://www.facebook.com/IRRI.ricenews> [image: Twitter] <http://twitter.com/RiceResearch> [image: Flickr] <http://www.flickr.com/photos/ricephotos/collections/> [image: Youtube] <http://www.youtube.com/user/irrivideo/featured> [image: Scribd] <http://www.scribd.com/IRRI_resources> [image: Linkedin] <http://www.linkedin.com/company/international-rice-research-institute> [image: Soundcloud] <https://soundcloud.com/irri-radio> [image: Google+] <https://plus.google.com/103972671963502739315> The International Rice Research Institute <http://irri.org> is a member of the CGIAR <http://www.cgiar.org/> -- The International Rice Research Institute <http://irri.org> is a member of the CGIAR <http://cgiar.org> consortium
_______________________________________________ Owasp-modsecurity-core-rule-set mailing list Owasp-modsecurity-core-rule-set@lists.owasp.org https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
