Chaim, In the light of Walter's investigation into the deficiencies of the PHP rules in 3.0.0rc1 and his plan to work on these a lot, I think we can cut down the paranoia mode side-project by leaving 958977 / 933110 and 958979 / 933120 aside and have you and Walter cover them.
(sidenote for the list: Walter got in touch with Chaim and me on the weekend. He plans to write a longer message to the list one of these days). From my point of view, the question on how to cut the monolithic data files is really tricky. From the point of the paranoia mode it is a tangent issue, but it seems so difficult for me, that I would rather drop these rules from the our project then risking further delay. > 1) A parameter to indicate if PHP is in use in the config Please don't. Unless it's on by default. I think the CRS should work out of the box and deliver full coverage out of the box. If PHP protection is off by default, people will deploy ModSec without PHP protection and won't even notice. Cheers, Christian On Mon, Feb 15, 2016 at 04:13:32AM +0000, Chaim Sanders wrote: > It seems like this falls under the PHP work that will need to be done > prior to RC1 - I do agree it hasn’t been settled yet and since we’re gonna > be messing with it - we should most certainly come to a conclusion. I am > finding it hard to isolate the main points but my understanding is that we > want to add: > 1) A parameter to indicate if PHP is in use in the config > 2) Possible 1 additional data file with ‘paranoid’ versus regular PHP > function names > 3) to work with the issue of PHP bypasses - more on this in the more > detailed email. > > While I encourage the addition of other PHP function names this can be > done right now we don’t need a specific task for it for RC1 (but we should > add a general task to update it. Perhaps @christian what do you think of > making a task for it and assigning it to me. > > On 2/12/16, 4:19 PM, > "owasp-modsecurity-core-rule-set-boun...@lists.owasp.org on behalf of > Christian Folini" <owasp-modsecurity-core-rule-set-boun...@lists.owasp.org > on behalf of christian.fol...@netnea.com> wrote: > > >And another one. > > > >This conversation is also not settled. Please help us to come > >to a conclusion. > > > >The biggest issue is the question of the granularity of the > >data files. > > > >Ahoj, > > > >Christian > > > >On Sun, Feb 07, 2016 at 09:12:06PM +0100, Christian Folini wrote: > >> Hello, > >> > >> Your ideas are extremely valuable Walter. And even if I do not agree > >> with your solutions to the letter, I love the fact that we are having > >> technical discussions on the merit of individual rules. > >> > >> Chaim seems to be on the same page with me on the idea of > >> environment-specific rules and how to handle the enabling / disabling. > >> > >> And then the problem of the granularity of the files. There is a > >> danger in overengineering things and making the configuration / tuning > >> too complex. In that light, I think splitting makes sense, but > >> ideally, we end up with > >> > >> xxx-standard.data > >> xxx-paranoia.data > >> > >> However, you have a point that you can not alter the contents of these > >> data files afterwards in a local setup in a safe way. So disabling > >>Windows > >> commands won't work unless we add specific data files with their own > >> rules, which can then be disabled. > >> > >> I am really torn here. > >> > >> In my stated fear of overloading the paranoia project / pull request, > >> I would appreciate to handle the big updates to individual rules > >> in separate pull requests - if possible. If it has to be, we include it. > >> > >> Ahoj, > >> > >> Christian > >> > >> > >> On Sun, Feb 07, 2016 at 03:46:32PM +0000, Chaim Sanders wrote: > >> > I like the ideas. The first thing I’d note is that individuals are > >>free to > >> > exclude or remove entire files so as long as the PHP file is named > >> > REQUEST-XYZ-PHP-XYZ people running ASP should get this hint > >>(hopefully) > >> > this is why we name them in this way. In terms of splitting the > >>files, I > >> > most certainly agree with the ‘critical’. Again I caution against > >> > splitting data files TOO finely as it will get confusing to support > >>your > >> > own instance but this is just a minor issue. As for those > >>parenthesis. You > >> > are right and this is actually my fault. I added them to prevent false > >> > positives being reported commonly on Cpanel. > >> > > >> > On 2/7/16, 8:58 AM, > >> > "owasp-modsecurity-core-rule-set-boun...@lists.owasp.org on behalf of > >> > Walter Hop" <owasp-modsecurity-core-rule-set-boun...@lists.owasp.org > >>on > >> > behalf of mod...@spam.lifeforms.nl> wrote: > >> > > >> > >On 02 Feb 2016, at 09:25, Christian Folini > >><christian.fol...@netnea.com> > >> > >wrote: > >> > >> > >> > >> Like the previous post on 950907 / 932100, this is controversial > >>because > >> > >> of possible false positives due to a data file with strings: > >> > >> > >> > > >>>>http://scanmail.trustwave.com/?c=4062&d=7tO31v4ao2_NNeZ84J8wNSWydu00_AA > >>>>Pk > >> > > >>>>vSmewhU_w&s=5&u=https%3a%2f%2fraw%2egithubusercontent%2ecom%2fSpiderLab > >>>>s% > >> > > >>>>2fowasp-modsecurity-crs%2fv3%2e0%2e0-rc1%2frules%2fphp-function-names%2 > >>>>ed > >> > >>ata > >> > >> > >> > > >>>>http://scanmail.trustwave.com/?c=4062&d=7tO31v4ao2_NNeZ84J8wNSWydu00_AA > >>>>Pk > >> > > >>>>qKjKQ1VqQ&s=5&u=https%3a%2f%2fraw%2egithubusercontent%2ecom%2fSpiderLab > >>>>s% > >> > > >>>>2fowasp-modsecurity-crs%2fv3%2e0%2e0-rc1%2frules%2fphp-config-directive > >>>>s% > >> > >>2edata > >> > >> > >> > >> Could these files be split in the manner explained in the message > >> > >> before? > >> > >> > >> > >> After all, this rule scans input against strings like > >> > >> dl, eval, exec, from, precision. > >> > > > >> > >I would advocate the same separation as in my previous post re > >> > >os-commands. > >> > >But we need a little more firepower for PHP in my opinion. > >> > >A detailed approach could go like this... > >> > > > >> > >1) Have the user indicate with setvar if they want to protect PHP, > >>and > >> > >predicate the rules on that. All the non PHP shops will be solved > >>from FP > >> > >instantly! > >> > > > >> > >2) We need to review the current data file: > >> > >- I’m missing some stuff there that I see in the wild every day such > >>as: > >> > >gzinflate, gzdecode, gzuncompress, strrev, zlib_decode… > >> > >- I also see in some cases the parenthesis is added, like: /file(/ > >>and > >> > >/assert(/. This seems easy to evade. In PHP, calling “assert > >>($foo)” > >> > >or “assert\t\t\t($foo)” is just as valid. In this case, it was > >>probably > >> > >done because those terms lead to FP. If so, we might move super > >>common > >> > >terms with relatively low attacker interest, like assert, to the > >>paranoid > >> > >list. But, in any case they should become more robust regexps like > >> > >“\bfile\s*(“. > >> > > > >> > >3) Separate the data file. Start with a small > >> > >“php-commands-critical.data” list. This should contain the usual > >>suspects > >> > >used for: > >> > >- obfuscation: base64_decode, gzdecode, etc... > >> > >- execution: passthru, proc_open, shell_exec, call_user_func, > >> > >require_once, include_once... > >> > >- IO: file_get_contents, file_put_contents, fopen, fwrite... > >> > >- environment inspection and setting: error_reporting, > >>function_exists, > >> > >ini_set, auto_prepend_file, sys_get_temp_dir... > >> > > > >> > >If any of those strings is present, it’s certainly an attack, except > >>for > >> > >that single person who runs a PHP-related wiki. The list should only > >> > >contain the real red flags, don’t try to cleverly prevent FP using > >> > >complex regexps, and cannot contain any common English words (so > >> > >unfortunately “require” and “include” are off limits...) > >> > > > >> > >4) Other functions could go in “php-commands-functions.data”. This > >>list > >> > >would be small and contain only *proper functions* having PHP syntax > >> > >“function()” and have names often used in natural text. To prevent > >>FP we > >> > >look for the parenthesis. Let’s take example “system”, this is often > >> > >found in natural text, but an attacker wants to do “system(“ so they > >>need > >> > >parentheses and a word boundary at the start. So a regexp might look > >>like > >> > >"\bsystem\s*\(“. Because we have the regexp, FP is lowered. It would > >> > >still match stuff like “F*ck the system (yeah yeah)” > >> > > > >> > >5) Other PHP related terms could stay in “php-commands-common.data”. > >>This > >> > >would list the PHP commands and terms of lesser interest. > >> > > > >> > >6) We’ll have to review if there are terms which are so common in > >>natural > >> > >text that they’ll have to be moved to “php-commands-paranoid.data”. I > >> > >think this list might be as small as possible. Maybe stuff like “dl”, > >> > >“require”, “include”. It’s still likely to produce FP... > >> > > > >> > >7) String matching for PHP functions is not powerful enough. We need > >> > >generic rules that prevent PHP evasion. I don’t remember seeing this > >>in > >> > >the wild, but an easy way to evade textual signature matches would > >>be PHP > >> > >variable functions > >> > > >>>(http://scanmail.trustwave.com/?c=4062&d=7tO31v4ao2_NNeZ84J8wNSWydu00_AA > >>>Pk > >> > > >>>vCjeQMDpQ&s=5&u=http%3a%2f%2fphp%2enet%2fmanual%2fen%2ffunctions%2evaria > >>>bl > >> > >e-functions%2ephp%29 For instance, you can do the following to get > >>around > >> > >the CRS: > >> > > > >> > >$aB_4c = 'fil' . 'e_get' . '_contents'; > >> > >$aB_4c('...'); > >> > > > >> > >So we want a generic signature to detect variable function call > >>syntax. > >> > >Fortunately its syntax is very uncommon in natural text, so I’d > >>advocate > >> > >putting this rule in normal mode. > >> > > > >> > >Anyway my €0.02 or maybe a bit more :) > >> > > > >> > > > >> > >_______________________________________________ > >> > >Owasp-modsecurity-core-rule-set mailing list > >> > >Owasp-modsecurity-core-rule-set@lists.owasp.org > >> > > >>>http://scanmail.trustwave.com/?c=4062&d=7tO31v4ao2_NNeZ84J8wNSWydu00_AAP > >>>kq > >> > > >>>emcgoEpA&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flistinfo%2f > >>>ow > >> > >asp-modsecurity-core-rule-set > >> > > >> > > >> > ________________________________ > >> > > >> > This transmission may contain information that is privileged, > >>confidential, and/or exempt from disclosure under applicable law. If you > >>are not the intended recipient, you are hereby notified that any > >>disclosure, copying, distribution, or use of the information contained > >>herein (including any reliance thereon) is strictly prohibited. If you > >>received this transmission in error, please immediately contact the > >>sender and destroy the material in its entirety, whether in electronic > >>or hard copy format. > >> > _______________________________________________ > >> > Owasp-modsecurity-core-rule-set mailing list > >> > Owasp-modsecurity-core-rule-set@lists.owasp.org > >> > > >>http://scanmail.trustwave.com/?c=4062&d=zdK-1lANbYH0zbTgGyqCcBKbND23Kjo7G > >>5xS4hMPhQ&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flistinfo%2f > >>owasp-modsecurity-core-rule-set > >> > >> -- > >> mailto:christian.fol...@netnea.com > >> > >>http://scanmail.trustwave.com/?c=4062&d=zdK-1lANbYH0zbTgGyqCcBKbND23Kjo7G > >>5kHvhVUiA&s=5&u=http%3a%2f%2fwww%2echristian-folini%2ech > >> twitter: @ChrFolini > >> _______________________________________________ > >> Owasp-modsecurity-core-rule-set mailing list > >> Owasp-modsecurity-core-rule-set@lists.owasp.org > >> > >>http://scanmail.trustwave.com/?c=4062&d=zdK-1lANbYH0zbTgGyqCcBKbND23Kjo7G > >>5xS4hMPhQ&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flistinfo%2f > >>owasp-modsecurity-core-rule-set > > > >-- > >mailto:christian.fol...@netnea.com > >http://scanmail.trustwave.com/?c=4062&d=zdK-1lANbYH0zbTgGyqCcBKbND23Kjo7G5 > >kHvhVUiA&s=5&u=http%3a%2f%2fwww%2echristian-folini%2ech > >twitter: @ChrFolini > >_______________________________________________ > >Owasp-modsecurity-core-rule-set mailing list > >Owasp-modsecurity-core-rule-set@lists.owasp.org > >http://scanmail.trustwave.com/?c=4062&d=zdK-1lANbYH0zbTgGyqCcBKbND23Kjo7G5 > >xS4hMPhQ&s=5&u=https%3a%2f%2flists%2eowasp%2eorg%2fmailman%2flistinfo%2fow > >asp-modsecurity-core-rule-set > > > ________________________________ > > This transmission may contain information that is privileged, confidential, > and/or exempt from disclosure under applicable law. If you are not the > intended recipient, you are hereby notified that any disclosure, copying, > distribution, or use of the information contained herein (including any > reliance thereon) is strictly prohibited. If you received this transmission > in error, please immediately contact the sender and destroy the material in > its entirety, whether in electronic or hard copy format. > _______________________________________________ > Owasp-modsecurity-core-rule-set mailing list > Owasp-modsecurity-core-rule-set@lists.owasp.org > https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set _______________________________________________ Owasp-modsecurity-core-rule-set mailing list Owasp-modsecurity-core-rule-set@lists.owasp.org https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
