Hi Ryan, Piotr and all,
I'm just trying to get my head round this thread, so I'd appreciate it if
you'd see whether the following is a reasonable summary:
(Below I refer to the U+0084 character as an example, but the same is true
for the other non-ascii characters in the rule)
Rule 981318 checks for the U+0084 character because some application
environments transliterate that to an apostrophe/single quote, not because
U+0084 'dangerous' in it's own right.
False positives occur because when processing the U+0084 character (\xc2\xb4
in UTF8) the PCRE interprets the \xc2\xb4 as individual bytes and will
trigger on any string where the first/last character's first/last byte
(respectively) is \xc2 or \xb4.
There isn't a direct fix for this PCRE multibyte issue but by::
1)Adding a t:utf8toUnicode to the 981318 rule it will cause any \xc2\xb4 in
the input to be transformed to a %u0084 representation internally.
2)Setting up the
SecUnicodeMapFile /etc/apache2/unicode.mapping 20127
it will cause the %u0084 representation to be transliterated to an ascii
apostrophe/single quote
The U+0084 character in the 981318 is now unnecessary because any instances
of that in the input will now just appear to the rule as an apostrophe
covered elsewhere in the rule.
This allows us to remove the U+0084 from the rule and eliminate the
multibyte PCRE collision issue.
So, in essence, t:utf8toUnicode and SecUnicodeMapFile don't solve the
underlying PCRE problem per se but avoid needing to put multibyte data into
the PCRE for the rule to have the same effect.
So the best way to 'fix' rule 981318 would be to:
-Set up the SecUnicodeMapFile
-Add t:utf8toUnicode to the 981318 rule
-Remove the U+0084 (and other non-ascii) characters from the 981318 regex.
Does that make sense?
Similarly if you were ultra confident nothing in your application stack did
do U+0084 to 'single quote' transliteration (and for other characters) you
could just remove them from the 981318 rule.
Paul
On Tue, Mar 18, 2014 at 1:16 AM, Ryan Barnett <ryan.barn...@owasp.org>wrote:
> On 3/17/14 10:01 AM, "Piotr Gackiewicz" <p.gackiew...@intertele.pl> wrote:
>
> >On Mon, 17 Mar 2014, Ramy Darwish wrote:
> >
> >> Hi Piotr,
> >>
> >> In my experience, you need to use a utf8toUnicode transformation to
> >>properly
> >> map UTF8 strings to Unicode before the rule processes the input as
> >>Unicode,
> >> as well as fine-tune the urlDecodeUni transformation to properly
> >>normalize
> >> Central European Unicode characters.
> >>
> >> Provide a code point declaration for the urlDecodeUni transformation,
> >> in order to properly normalize Unicode strings (in modsecurity.conf)
> >> ---------------------------------------------------
> >> # With the 1250 code point for Central Europe:
> >> SecUnicodeMapFile /etc/modsecurity/unicode.mapping
> >> SecUnicodeCodePage 1250
> >> ---------------------------------------------------
> >>
> >> See these resources for more info:
> >> http:
> >>//
> blog.spiderlabs.com/2011/06/modsecurity-advanced-topic-of-the-week-unic
> >>ode-mapping-support.html
> >
> >Thanks for the response.
> >Unfortunately, both You and Ryan Barnett in:
> >> http: //blog.spiderlabs.com/2012/08/waf-normalization-and-i18n.html
> >are probably missing the point.
> >
> >This is not the problem with URI normalization/Best-Fit Mapping. This is a
> >problem with UTF8 multibyte characters, put into rule regexp,
> >matched with UTF8-unaware PCRE code.
>
> You are correct. These rules were made prior to the UTF8/Unicode mapping
> and transformation functions were added to the ModSecurity code. These
> regexes in the rules will be updated (in CRS v.3.0.0) to be quote (')
> double-quote (") and backtick (`) characters instead. The
> SecUnicodeMapFile directive + t:utf8toUnicode.t:urlDecodeUni
> transformation functions should then properly handle the data.
>
> -Ryan
>
>
> _______________________________________________
> Owasp-modsecurity-core-rule-set mailing list
> Owasp-modsecurity-core-rule-set@lists.owasp.org
> https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
>
_______________________________________________
Owasp-modsecurity-core-rule-set mailing list
Owasp-modsecurity-core-rule-set@lists.owasp.org
https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
