Hi Konrad, On Fri, Nov 21, 2014 at 06:05:01PM +0100, konrad rzentarzewski wrote: > while migrating 1.4 => 1.5 i've came accross problem with regex lists, > which worked seamlessly before. > > in every frontend there's a construct: > > > acl worms_path url_reg -f /etc/haproxy/lists/worms_regex.lst > acl invalid_path path_reg -f /etc/haproxy/lists/invalid_paths.lst > block if invalid_path or worms_path or HTTP_URL_STAR !METH_OPTIONS or > METH_OPTIONS !HTTP_1.1 > > > and relevant files contain some lists of regexes: > > > ==> /etc/haproxy/lists/worms_regex.lst <== > ^/internal/ > (\.|%2E|%2e)(\.|%2E|%2e)(%2F|%2f|%5C|%5c|/|\\\\) > ([^\ ]*\ [^\ ]*\ |.*%00) > (<|%3C|%3c)(%73|s|S)(%63|c|C)(%72|r|R)(%69|i|I)(%70|p|P)(%74|t|T) > /(root\.exe\?|cmd\.exe\?|default\.ida\?) > > ==> /etc/haproxy/lists/invalid_paths.lst <== > # don't allow double slashes > //+ > > # don't allow %-encoding for [a-zA-Z0-9-~._] > %(2d|2e|30|31|32|33|34|35|36|37|38|39|41|42|43|44|45|46|47|48|49|4a|4b|4c|4d|4e|4f|50|51|52|53|54|55|56|57|58|59|5a|5f|61|62|63|64|65|66|67|68|69|6a|6b|6c|6d|6e|6f|70|71|72|73|74|75|76|77|78|79|7a|7e) > > > when applied to 1.5, it causes 100% CPU use in seconds. > no matter how much cores i use (nbproc 1-8) it maxes them out. > > is there any change in regex performance?
There were changes to the acl part but in theory this should not impact performance, especially not like this. Are you sure you compiled 1.5 the same way as you did for 1.4 ? Please run "haproxy -vv" for both versions and output the result. It's possible that you used libpcre for 1.4 and not for 1.5 for example. PCRE is extremely fast in general and not using it could easily cause what you're observing. > should i convert my acl or file lists somehow? > > i use haproxy 1.5.8 compiled from > https://github.com/bluerail/haproxy-centos spec with centos 5. OK this one seems to use libpcre. The output of both versions will help compare what could impact this use case. > this lb processes 2-8k connections. You mean pre second I guess ? BTW, you can even use PCRE_JIT in 1.5 which is even faster on most workloads. You need to set "USE_PCRE_JIT=1" in addition to "USE_PCRE=1". Best regards, Willy
