Hello,
When using a vulnerability scanner on haproxy 1.7.5, we discovered a scenario
under which the haproxy segfaults.
Unfortunately, this is a "bundled" scanner whith no access to the exact
requests, and the haproxy terminates the SSL for https, so not easy to capture
the actual traffic, but we managed to gather a core of haproxy.
This happens only when using the cookie SERVERID for session stickiness:
The backtrace is as follow:
I believe the scanner injects a screwed up header and/or cookie. The result
appears to be the incorrect memmove of the last header line :
(gdb) bt full
#0 _wordcopy_fwd_dest_aligned (dstp=14237664, srcp=14237696,
len=2305843009213653720) at wordcopy.c:196
a0 = <value optimized out>
a1 = <value optimized out>
a2 = 0
a3 = 0
sh_1 = 40
sh_2 = 24
#1 0x0000003727a838be in memmove (dest=0xd4575d, src=<value optimized out>,
len=18446744073709551439) at memmove.c:73
dstp = <value optimized out>
srcp = <value optimized out>
#2 0x0000000000411217 in buffer_replace2 (b=0xd456b0, pos=0xd4575d
"Accept-Encoding: gzip\r\n\r\n.32.31\r\nConnection: close\r\n\r\nre\r\n\r\n",
end=<value optimized out>, str=0x0, len=0) at src/buffer.c:90
delta = -21
#3 0x000000000045de02 in manage_client_side_cookies (s=0xcfa800, req=0xcfa810)
at src/proto_http.c:7976
delta = <value optimized out>
cur_hdr = 0xcfac6c
val = <value optimized out>
txn = <value optimized out>
sess = 0xc1c930
preserve_hdr = 0
cur_idx = 3
old_idx = 2
hdr_beg = 0xd4575d "Accept-Encoding: gzip\r\n\r\n.32.31\r\nConnection:
close\r\n\r\nre\r\n\r\n"
hdr_end = 0xd45770 "ip\r\n\r\n.32.31\r\nConnection:
close\r\n\r\nre\r\n\r\n"
hdr_next = 0xd45772 "\r\n\r\n.32.31\r\nConnection:
close\r\n\r\nre\r\n\r\n"
del_from = 0xd45763 "-Encoding: gzip\r\n\r\n.32.31\r\nConnection:
close\r\n\r\nre\r\n\r\n"
prev = <value optimized out>
att_beg = <value optimized out>
att_end = <value optimized out>
equal = <value optimized out>
val_beg = <value optimized out>
val_end = <value optimized out>
next = <value optimized out>
#4 0x0000000000460a86 in http_process_req_common (s=0xcfa800, req=0xcfa810,
an_bit=256, px=0x86a650) at src/proto_http.c:4474
sess = 0xc1c930
txn = 0xcfab50
msg = 0xcfabb0
rule = <value optimized out>
wl = <value optimized out>
verdict = <value optimized out>
deny_status = 2
#5 0x0000000000486d0e in process_stream (t=0xb8fa70) at src/stream.c:1798
max_loops = 199
ana_list = 2304
ana_back = 2304
flags = 2
srv = <value optimized out>
s = 0xcfa800
sess = 0xc1c930
rqf_last = <value optimized out>
rpf_last = 2147745792
rq_prod_last = <value optimized out>
rq_cons_last = <value optimized out>
rp_cons_last = 7
rp_prod_last = 0
req_ana_back = <value optimized out>
req = 0xcfa810
res = 0xcfa850
si_f = 0xcfaa38
si_b = 0xcfaa60
#6 0x0000000000415ac0 in process_runnable_tasks () at src/task.c:238
t = <value optimized out>
max_processed = <value optimized out>
#7 0x0000000000407028 in run_poll_loop () at src/haproxy.c:1724
next = <value optimized out>
#8 0x000000000040a308 in main (argc=<value optimized out>, argv=<value
optimized out>) at src/haproxy.c:2105
err = <value optimized out>
retry = <value optimized out>
limit = {rlim_cur = 4091, rlim_max = 4091}
errmsg =
"\000\347K\000\000\000\000\000\b\300n\000\000\000\000\000\020\340\377\377\377\177\000\000\000\300n\000\000\000\000\000\006\000\000\000\000\000\000\000H\341\377\377\377\177\000\000\200\341\377\377\377\177\000\000XpA\000\000\000\000\000\000\300n\000\000\000\000\000\226\070K\000\000\000\000\000\340\070\370/7\000\000\000\340\067K\000\000\000\000\000\000\000\000"
Basically, the memove in #1 is called with len=18446744073709551439 from
buffer_replace2, which is a negative value.
I am not sure exactly in which case this is possible (last line of the header
incorrect or something), but bi_end(b) is < to end, so the unsigned size_t
expected by memmove is incorrect.
I recompiled haproxy with the patch below and now it survives the vulnerability
scanner, but it might not be at the proper place of the code (maybe the logic
fault is better addressed above in manage_client_side_cookies... ):
diff -uNr haproxy-1.7.5/src/buffer.c haproxy-1.7.5p/src/buffer.c
--- haproxy-1.7.5/src/buffer.c 2017-04-03 10:28:32.000000000 +0200
+++ haproxy-1.7.5p/src/buffer.c 2017-05-26 13:04:58.225311000 +0200
@@ -87,7 +87,8 @@
return 0; /* no space left before wrapping data */
/* first, protect the end of the buffer */
- memmove(end + delta, end, bi_end(b) - end);
+ if ( bi_end(b) > end )
+ memmove(end + delta, end, bi_end(b) - end);
/* now, copy str over pos */
if (len)
the configuration is as follows:
frontend ft_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_443
bind 185.139.245.111:443 ssl crt
/etc/haproxy/ssl/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.pem ciphers
AES256+EECDH:AES256+EDH
bind-process 2 3 4
mode http
option http-buffer-request
http-response set-header Strict-Transport-Security "max-age=16000000;
includeSubDomains; preload;"
http-request del-header Origin
stick-table type ip size 1m expire 1m store gpc0,conn_rate(1s)
tcp-request connection track-sc1 src
tcp-request connection reject if { sc1_get_gpc0 gt 0 }
acl abuse_tcp_conn sc1_conn_rate(ft_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_443)
ge 80
acl flag_tcp_conn_abuser
sc1_inc_gpc0(ft_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_443) gt 0
tcp-request content reject if abuse_tcp_conn flag_tcp_conn_abuser
tcp-request connection reject if {
sc1_get_gpc0(XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_8443) gt 1 }
default_backend XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_8443
maxconn 5000
timeout client 5s
log global
backend XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_8443
mode http
balance roundrobin
option forwardfor
stick-table type ip size 1m expire 1m store
gpc0,http_req_rate(1s),http_err_rate(1s)
http-request track-sc2 src
acl abuse_too_many_errors
sc2_http_err_rate(XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_8443) ge 20
acl flag_abuser_too_many_errors
sc2_inc_gpc0(XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_8443) gt 0
reqtarpit . if abuse_too_many_errors flag_abuser_too_many_errors
reqtarpit . if { sc2_get_gpc0 gt 0 } flag_abuser_too_many_errors
timeout check 10s
timeout check 5s
option httpchk HEAD /XXXXXXXXX/ HTTP/1.1\r\nHost:localhost
cookie SERVERID insert indirect nocache
server ppmktplportals01fe 172.21.38.22:8443 check ssl verify none check
cookie ppmktplportals01fe
server ppmktplportals01fv 172.21.38.100:8443 check ssl verify none
check cookie ppmktplportals01fv
Let me know if I can help more,
Best Regards,
- Jean
This email and its content belong to Ingenico Group. The enclosed information
is confidential and may not be disclosed to any unauthorized person. If you
have received it by mistake do not forward it and delete it from your system.
Cet email et son contenu sont la propriété du Groupe Ingenico. L’information
qu’il contient est confidentielle et ne peut être communiquée à des personnes
non autorisées. Si vous l’avez reçu par erreur ne le transférez pas et
supprimez-le.
