https://bz.apache.org/bugzilla/show_bug.cgi?id=63170
Bug ID: 63170
Summary: mod_proxy_http2 segmentation fault and out of memory
condition
Product: Apache httpd-2
Version: 2.4.38
Hardware: PC
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy_http2
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Hello Team,
I've been using mod_proxy_http for a few years now in a Reverse Proxy scenario.
Recently, we have wanted to use HTTP/2 protocol so we have compiled and
configured mod_proxy_http2. The reverse proxy forwards request to another
system that is also configured for HTTP/2. All components are built from
source. When we enable HTTP/2 ("h2") protocol we are seeing frequent
segmentation faults as well as out of memory conditions. As soon as we
downgrade protocol to HTTP/1.1 the segfaults and OOM condition goes away.
These problems only occur on the Reverse Proxy, not on the receiving web
server.
Segfaults look like this:
[Tue Feb 12 20:48:15 2019] [notice] [pid 2880] mpm_unix.c(436): [client
AH00052: child pid 3224 exit signal Segmentation fault (11)
[Tue Feb 12 20:48:15 2019] [notice] [pid 2880] mpm_unix.c(436): [client
AH00052: child pid 3313 exit signal Segmentation fault (11)
On the Reverse Proxy, we are using the following software:
httpd-2.4.38
nghttp2-1.36.0
openssl-1.0.2q
pcre-8.41
apr-1.6.5
apr-util-1.6.1
On the web server behind the Reverse Proxy, we are using the following
software:
httpd-2.4.37
nghttp2-1.30.0
openssl-1.0.2o
pcre-8.41
apr-1.6.5
apr-util-1.6.1
The virtual host for the SSL configuration on the Reverse Proxy is the
following (with extra settings removed that aren't relevant)
<VirtualHost 172.31.8.194:443>
SSLEngine on
Protocols h2 h2c http/1.1
ServerName www.example.com
ProxyPreserveHost On
ProxyPass "/" "h2://www.example.com/"
ProxyPassReverse "/" "h2://www.example.com/"
</VirtualHost>
If we change the Protocols line to simply
Protocols http/1.1
Then the segfault and out of memory conditions stop.
I did some printf-level logging in modules/http2/mod_proxy_http2.c and I can
see that it gets stuck in some sort of infinite loop situation. I believe the
infinite loop is probably what's eating up the memory (so, for example, its
allocating memory for structures and char pointers a zillion times but not
releasing them).
Here is a trace level example of what's being spit out when mod_proxy_http2
gets into an infinite loop:
[Tue Feb 12 17:16:53 2019] [trace1] [pid 18535] mod_proxy_http2.c(351): [client
64.136.202.39:61387] eng(eng-proxy-51539607943): setup session
[Tue Feb 12 17:16:53 2019] [debug] [pid 18535] mod_proxy_http2.c(364): [client
64.136.202.39:61387] AH03373: eng(eng-proxy-51539607943): run session
eng-proxy-51539607943
[Tue Feb 12 17:16:53 2019] [debug] [pid 18535] mod_proxy_http2.c(640): [client
64.136.202.39:61387] AH03376: run_session, again
[Tue Feb 12 17:16:53 2019] [trace1] [pid 18535] mod_proxy_http2.c(351): [client
64.136.202.39:61387] eng(eng-proxy-38654706035): setup session
[Tue Feb 12 17:16:53 2019] [debug] [pid 18535] mod_proxy_http2.c(364): [client
64.136.202.39:61387] AH03373: eng(eng-proxy-38654706035): run session
eng-proxy-38654706035
[Tue Feb 12 17:16:53 2019] [debug] [pid 18535] mod_proxy_http2.c(640): [client
64.136.202.39:61387] AH03376: run_session, again
[Tue Feb 12 17:16:53 2019] [trace1] [pid 18535] mod_proxy_http2.c(351): [client
64.136.202.39:61387] eng(eng-proxy-51539607945): setup session
[Tue Feb 12 17:16:53 2019] [debug] [pid 18535] mod_proxy_http2.c(364): [client
64.136.202.39:61387] AH03373: eng(eng-proxy-51539607945): run session
eng-proxy-51539607945
I can't explain it exactly, but it almost feels like the web browser completes
its HTTP/2 requests but mod_proxy_http2 isn't notified so it goes crazy.
Another thing I noticed is that I only seem to be able to induce the OOM
condition when using FireFox 65 (Windows 10) and not Chrome or Edge.
Please let me know if there are other details I can provide. We've spent a lot
of time on this bug and simply don't know what to do.
Thanks,
Warren
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
