https://bz.apache.org/bugzilla/show_bug.cgi?id=66168

--- Comment #3 from [email protected] ---
Created attachment 40249
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=40249&action=edit
Minimal FastCGI responder sending an early empty FCGI_STDOUT record
(reproducer)

Root cause identified and reproduced without PHP on httpd 2.4.70.

Environment
-----------
httpd 2.4.70, mpm_event, mod_http2, mod_proxy_fcgi -> PHP-FPM
mod_brotli enabled via AddOutputFilterByType (mod_filter):
  AddOutputFilterByType BROTLI_COMPRESS text/html text/plain ...
libbrotli 1.1.0-2ubuntu0.1, Ubuntu 24.04.5 LTS
PHP-FPM 7.4.33 as real-world trigger (see below)

Summary
-------
The segfault is a NULL dereference of the brotli encoder state. It happens
when mod_brotli receives data after it has already processed EOS:

1. The FastCGI backend sends an empty FCGI_STDOUT record in the middle of
   the response and continues with further FCGI_STDOUT records afterwards.
   PHP-FPM does exactly this due to PHP bug #72185 (all 7.x, 8.0 < 8.0.20,
   8.1 < 8.1.7): when the stream switches between STDOUT and STDERR while
   its 8 KB output buffer is full, it emits an empty record.
2. mod_proxy_fcgi treats the empty FCGI_STDOUT record as end of stream and
   passes EOS down the output filter chain, but keeps reading from the
   backend and forwards the following FCGI_STDOUT records as well,
   i.e. data after EOS.
3. mod_brotli finishes and destroys the encoder on EOS (ctx->state = NULL)
   but stays in the filter chain (wrapped by mod_filter's filter_harness).
   The next data bucket reaches process_chunk(), and
   BrotliEncoderCompressStream(NULL, BROTLI_OPERATION_PROCESS, ...)
   dereferences NULL.

The same call chain was already reported in bug 63216 (2.4.38) and in this
bug (2.4.41 - 2.4.57).

Reproduction without PHP
------------------------
The attached fcgi_early_eos.py (Python 3, stdlib only) is a minimal
FastCGI responder. With mode=early it sends a header record, 20
FCGI_STDOUT records, an empty FCGI_STDOUT record, 5 more FCGI_STDOUT
records and a final "END" line.

  python3 fcgi_early_eos.py --listen 127.0.0.1:9123      (as non-root)

  ProxyPass "/fcgitest" "fcgi://127.0.0.1:9123/"

  curl --http1.1 -o /dev/null -H 'Accept-Encoding: br'
'https://<host>/fcgitest?mode=early'
  curl --http2   -o /dev/null -H 'Accept-Encoding: br'
'https://<host>/fcgitest?mode=early'

Result: both requests crash the child process (AH00052). This happens
with HTTP/1.1 as well as HTTP/2, so mod_http2 is not involved.

Controls:
  - mode=normal (no early empty record), brotli:      no crash
  - mode=early, Accept-Encoding: gzip (mod_deflate):   no crash
  - mode=early, Accept-Encoding: identity:             response truncated
    at the early EOS (final "END" line missing)

Reproduction with PHP-FPM
-------------------------
PHP-FPM <7.x.y>, log_errors on (messages are sent as FCGI_STDERR):

  <?php
  ini_set('display_errors', 1);
  for($i=0;$i<500;$i++)
  {
  f;
  }
  ?>

Whether the empty record is hit depends on the exact byte layout of the
output (script path length, line numbers, headers). With a different path
the loop count may need to be varied (e.g. 100..1000). This explains why
the crash appears sporadic and why it looked DocumentRoot-dependent in
the original report.

Analysis details
----------------
Crash on 2.4.70 (no debug symbols for httpd; the symbolized backtrace in
the original report shows the same call chain):
  #0 BrotliEncoderCompressStream+45: mov 0x1b3c(%rdi),%r8d
     rdi = 0 (encoder state), rsi = 0 (BROTLI_OPERATION_PROCESS)
  #1 mod_brotli, process_chunk (avail_out = 0, next_out = NULL)
  #2 mod_filter, filter_harness
  #3 mod_proxy_fcgi, dispatch

brotli_ctx_t in frame #1:
  state     = NULL
  bb        = valid pointer
  total_in  = 15957 (= 81 x 197 bytes)
  total_out = 1043
The encoder had already produced output and was cleaned up regularly
before the crash; this is not a corrupted context.

FilterTrace BYTYPE:BROTLI_COMPRESS 1 on the crashing request:
  (TRANSIENT 197, FLUSH) x 81, EOS, TRANSIENT 197  -> segfault

strace of the PHP-FPM worker: two write()s of 8200 bytes (the full 8192
byte buffer plus an appended record header without content), ending in
  01 07 00 01 00 00 00 00   (empty FCGI_STDERR)
  01 06 00 01 00 00 00 00   (empty FCGI_STDOUT)
followed by further FCGI_STDOUT records.

Impact
------
Only the child process crashes (AH00052 is logged by the parent), but all
other connections handled by that child are lost. On shared hosting any
customer running an affected PHP version can trigger this with ordinary
PHP warnings. Independent of compression, clients receive a truncated
response at the early EOS.

Suggested fixes
---------------
- mod_proxy_fcgi: after an empty FCGI_STDOUT record, do not pass further
  STDOUT data down the filter chain; discard it and log once that the
  backend sent data after the end of the stream.
- mod_brotli: never call the encoder with a NULL state; ignore buckets
  that arrive after EOS instead of crashing.

Workarounds
-----------
- SetEnv no-brotli 1 for responses from affected backends
- keep PHP messages off FCGI_STDERR (error_log to a file or
  fastcgi.logging = 0), which avoids the stream switches that trigger
  the empty record in PHP-FPM
- upgrade PHP-FPM to 8.0.20 / 8.1.7 or later

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to