Package: libapache2-mod-authnz-external
Version: 3.3.2-3
Severity: important
Tags: patch

Dear Debian maintainers,

I would like to report a subprocess-record lifetime bug in mod_authnz_external. In exec_external(), the address of a stack-allocated apr_proc_t is registered with the request pool for cleanup. APR retains that address and may dereference it after exec_external() has returned.

We found this while investigating intermittent SVN-over-HTTPS failures on two Debian 13 hosting servers. A minimal proposed patch is attached. Customer names, addresses, repository names and hostnames have been omitted below.

1. Environment

Both production systems:

   Debian GNU/Linux 13 (trixie), amd64
   apache2 / apache2-bin:             2.4.71-1~deb13u1
   libapache2-mod-authnz-external:    3.3.2-3
   libapr1t64:amd64:                 1.7.5-1
   libapache2-mod-svn / libsvn1:     1.14.5-3
   Apache MPM:                      event

Relevant loaded modules include authnz_external, dav_svn, authz_svn, cgid, mod_perl and Passenger. The presence of mod_perl or Passenger is not evidence that either is responsible; we have not established a conflict involving those modules.

The unmodified mod_authnz_external.so on both hosts had SHA256:

   35fe6ad5012969d79890bea70af20cfc456683f2e92479c6ce84505f791ea9dd

This matched the module extracted independently from Debian's official 3.3.2-3 amd64 package. dpkg -V reported no modification to the installed package before our local fix.

2. Relevant configuration and observed workload

The SVN virtual hosts use an external PHP CLI authenticator via a pipe. The relevant configuration is equivalent to:

   DefineExternalAuth phplogin pipe /path/to/product/files/phpgitlogin.php

   <Location /svn/>
       DAV svn
       SVNParentPath /path/to/product/files/svn
       AuthType Basic
       AuthName "SVN Access"
       AuthBasicProvider external
       AuthExternal phplogin
       Require valid-user
       AuthzSVNAccessFile /path/to/product/files/authz
       AuthzSVNAnonymous off
   </Location>

The affected client workload included updates and fresh checkouts containing many files, with multiple requests/connections. Client user agents included SVN 1.14.5 and SVN 1.15.0 on Windows, both using serf 1.3.10.

The original report followed migration to another hosting server. This does not establish that the bug was introduced by a recent Apache upgrade: the same source defect appears to have existed for some time, and another server already had similar log entries.

3. Production symptoms

The customer reported these errors, depending on the attempt and compression setting:

   svn: E120104: ra_serf: An error occurred during decompression
   svn: E120106: ra_serf: The server sent a truncated HTTP response body.
   svn: E175009: The XML response contains invalid XML
   svn: E130003: Malformed XML: no element found at line 22

An update sometimes processed only a few files before aborting. Disabling HTTP compression did not resolve it. A client workaround enabling bulk updates and extending the timeout helped; because both options were changed together, that alone did not isolate the cause.

Representative server log messages, with identifying details redacted:

   [authnz_external:error] (10)No child processes:
       Could not get status from child process
   [authnz_external:error] AuthExtern phplogin
[/path/to/product/files/phpgitlogin.php]: Failed (-5) for user [redacted]
   [auth_basic:error] AH01617: user [redacted]: authentication failure
       for "/svn/[repository]/!svn/...": Password Mismatch

The "Password Mismatch" line follows the failure to collect the authenticator's exit status; it should not be interpreted as proof of an incorrect password.

We also observed:

   [cgid:error] AH01239: cgid daemon process died, restarting

On the second server, scanning 15 retained Apache error-log files spanning 1-15 November 2026 found 12 occurrences of "Could not get status from child process", 12 corresponding "Failed (-5)" messages and 58 CGI daemon restart messages. No "Segmentation fault" messages were found in that scan. These counts alone do not prove that every CGI restart had the same cause.

4. Source-level problem

In exec_external() in mod_authnz_external.c, the relevant code is:

   apr_pool_t *p = r->pool;
   apr_proc_t proc;
   ...
   rc = apr_proc_create(&proc, ...);
   ...
   apr_pool_note_subprocess(p, &proc, APR_KILL_AFTER_TIMEOUT);
   ...
   rc = apr_proc_wait(&proc, &status, &why, APR_WAIT);
   ...
   return status;

The registration remains attached to the request pool after this function returns, but proc is a local automatic variable whose lifetime has ended.

In APR 1.7.5, apr_pool_note_subprocess() allocates a process_chain node and stores the supplied pointer directly:

   pc->proc = proc;

It does not copy the apr_proc_t. Later, free_proc_chain() uses pc->proc with apr_proc_wait() and, depending on the result, apr_proc_kill(). Therefore request-pool cleanup can read an overwritten process record.

References:

https://sources.debian.org/src/libapache2-mod-authnz-external/3.3.2-2/mod_authnz_external.c/
   https://github.com/apache/apr/blob/1.7.5/memory/unix/apr_pools.c
   https://github.com/apache/apr/blob/1.7.5/threadproc/unix/proc.c

The source link above is for Debian revision -2; our installed package is -3. We also examined the official 3.3.2 source archive and the matching -3 binary. The binary disassembly confirms a stack-derived address passed to apr_pool_note_subprocess(): r15 is set to rsp+0x60 at 0x27a1 and passed as the process-record argument at 0x27ec/0x27f2.

We additionally inspected the 3.3.3 upstream source archive available from Debian's package pool; it still contains the same stack-variable registration. We have not built or performed full Apache integration tests against that version.

5. Consequences and isolated mechanism test

For example, if reused stack storage leaves pid equal to zero, Linux waitpid(0, ...) can collect the status of another child in the same process group. Its intended waiter may subsequently receive ECHILD. If cleanup sees a running child and proceeds to kill(0, SIGTERM), the signal targets processes in the caller's process group, subject to normal permission checks.

This provides a plausible common mechanism for lost authenticator exit statuses, worker/CGI termination and truncated HTTP responses. We did not trace the original production requests at syscall level, so we cannot claim to have captured that exact chain for each incident.

To validate the mechanism, we ran an isolated Python/ctypes harness against the real Debian APR 1.7.5 shared library. The harness:

   - creates an APR pool;
   - creates an authentication-like child and registers its process record;
   - collects that child's successful exit status;
   - creates an unrelated child;
- deliberately overwrites the original record with zeros to model reuse of
     storage after the original function has returned;
- destroys the APR pool and checks the unrelated child's status and signals.

Each scenario ran in a separate session/process group. This is a deterministic test of the cleanup consequences of an overwritten record, not a full reproduction of the original unpatched Apache request lifecycle.

Results:

   Original pointer, unrelated child already exited:
       APR cleanup collected the unrelated child's exit status.

   Pool-owned copy, same scenario:
       The unrelated child's exit status remained available to its owner.

   Original pointer, unrelated child still running:
Cleanup signalled the isolated test process group and terminated the child.

   Pool-owned copy, same scenario:
       No group signal occurred; the unrelated child remained running.

All four expectations passed. I can provide the harness and its output if useful.

6. Proposed patch

The attachment changes the registration to retain a request-pool-owned copy:

   apr_pool_note_subprocess(p, apr_pmemdup(p, &proc, sizeof(proc)),
                           APR_KILL_AFTER_TIMEOUT);

The existing synchronous apr_proc_wait() continues to operate on the local proc. The copy retained by APR remains valid until request-pool cleanup. The source already includes apr_strings.h, which declares apr_pmemdup().

The patch targets the original 3.3.2 source and passed a dry-run application check. It intentionally makes only this lifetime correction. It does not change authentication decisions, access rules, signal-handler handling, or the Apache MPM.

7. Build, deployment and verification

We built a local package version 3.3.2-3+local1 from Debian's 3.3.2-3 packaging in an isolated Debian 13 environment and deployed that identical package on both servers, preserving backups of the original package, module and Apache configuration.

The fixed module's SHA256 is:

   829c3a8b2e076199abbe18e9b3d9ce1230a4f2514c8ad925838dc0d1ba799c2c

In the isolated Debian 13 environment, and again in a separate localhost-only Apache test instance on the first production host, the fixed module passed:

- 240 authenticated HTTP/HTTPS requests with up to 16 concurrent workers; - invalid-password checks over HTTP and HTTPS, both correctly returning 401;
   - four concurrent SVN checkouts over HTTPS, each containing 201 files,
     including a 1 MiB binary file checked byte-for-byte;
   - one SVN commit and three concurrent SVN updates over HTTPS;
- checks for ECHILD/Failed(-5), unexpected CGI restarts and segmentation faults.

The temporary server test instance and repository were removed afterwards. No customer repository was used for those writes.

On both production servers, Apache was fully restarted and the new module was verified in the running process's mappings against the installed file inode. Apache configuration tests passed. Repository endpoints remained reachable with valid TLS and the expected HTTP 401 challenge when accessed without credentials. No new matching process-error messages appeared during the short post-deployment checks.

A fresh checkout using the originally affected customer's real credentials has not been performed, and long-term production observation is still pending. Existing Passenger configuration warnings were present before the change and were not addressed as part of this fix.

We subsequently enabled SVNAllowBulkUpdates Prefer as a separate operational trial. The fixed-module integration tests on the first host had already passed before that global setting was introduced. We are not treating the bulk-update preference as a substitute for the lifetime fix or as proof of improved performance.

Please review the proposed correction and consider an appropriate Debian update and upstream forwarding. Please let me know if you would prefer the process record to be allocated directly from the request pool instead of copying it at registration, or if additional regression-test coverage would be helpful.

Kind regards,
Sven Ruttloff
SaaS-Secure.com
--- a/mod_authnz_external.c
+++ b/mod_authnz_external.c
@@ -520,7 +520,10 @@
 
     if (isdaemon) return 0;
 
-    apr_pool_note_subprocess(p, &proc, APR_KILL_AFTER_TIMEOUT);
+    /* The request pool outlives this stack frame. APR retains the pointer,
+     * so its subprocess cleanup must refer to a pool-owned copy. */
+    apr_pool_note_subprocess(p, apr_pmemdup(p, &proc, sizeof(proc)),
+                            APR_KILL_AFTER_TIMEOUT);
 
     if (usepipein)
     {

Reply via email to