As recently disclosed in; https://httpd.apache.org/security/vulnerabilities_24.html#CVE-2016-4975
Sergey Bobrov brought this disclosure of a moderate vulnerability in mod_userdir to the httpd project security@ team. Given various examples of invalid and valid request URL's, even the valid input could inject a %0D or %0A URL sequence as a raw byte into the response headers stream. The actual %0D or %0A are valid request URL bytes, but their decoded forms must not be exchanged over HTTP. This behavior pattern is not isolated to mod_userdir, and a fix to the userdir module alone would be insufficient to address the class of issues. When the project addressed bad request input in 2.4.25, there was a corresponding change (with further cleanup in 2.4.26) to review the resulting response output headers, and reject un-encoded ctrl characters as 500 responses (module/app/script error, uncorrectable by the user-agent). That change provided one last backstop against module processing of decoded input resulting in control characters injected in output headers with malicious intent. Note that response filter/rejection did not address a specific CVE and was not a security bugfix, but it was a mitigation that worked into the correction of many actual request handling security defects addressed by; https://httpd.apache.org/security/vulnerabilities_24.html#CVE-2016-8743 This change, in 2.4.25 and 2.4.26 (with similar fixes in 2.2.final), were collected here from a great number of commits assembled publicly, without comment as to their impact on security during the review process; https://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x-merge-http-strict/ This broad refactoring for strictness was also supported with the vulnerability reports and observations presented to security@httpd by Régis Leroy, and by Peter Allor and his colleagues at IBM. It is worth also noting that the bulk of this refactoring had already long existed in trunk by the efforts of committer Stefan Fritsch (sf) to introduce RFC compliance checks, which under usual principals could not have been backported. (Many sites adopting this fix first encountered significant breakage and fallout in their custom, in-house tooling which was never RFC compliant and relied on the server's permissiveness of LF line endings. In a normal scenario, such behavior changes are deferred to 2.next.) This is not the end of the list of issues. Although it failed to gain traction as a distinct CVE, June 14th of '16, Sergey Bobrov brought yet another vulnerability to the security@httpd team in the handling of the RedirectMatch transformations, yielding the same results through a different defect in a distinct module. The confusion arose as some security team participants viewed these one-in-the-same in the realm of bad output. Jacob Champion and I further reviewed a great number of mappings that occur throughout the httpd code base, and that audit convinced the security team that mitigating the results, and distributing this workaround broadly and widely, was more important than correcting the individual defects. So over a year after distribution of the mitigation, there are two groups of action items to address next; 1. There are several patches which will be suggested to the many redirection modules, to ensure control chars are re-encoded, and those request responses will no longer return 500 errors, but will return location: headers with properly escaped ctrl characters. There are further audit results which can be reconsidered. We can do this now at a measured and public pace. 2. The underlying API, decoding %-escapes early, lends itself to the impossibility of handling every request correctly. The only way to fix this is to break several assumptions by module authors, and change the actual definitions of several URL-related structure members. The encoded/decoded state needs to be better defined, and better APIs to manipulate the URL need to be introduced. We cannot do this in 2.4.x, but it needs to be incorporated in whatever 2.next (or 3.0) follows the current maintenance release. Thank you for taking the time to read this background, we will be forwarding various ideas from the security@ private list here to spark discussion around revamping the URL manipulation API, and pushing forward the true corrections of the underlying URL escaping bugs over the coming weeks.
