Re: T of 2.4.24

[Yann Ylavic]

On Thu, Dec 8, 2016 at 7:16 PM, William A Rowe Jr  wrote:
>
> It does raise the question again of whether the httpd project can distribute
> a source code package on www.apache.org/dist/httpd/ which is not voted
> on by the project, and whether it violates the spirit of the pmc consensus
> to no longer be the distributor of dependencies which frequently fall into
> a poorly maintained/updated state.

Current httpd-2.4.23-deps.tar.*/srclib seem to contain APR(-util)
only, no expat or PCRE, wasn't this decision taken already?

httpd-2.2.32.tar.*/srclib contain PCRE 5.0 (according to Changelog),
no expat, but it looks off topic for this T

Am I missing something?

Re: T of 2.4.24

[William A Rowe Jr]

On Thu, Dec 8, 2016 at 12:16 PM, William A Rowe Jr 
wrote:

> On Thu, Dec 8, 2016 at 12:03 PM, Jim Jagielski  wrote:
>
>> AFAICT there is no consensus. But is this really a blocker?
>
>
> I don't know, expat is at 2.2.0 and PCRE is at 8.39 with significant
> vulnerability
> fixes (everyone seems very enamored with fuzz generators this past few
> years.)
>
> It doesn't block creation of httpd-2.4.24.tar.gz, obviously.
>
> It does raise the question again of whether the httpd project can
> distribute
> a source code package on www.apache.org/dist/httpd/ which is not voted
> on by the project, and whether it violates the spirit of the pmc consensus
> to no longer be the distributor of dependencies which frequently fall into
> a poorly maintained/updated state.
>

@VP Legal, is this worth an escalation? You didn't see fit to respond today,
but I think this falls under the purview of your committee, w.r.t.
unapproved
release artifacts living at www.apache.org/dist/. Did you have any thoughts
or opinions one way or another?

Re: svn commit: r1773293 - /httpd/httpd/trunk/modules/http/http_filters.c

[William A Rowe Jr]

On Thu, Dec 8, 2016 at 1:57 PM,  wrote:

> Author: covener
> Date: Thu Dec  8 19:57:57 2016
> New Revision: 1773293
>
> URL: http://svn.apache.org/viewvc?rev=1773293=rev
> Log:
> change error handling for bad resp headers
>
>  - avoid looping between ap_die and the http filter
>  - remove the header that failed the check
>  - keep calling apr_table_do until our fn stops matching
>
>
> This is still not great. We get the original body, a 500 status
> code and status line.
>
> (r1773285 + fix for first return from check_headers)
>

I'm not clear how the original body makes any sense to send.
We should be eating the original body and sending a 500
ErrorDocument body.

If it isn't possible to do that, I see two alternatives;

1. clobber the bad header, simply omit it from the headers
which we transmit.

2. replace all bad characters in the header with some other
value, such as space characters. In the case of bad field
name characters, perhaps some permitted symbol such
as '$' (no, '?' isn't in the allowed list, that would be ideal.)

I don't see any obviously great solution.

Re: svn commit: r1773293 - /httpd/httpd/trunk/modules/http/http_filters.c

[Jacob Champion]

On 12/08/2016 02:06 PM, Eric Covener wrote:

On Thu, Dec 8, 2016 at 4:55 PM, Jacob Champion  wrote:

Hmm, I suppose that if something like Transfer-/Content-Encoding is one of
the headers to get axed here, we'll end up breaking the protocol entirely if
and when we push out the original response body...


Probably SOL already if they have bad bytes in them.


That's fair. I guess that makes me feel a bit better. :)

--Jacob

Re: svn commit: r1773293 - /httpd/httpd/trunk/modules/http/http_filters.c

[Eric Covener]

On Thu, Dec 8, 2016 at 4:55 PM, Jacob Champion  wrote:
> Hmm, I suppose that if something like Transfer-/Content-Encoding is one of
> the headers to get axed here, we'll end up breaking the protocol entirely if
> and when we push out the original response body...

Probably SOL already if they have bad bytes in them.


-- 
Eric Covener
cove...@gmail.com

Re: svn commit: r1773293 - /httpd/httpd/trunk/modules/http/http_filters.c

[Jacob Champion]

On 12/08/2016 11:57 AM, cove...@apache.org wrote:

Author: covener
Date: Thu Dec  8 19:57:57 2016
New Revision: 1773293

URL: http://svn.apache.org/viewvc?rev=1773293=rev
Log:
change error handling for bad resp headers

 - avoid looping between ap_die and the http filter
 - remove the header that failed the check
 - keep calling apr_table_do until our fn stops matching


This is still not great. We get the original body, a 500 status
code and status line.

(r1773285 + fix for first return from check_headers)



Modified:
httpd/httpd/trunk/modules/http/http_filters.c

Modified: httpd/httpd/trunk/modules/http/http_filters.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/http_filters.c?rev=1773293=1773292=1773293=diff
==
--- httpd/httpd/trunk/modules/http/http_filters.c (original)
+++ httpd/httpd/trunk/modules/http/http_filters.c Thu Dec  8 19:57:57 2016
@@ -632,6 +632,7 @@ apr_status_t ap_http_filter(ap_filter_t
 struct check_header_ctx {
 request_rec *r;
 int strict;
+const char *badheader;
 };

 /* check a single header, to be used with apr_table_do() */
@@ -643,6 +644,7 @@ static int check_header(void *arg, const
 if (name[0] == '\0') {
 ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, ctx->r, APLOGNO(02428)
   "Empty response header name, aborting request");
+ctx->badheader = name;
 return 0;
 }

@@ -657,6 +659,7 @@ static int check_header(void *arg, const
   "Response header name '%s' contains invalid "
   "characters, aborting request",
   name);
+ctx->badheader = name;
 return 0;
 }

@@ -666,6 +669,7 @@ static int check_header(void *arg, const
   "Response header '%s' value of '%s' contains invalid "
   "characters, aborting request",
   name, val);
+ctx->badheader = name;
 return 0;
 }
 return 1;
@@ -680,13 +684,21 @@ static APR_INLINE int check_headers(requ
 struct check_header_ctx ctx;
 core_server_config *conf =
 ap_get_core_module_config(r->server->module_config);
+int rv = 1;

 ctx.r = r;
 ctx.strict = (conf->http_conformance != AP_HTTP_CONFORMANCE_UNSAFE);
-if (!apr_table_do(check_header, , r->headers_out, NULL))
-return 0; /* problem has been logged by check_header() */
+ctx.badheader = NULL;

-return 1;
+while (!apr_table_do(check_header, , r->headers_out, NULL)){
+   if (ctx.badheader) {
+   apr_table_unset(r->headers_out, ctx.badheader);
+   apr_table_unset(r->err_headers_out, ctx.badheader);


Hmm, I suppose that if something like Transfer-/Content-Encoding is one 
of the headers to get axed here, we'll end up breaking the protocol 
entirely if and when we push out the original response body...


Ugh, this is tough.


+   }
+   rv = 0; /* problem has been logged by check_header() */
+}
+
+return rv;
 }

 typedef struct header_struct {
@@ -1249,8 +1261,7 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_
 }

 if (!check_headers(r)) {
-ap_die(HTTP_INTERNAL_SERVER_ERROR, r);
-return AP_FILTER_ERROR;
+r->status = 500;
 }

 /*

Re: svn commit: r1773163 - in /httpd/httpd/branches/2.4.x-merge-http-strict: ./ modules/http/http_filters.c

[William A Rowe Jr]

It should be complete enough to apply complete to httpd-2.4.x.




On Dec 8, 2016 1:32 PM, "Eric Covener"  wrote:

> On Wed, Dec 7, 2016 at 6:40 PM,   wrote:
> > Author: wrowe
> > Date: Wed Dec  7 23:40:20 2016
> > New Revision: 1773163
> >
> > URL: http://svn.apache.org/viewvc?rev=1773163=rev
> > Log:
> > After eliminating unusual whitespace in Unsafe mode (e.g. \f \v), we are
> left
> > with the same behavior in both of these cases. Simplify. Noted by rpluem.
> >
> > Backports: 1773162
> >
> > Modified:
> > httpd/httpd/branches/2.4.x-merge-http-strict/   (props changed)
> > httpd/httpd/branches/2.4.x-merge-http-strict/modules/
> http/http_filters.c
> >
>
> Will this branch outlive the backport?  I think the backport occurred
> a little before this rev.
>

Re: 2.4 HEAD: some looping issue with check_headers()

[Eric Covener]

On Thu, Dec 8, 2016 at 2:34 PM, Jacob Champion  wrote:
> On 12/08/2016 11:14 AM, Eric Covener wrote:
>>
>> On Thu, Dec 8, 2016 at 1:58 PM, William A Rowe Jr 
>> wrote:
>>>
>>> Still, I think we want to add a guard to rip out the offending header
>>> and not ap_die() in handling a 500 error, that's quite the loop, and
>>> if you could create the problem, so can another unsuspecting admin.
>>
>>
>> Is 500 status code and status line w/ original body but zapped "bad"
>> headers reasonable?
>
>
> Seems reasonable to me...
>
> Is there something lower-level than ap_die() that doesn't actually go back
> through the filter system again? Having strong guarantees on behavior would
> be useful in this situation.
>

I didn't see anything too promising.  I committed the ugly thing I had
because I will be tied up for a bit.

-- 
Eric Covener
cove...@gmail.com

Re: 2.4 HEAD: some looping issue with check_headers()

[Jacob Champion]

On 12/08/2016 11:14 AM, Eric Covener wrote:

On Thu, Dec 8, 2016 at 1:58 PM, William A Rowe Jr  wrote:

Still, I think we want to add a guard to rip out the offending header
and not ap_die() in handling a 500 error, that's quite the loop, and
if you could create the problem, so can another unsuspecting admin.


Is 500 status code and status line w/ original body but zapped "bad"
headers reasonable?


Seems reasonable to me...

Is there something lower-level than ap_die() that doesn't actually go 
back through the filter system again? Having strong guarantees on 
behavior would be useful in this situation.


--Jacob

Re: svn commit: r1773163 - in /httpd/httpd/branches/2.4.x-merge-http-strict: ./ modules/http/http_filters.c

[Eric Covener]

On Wed, Dec 7, 2016 at 6:40 PM,   wrote:
> Author: wrowe
> Date: Wed Dec  7 23:40:20 2016
> New Revision: 1773163
>
> URL: http://svn.apache.org/viewvc?rev=1773163=rev
> Log:
> After eliminating unusual whitespace in Unsafe mode (e.g. \f \v), we are left
> with the same behavior in both of these cases. Simplify. Noted by rpluem.
>
> Backports: 1773162
>
> Modified:
> httpd/httpd/branches/2.4.x-merge-http-strict/   (props changed)
> httpd/httpd/branches/2.4.x-merge-http-strict/modules/http/http_filters.c
>

Will this branch outlive the backport?  I think the backport occurred
a little before this rev.

Re: 2.4 HEAD: some looping issue with check_headers()

[Eric Covener]

On Thu, Dec 8, 2016 at 1:58 PM, William A Rowe Jr  wrote:
> Still, I think we want to add a guard to rip out the offending header
> and not ap_die() in handling a 500 error, that's quite the loop, and
> if you could create the problem, so can another unsuspecting admin.

Is 500 status code and status line w/ original body but zapped "bad"
headers reasonable?

-- 
Eric Covener
cove...@gmail.com

Re: 2.4 HEAD: some looping issue with check_headers()

[William A Rowe Jr]

On Thu, Dec 8, 2016 at 12:49 PM, Eric Covener  wrote:

> On Thu, Dec 8, 2016 at 1:46 PM, Eric Covener  wrote:
> > This is even after trying to zap the offending header before ap_die.
>
> I think this is more about how I am injecting the bad characters -- It
> is re-running after the ap_die.
>

Still, I think we want to add a guard to rip out the offending header
and not ap_die() in handling a 500 error, that's quite the loop, and
if you could create the problem, so can another unsuspecting admin.

Re: 2.4 HEAD: some looping issue with check_headers()

[Eric Covener]

On Thu, Dec 8, 2016 at 1:46 PM, Eric Covener  wrote:
> This is even after trying to zap the offending header before ap_die.

I think this is more about how I am injecting the bad characters -- It
is re-running after the ap_die.

-- 
Eric Covener
cove...@gmail.com

Re: T of 2.4.24

[Jim Jagielski]

Scratch that...

Instead, I plan on doing it on Monday, to provide some additional
time for some things to get locked down and resolved.

My apologies for those waiting for 2.4.24...

> On Dec 8, 2016, at 9:55 AM, Jim Jagielski  wrote:
> 
> Things are looking good for a T of 2.4.24 sometime late
> today.
> 
> If you have any issues or concerns, let me know asap.

Re: 2.4 HEAD: some looping issue with check_headers()

[Eric Covener]

I should mention I was actually "on" trunk because I thought there'd
be a quick fix.  But the relevant stuff is recently backported

On Thu, Dec 8, 2016 at 1:46 PM, Eric Covener  wrote:
> I am playing with check_headers() and injecting bad characters, and I
> am getting some looping:
>
> #2155 0x0048af80 in ap_http_header_filter (f=0x7fffc80061d0,
> b=0x7fffc09c2bd8) at http_filters.c:1262
> #2156 0x004396d0 in ap_pass_brigade (next=0x7fffc80061d0,
> bb=0x7fffc09c2bd8) at util_filter.c:610
> #2157 0x004408aa in ap_content_length_filter
> (f=0x7fffc8006198, b=0x7fffc09c2bd8) at protocol.c:1776
> #2158 0x004396d0 in ap_pass_brigade (next=0x7fffc8006198,
> bb=0x7fffc09c2bd8) at util_filter.c:610
> #2159 0x7fffe838daa5 in ap_headers_error_filter (f=0x7fffc09c2948,
> in=0x7fffc09c2bd8) at mod_headers.c:917
> #2160 0x004396d0 in ap_pass_brigade (next=0x7fffc09c2948,
> bb=0x7fffc09c2bd8) at util_filter.c:610
> #2161 0x7fffe6103dfc in session_output_filter (f=0x7fffc09c2910,
> in=0x7fffc09c2bd8) at mod_session.c:489
> #2162 0x004396d0 in ap_pass_brigade (next=0x7fffc09c2910,
> bb=0x7fffc09c2bd8) at util_filter.c:610
> #2163 0x00440b5e in ap_old_write_filter (f=0x7fffc09c29e0,
> bb=0x7fffc09c2bd8) at protocol.c:1845
> #2164 0x004396d0 in ap_pass_brigade (next=0x7fffc09c29e0,
> bb=0x7fffc09c2bd8) at util_filter.c:610
> #2165 0x004400a7 in end_output_stream (r=0x7fffc8004980) at
> protocol.c:1540
> #2166 0x0044011f in ap_finalize_request_protocol
> (r=0x7fffc8004980) at protocol.c:1565
> #2167 0x00486612 in ap_send_error_response (r=0x7fffc8004980,
> recursive_error=500) at http_protocol.c:1395
> #2168 0x00486da8 in ap_die_r (type=500, r=0x7fffc8004980,
> recursive_error=500) at http_request.c:224
> #2169 0x00486dd4 in ap_die (type=500, r=0x7fffc8004980) at
> http_request.c:229
> #2170 0x0048af80 in ap_http_header_filter (f=0x7fffc80061d0,
> b=0x7fffc09c2798) at http_filters.c:1262
>
> This is even after trying to zap the offending header before ap_die.
> --
> Eric Covener
> cove...@gmail.com



-- 
Eric Covener
cove...@gmail.com

2.4 HEAD: some looping issue with check_headers()

[Eric Covener]

I am playing with check_headers() and injecting bad characters, and I
am getting some looping:

#2155 0x0048af80 in ap_http_header_filter (f=0x7fffc80061d0,
b=0x7fffc09c2bd8) at http_filters.c:1262
#2156 0x004396d0 in ap_pass_brigade (next=0x7fffc80061d0,
bb=0x7fffc09c2bd8) at util_filter.c:610
#2157 0x004408aa in ap_content_length_filter
(f=0x7fffc8006198, b=0x7fffc09c2bd8) at protocol.c:1776
#2158 0x004396d0 in ap_pass_brigade (next=0x7fffc8006198,
bb=0x7fffc09c2bd8) at util_filter.c:610
#2159 0x7fffe838daa5 in ap_headers_error_filter (f=0x7fffc09c2948,
in=0x7fffc09c2bd8) at mod_headers.c:917
#2160 0x004396d0 in ap_pass_brigade (next=0x7fffc09c2948,
bb=0x7fffc09c2bd8) at util_filter.c:610
#2161 0x7fffe6103dfc in session_output_filter (f=0x7fffc09c2910,
in=0x7fffc09c2bd8) at mod_session.c:489
#2162 0x004396d0 in ap_pass_brigade (next=0x7fffc09c2910,
bb=0x7fffc09c2bd8) at util_filter.c:610
#2163 0x00440b5e in ap_old_write_filter (f=0x7fffc09c29e0,
bb=0x7fffc09c2bd8) at protocol.c:1845
#2164 0x004396d0 in ap_pass_brigade (next=0x7fffc09c29e0,
bb=0x7fffc09c2bd8) at util_filter.c:610
#2165 0x004400a7 in end_output_stream (r=0x7fffc8004980) at
protocol.c:1540
#2166 0x0044011f in ap_finalize_request_protocol
(r=0x7fffc8004980) at protocol.c:1565
#2167 0x00486612 in ap_send_error_response (r=0x7fffc8004980,
recursive_error=500) at http_protocol.c:1395
#2168 0x00486da8 in ap_die_r (type=500, r=0x7fffc8004980,
recursive_error=500) at http_request.c:224
#2169 0x00486dd4 in ap_die (type=500, r=0x7fffc8004980) at
http_request.c:229
#2170 0x0048af80 in ap_http_header_filter (f=0x7fffc80061d0,
b=0x7fffc09c2798) at http_filters.c:1262

This is even after trying to zap the offending header before ap_die.
-- 
Eric Covener
cove...@gmail.com

Re: T of 2.4.24

[William A Rowe Jr]

On Thu, Dec 8, 2016 at 12:03 PM, Jim Jagielski  wrote:

> AFAICT there is no consensus. But is this really a blocker?


I don't know, expat is at 2.2.0 and PCRE is at 8.39 with significant
vulnerability
fixes (everyone seems very enamored with fuzz generators this past few
years.)

It doesn't block creation of httpd-2.4.24.tar.gz, obviously.

It does raise the question again of whether the httpd project can distribute
a source code package on www.apache.org/dist/httpd/ which is not voted
on by the project, and whether it violates the spirit of the pmc consensus
to no longer be the distributor of dependencies which frequently fall into
a poorly maintained/updated state.

So it's simply a question about the -deps package, and since that is never
given a release vote, it really isn't holding up any tag & roll.



> > On Dec 8, 2016, at 12:38 PM, William A Rowe Jr 
> wrote:
> >
> > On Thu, Dec 8, 2016 at 8:55 AM, Jim Jagielski  wrote:
> > Things are looking good for a T of 2.4.24 sometime late
> > today.
> >
> > If you have any issues or concerns, let me know asap.
> >
> > Do we have any consensus on dropping the stale and vulnerable
> > expat or pcre packages from the pretending-not-to-be-released
> > -deps artifact in the www.a.o/dist/httpd/ releases tree?
> >
> >
> >
>
>

Re: T of 2.4.24

[Jim Jagielski]

AFAICT there is no consensus. But is this really a
blocker?

> On Dec 8, 2016, at 12:38 PM, William A Rowe Jr  wrote:
> 
> On Thu, Dec 8, 2016 at 8:55 AM, Jim Jagielski  wrote:
> Things are looking good for a T of 2.4.24 sometime late
> today.
> 
> If you have any issues or concerns, let me know asap.
> 
> Do we have any consensus on dropping the stale and vulnerable
> expat or pcre packages from the pretending-not-to-be-released 
> -deps artifact in the www.a.o/dist/httpd/ releases tree?
> 
> 
> 

Re: T of 2.4.24

[William A Rowe Jr]

On Thu, Dec 8, 2016 at 8:55 AM, Jim Jagielski  wrote:

> Things are looking good for a T of 2.4.24 sometime late
> today.
>
> If you have any issues or concerns, let me know asap.
>

Do we have any consensus on dropping the stale and vulnerable
expat or pcre packages from the pretending-not-to-be-released
-deps artifact in the www.a.o/dist/httpd/ releases tree?

Re: PCRE 10 and puzzling edge cases

[William A Rowe Jr]

I've beaten my head against this wall for a bit longer, and came up with
several places where pcre2 changed return types for void *what query
interogations (especially from int to uint32, badness on x86_64-linux).

The attached patch picks up these bad void * type assignments. Still
no tremendous improvement, missing something blatantly obvious,
I expect.



On Mon, Dec 5, 2016 at 10:59 PM, William A Rowe Jr 
wrote:

> I've written the following patch to trunk to allow us to configure,
> compile and link against PCRE2 (10.x). The autoconf in particular is
> streamlined for cross-compilation detection, while retaining the ability to
> override the path to (and name of) pcre[2]-config.
>
> It isn't in a commit-ready state due to t/TEST t/apache/expr.t failures
> (among others), and the defects appear to revolve around the way substring
> patterns are recorded.
>
> Attached the test failure cases (many similar test patterns do succeed,
> interestingly.) One test looks outright wrong. I'd rather not beat my head
> against these if the answer is blatantly obvious.
>
> If anyone has patience for exploring this further, any help is welcomed.
> Philip starts with this assertion; "The original, very widely deployed PCRE
> library, originally released in 1997, is at version 8.39, and the API and
> feature set are stable—future releases will be for bugfixes only. All new
> future features will be to PCRE2, not the original PCRE 8.x series." But he
> has gone on to state that many fuzzing error cases which are handled
> correctly in PCRE2 cannot be realistically fixed in PCRE 8.x. I've placed
> this up there with other parsing rewrites in httpd, that starting over is
> simply the correct answer, and I'd like to see if we can have httpd 3.0
> choosing PCRE2 over PCRE in the near future (and perhaps backport this if
> we determine behavior is consistent.)
>
> Cheers,
>
> Bill
>
>
Index: configure.in
===
--- configure.in	(revision 1773161)
+++ configure.in	(working copy)
@@ -223,18 +223,18 @@
 AC_ARG_WITH(pcre,
 APACHE_HELP_STRING(--with-pcre=PATH,Use external PCRE library))
 
-AC_PATH_PROG(PCRE_CONFIG, pcre-config, false)
-if test -d "$with_pcre" && test -x "$with_pcre/bin/pcre-config"; then
-   PCRE_CONFIG=$with_pcre/bin/pcre-config
-elif test -x "$with_pcre"; then
-   PCRE_CONFIG=$with_pcre
-fi
+AC_CHECK_TARGET_TOOLS(PCRE_CONFIG, [pcre2-config pcre-config],
+  [`which $with_pcre 2>/dev/null`],
+  [$with_pcre/bin:$with_pcre])
 
-if test "$PCRE_CONFIG" != "false"; then
+if test "x$PCRE_CONFIG" != "x"; then
   if $PCRE_CONFIG --version >/dev/null 2>&1; then :; else
-AC_MSG_ERROR([Did not find pcre-config script at $PCRE_CONFIG])
+AC_MSG_ERROR([Did not find working script at $PCRE_CONFIG])
   fi
   case `$PCRE_CONFIG --version` in
+  [1[0-9].*])
+AC_DEFINE(HAVE_PCRE2, 1, [Detected PCRE2]) 
+;;
   [[1-5].*])
 AC_MSG_ERROR([Need at least pcre version 6.7])
 ;;
@@ -244,10 +244,10 @@
   esac
   AC_MSG_NOTICE([Using external PCRE library from $PCRE_CONFIG])
   APR_ADDTO(PCRE_INCLUDES, [`$PCRE_CONFIG --cflags`])
-  APR_ADDTO(PCRE_LIBS, [`$PCRE_CONFIG --libs`])
+  APR_ADDTO(PCRE_LIBS, [`$PCRE_CONFIG --libs8 2>/dev/null || $PCRE_CONFIG --libs`])
   APR_ADDTO(HTTPD_LIBS, [\$(PCRE_LIBS)])
 else
-  AC_MSG_ERROR([pcre-config for libpcre not found. PCRE is required and available from http://pcre.org/])
+  AC_MSG_ERROR([pcre(2)-config for libpcre not found. PCRE is required and available from http://pcre.org/])
 fi
 APACHE_SUBST(PCRE_LIBS)
 
Index: server/util_pcre.c
===
--- server/util_pcre.c	(revision 1773161)
+++ server/util_pcre.c	(working copy)
@@ -46,10 +46,18 @@
 #include "httpd.h"
 #include "apr_strings.h"
 #include "apr_tables.h"
+
+#ifdef HAVE_PCRE2
+#define PCRE2_CODE_UNIT_WIDTH 8
+#include "pcre2.h"
+#define PCREn(x) PCRE2_ ## x
+#else
 #include "pcre.h"
+#define PCREn(x) PCRE_ ## x
+#endif
 
 /* PCRE_DUPNAMES is only present since version 6.7 of PCRE */
-#ifndef PCRE_DUPNAMES
+#if !defined(PCRE_DUPNAMES) && !defined(HAVE_PCRE2)
 #error PCRE Version 6.7 or later required!
 #else
 
@@ -74,11 +82,19 @@
 
 AP_DECLARE(const char *) ap_pcre_version_string(int which)
 {
+#ifdef HAVE_PCRE2
+static char buf[80];
+#endif
 switch (which) {
 case AP_REG_PCRE_COMPILED:
-return APR_STRINGIFY(PCRE_MAJOR) "." APR_STRINGIFY(PCRE_MINOR) " " APR_STRINGIFY(PCRE_DATE);
+return APR_STRINGIFY(PCREn(MAJOR)) "." APR_STRINGIFY(PCREn(MINOR)) " " APR_STRINGIFY(PCREn(DATE));
 case AP_REG_PCRE_LOADED:
+#ifdef HAVE_PCRE2
+pcre2_config(PCRE2_CONFIG_VERSION, buf);
+return buf;
+#else
 return pcre_version();
+#endif
 default:
 return "Unknown";
 }
@@ -118,7 +134,11 @@
 
 AP_DECLARE(void) ap_regfree(ap_regex_t *preg)
 {
+#ifdef HAVE_PCRE2
+pcre2_code_free(preg->re_pcre);

T of 2.4.24

[Jim Jagielski]

Things are looking good for a T of 2.4.24 sometime late
today.

If you have any issues or concerns, let me know asap.

Re: Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

Hi everybody,

thanks a lot for the useful feedback. Quoting Jacob from another thread:

2016-12-08 0:04 GMT+01:00 Jacob Champion :
>
>
> I thought the original bug report was related to 204 processing only, and
> then Luca asked if his patch should also include informational-status
> checks as well.


My understanding of the 1xx status handling was not correct, I created a
new patch only for the 204 bug:

http://home.apache.org/~elukey/httpd-trunk-core-http_204_nocontent.patch

This should avoid returning a C-L header (set by ap_content_length_filter
or a CGI backend) and any message-body for HTTP 204 responses.

If nobody disagrees, I'd like to commit it and create a specific test in
the httpd test suite.

Thanks!

Luca

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

[Stefan Eissing]

> Am 08.12.2016 um 11:25 schrieb Yann Ylavic :
> 
> On Thu, Dec 8, 2016 at 3:12 AM, William A Rowe Jr  wrote:
>> On Dec 7, 2016 6:23 PM, "Jacob Champion"  wrote:
>> 
>> On 12/07/2016 04:00 PM, William A Rowe Jr wrote:
>>> 
>>> Consider for a moment the case of an HTTP/1.1 upgrade request
>>> unrecognized by a proxy agent.
>> 
>> 
>> It was my understanding that this is an impossible situation for a
>> conforming proxy, since Upgrade is hop-by-hop. What am I missing?
>> 
>> 
>> The fact that there is no way for us to predict what new headers we are
>> passed in the future are defined in the future to be hop-by-hop, which
>> result in a 105 response code with a similarly imponderable conundrum. No
>> way for RFC2068 servers to know 101 became a hop by hop unhandleable
>> response.
> 
> "Connection: Upgrade, ..." is a MUST when the Upgrade header is used,
> so servers/proxies do know.

Exactly.

There are three things mixed in this dicussion:
1. what the HTTP/1.1 protocol allows and asks us to do
2. what we need to change for 2.4.24
3. what our implementation should be in the future

As to 1: 
basically https://tools.ietf.org/html/rfc7231#section-6.2 says it all. HTTP/1.1 
defines 1xx interim responses and requires clients, on receiving them, to act 
on the known and ignore the unknown ones and continue processing. 

For HTTP/1.1, as a server, we should not generate 1xx interims on HTTP/1.1 
connections without a really good reason. There is high probability on unasked 
and especially unknown 1xx breakage in HTTP/1.1 clients and proxies. 

In HTTP/2, we are pushing to change that behaviour, due to the opportunities 
that 1xx responses present and the relatively small set of implementations. 
mod_http2 does fully support it in the upcoming 2.4.24.

As to 2:
I agree that our interim handling and generation could do with some love. But 
it is working well enough, from my point of view, that I would say we can 
release a 2.4.24 on it. Even the one item that William mentioned, a HTTP/1.1 
client talking via mod_proxy_http2 to a backend, works correctly. 
mod_proxy_http2 does not forward 1xx responses to a HTTP/1.1 main connection.

As to 3:
Possibilities are endless. From my PoV it was not simple to track down the 
working of our implementation when I worked on mod_http2 and mod_proxy_http2 
and on the 1xx dependencies between them. I agree with the comments made that 
it is not clear which header are send out under which processing phase. However 
I think the possibility of modules/hooks to add headers to 1xx response should 
be there. The protocol allows it and we cannot peak into the future and see all 
uses cases. Just look at 103 and the Link header.

What is unsatisfying from HTTP/2 implementation PoV is that interim responses 
need to be serialized and parsed again. That is a course for breakage and, 
since mod_http2 had to make its own parser, needs to be fixed. In the 
beginning, I though I could intercept the 1xx generation and skip the parsing. 
However some modules like mod_proxy_http sends down a HTTP/1.1 serialized 
format.

-Stefan

Re: About Interim Response Headers (was: Content-Length header for 1xx status codes)

[Yann Ylavic]

On Thu, Dec 8, 2016 at 3:12 AM, William A Rowe Jr  wrote:
> On Dec 7, 2016 6:23 PM, "Jacob Champion"  wrote:
>
> On 12/07/2016 04:00 PM, William A Rowe Jr wrote:
>>
>> Consider for a moment the case of an HTTP/1.1 upgrade request
>> unrecognized by a proxy agent.
>
>
> It was my understanding that this is an impossible situation for a
> conforming proxy, since Upgrade is hop-by-hop. What am I missing?
>
>
> The fact that there is no way for us to predict what new headers we are
> passed in the future are defined in the future to be hop-by-hop, which
> result in a 105 response code with a similarly imponderable conundrum. No
> way for RFC2068 servers to know 101 became a hop by hop unhandleable
> response.

"Connection: Upgrade, ..." is a MUST when the Upgrade header is used,
so servers/proxies do know.