Re: svn commit: r1808230 - /httpd/httpd/trunk/server/protocol.c
On Fri, Sep 22, 2017 at 1:02 PM, Joe Ortonwrote: > On Fri, Sep 22, 2017 at 11:39:54AM -0500, William A Rowe Jr wrote: >> This defect still appears to exist in 2.4.28-dev, no? >> >> The rewrite appears to have enjoyed both committer and external testing and >> the patch looks suitable for backport. It has enjoyed careful consideration >> by >> at least four committers. >> >> Reading https://bz.apache.org/bugzilla/show_bug.cgi?id=61222#c19 Joe was >> eyeing some additional improvements, but it seems worthwhile to get this >> defect fixed in today's T >> >> Joe, is there a reason to hold on backporting, why this hasn't been promoted >> to 2.4 STATUS? If you are satisfied, here's my +1 for the backport to speed >> things up. > > I don't plan any additional changes, no. But I'm not very confident we > should be throwing a major rewrite of a core filter at 2.4 users with > only light testing, especially since there are security fixes pending. > > I have put this patch in Fedora "Raw Hide" builds to give some extra > exposure, and I'd love to hear more testing results here. Given that the > bug has sat festering for a long time (maybe since 2.2??) I don't see > any urgency, I'd rather get a bit more testing and wait until after .28 > to ship and avoid regressions. Cool, add my +1 into your STATUS proposal once 2.4.29-dev rolls around, and let's let this live on the maintenance dev branch as long as possible to pick up any regressions. Thanks for all your effort on this!
Re: svn commit: r1808230 - /httpd/httpd/trunk/server/protocol.c
On Fri, Sep 22, 2017 at 11:39:54AM -0500, William A Rowe Jr wrote: > This defect still appears to exist in 2.4.28-dev, no? > > The rewrite appears to have enjoyed both committer and external testing and > the patch looks suitable for backport. It has enjoyed careful consideration by > at least four committers. > > Reading https://bz.apache.org/bugzilla/show_bug.cgi?id=61222#c19 Joe was > eyeing some additional improvements, but it seems worthwhile to get this > defect fixed in today's T > > Joe, is there a reason to hold on backporting, why this hasn't been promoted > to 2.4 STATUS? If you are satisfied, here's my +1 for the backport to speed > things up. I don't plan any additional changes, no. But I'm not very confident we should be throwing a major rewrite of a core filter at 2.4 users with only light testing, especially since there are security fixes pending. I have put this patch in Fedora "Raw Hide" builds to give some extra exposure, and I'd love to hear more testing results here. Given that the bug has sat festering for a long time (maybe since 2.2??) I don't see any urgency, I'd rather get a bit more testing and wait until after .28 to ship and avoid regressions. Regards, Joe > On Wed, Sep 13, 2017 at 5:59 AM,wrote: > > Author: jorton > > Date: Wed Sep 13 10:59:51 2017 > > New Revision: 1808230 > > > > URL: http://svn.apache.org/viewvc?rev=1808230=rev > > Log: > > * server/protocol.c (ap_content_length_filter): Rewrite the content > > length filter to avoid arbitrary memory consumption for streaming > > responses (e.g. large CGI script output). Ensures C-L is still > > generated in common cases (static content, small CGI script output), > > but this DOES change behaviour and some responses will end up > > chunked rather than C-L computed. > > > > PR: 61222 > > Submitted by: jorton, rpluem
Re: Time for 2.4.28 ?
On Fri, Sep 22, 2017 at 1:25 PM, Jim Jagielskiwrote: > We can wait. No reason to rush if we can hold off for a bit > and ensure that 2.4.28 is as ready to go as possible. Since the C-L issue is not a (recent) regression, I would just as well bank the aging CVE fix now with a release rather than waiting for a potentially destabilizing change.
Re: Time for 2.4.28 ?
We can wait. No reason to rush if we can hold off for a bit and ensure that 2.4.28 is as ready to go as possible. > On Sep 22, 2017, at 1:23 PM, William A Rowe Jrwrote: > > On Fri, Sep 22, 2017 at 7:06 AM, Jim Jagielski wrote: >> STATUS looks clean. >> >> Hoping to do a T this afternoon, eastern, unless I hear >> any objections or concerns re: timing. > > svn looks good here. Only one potentially missed item IMO, it could wait > till 2.4.29, but if we hear right back from jorton or rpluem on the readiness > of the patch, I'd love to see us quit consuming extra memory r.e. PR61222, > so the backport has my +1 already. If you look at comments in this ticket, > the review has already happened and everyone seemed mostly satisfied. > > Suggesting a parallel apr/-util release over on dev@apr. Thanks for RM'ing!
Re: Time for 2.4.28 ?
On Fri, Sep 22, 2017 at 7:06 AM, Jim Jagielskiwrote: > STATUS looks clean. > > Hoping to do a T this afternoon, eastern, unless I hear > any objections or concerns re: timing. svn looks good here. Only one potentially missed item IMO, it could wait till 2.4.29, but if we hear right back from jorton or rpluem on the readiness of the patch, I'd love to see us quit consuming extra memory r.e. PR61222, so the backport has my +1 already. If you look at comments in this ticket, the review has already happened and everyone seemed mostly satisfied. Suggesting a parallel apr/-util release over on dev@apr. Thanks for RM'ing!
Re: svn commit: r1808230 - /httpd/httpd/trunk/server/protocol.c
This defect still appears to exist in 2.4.28-dev, no? The rewrite appears to have enjoyed both committer and external testing and the patch looks suitable for backport. It has enjoyed careful consideration by at least four committers. Reading https://bz.apache.org/bugzilla/show_bug.cgi?id=61222#c19 Joe was eyeing some additional improvements, but it seems worthwhile to get this defect fixed in today's T Joe, is there a reason to hold on backporting, why this hasn't been promoted to 2.4 STATUS? If you are satisfied, here's my +1 for the backport to speed things up. On Wed, Sep 13, 2017 at 5:59 AM,wrote: > Author: jorton > Date: Wed Sep 13 10:59:51 2017 > New Revision: 1808230 > > URL: http://svn.apache.org/viewvc?rev=1808230=rev > Log: > * server/protocol.c (ap_content_length_filter): Rewrite the content > length filter to avoid arbitrary memory consumption for streaming > responses (e.g. large CGI script output). Ensures C-L is still > generated in common cases (static content, small CGI script output), > but this DOES change behaviour and some responses will end up > chunked rather than C-L computed. > > PR: 61222 > Submitted by: jorton, rpluem > > Modified: > httpd/httpd/trunk/server/protocol.c > > Modified: httpd/httpd/trunk/server/protocol.c > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/server/protocol.c?rev=1808230=1808229=1808230=diff > == > --- httpd/httpd/trunk/server/protocol.c (original) > +++ httpd/httpd/trunk/server/protocol.c Wed Sep 13 10:59:51 2017 > @@ -1708,62 +1708,88 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_ > ctx->tmpbb = apr_brigade_create(r->pool, > r->connection->bucket_alloc); > } > > -/* Loop through this set of buckets to compute their length > - */ > +/* Loop through the brigade to count the length. To avoid > + * arbitrary memory consumption with morphing bucket types, this > + * loop will stop and pass on the brigade when necessary. */ > e = APR_BRIGADE_FIRST(b); > while (e != APR_BRIGADE_SENTINEL(b)) { > +apr_status_t rv; > + > if (APR_BUCKET_IS_EOS(e)) { > eos = 1; > break; > } > -if (e->length == (apr_size_t)-1) { > +/* For a flush bucket, fall through to pass the brigade and > + * flush now. */ > +else if (APR_BUCKET_IS_FLUSH(e)) { > +e = APR_BUCKET_NEXT(e); > +} > +/* For metadata bucket types other than FLUSH, loop. */ > +else if (APR_BUCKET_IS_METADATA(e)) { > +e = APR_BUCKET_NEXT(e); > +continue; > +} > +/* For determinate length data buckets, count the length and > + * continue. */ > +else if (e->length != (apr_size_t)-1) { > +r->bytes_sent += e->length; > +e = APR_BUCKET_NEXT(e); > +continue; > +} > +/* For indeterminate length data buckets, perform one read. */ > +else /* e->length == (apr_size_t)-1 */ { > apr_size_t len; > const char *ignored; > -apr_status_t rv; > - > -/* This is probably a pipe bucket. Send everything > - * prior to this, and then read the data for this bucket. > - */ > + > rv = apr_bucket_read(e, , , eblock); > +if ((rv != APR_SUCCESS) && !APR_STATUS_IS_EAGAIN(rv)) { > +ap_log_rerror(APLOG_MARK, APLOG_ERR, rv, r, APLOGNO(00574) > + "ap_content_length_filter: " > + "apr_bucket_read() failed"); > +return rv; > +} > if (rv == APR_SUCCESS) { > -/* Attempt a nonblocking read next time through */ > eblock = APR_NONBLOCK_READ; > +e = APR_BUCKET_NEXT(e); > r->bytes_sent += len; > } > else if (APR_STATUS_IS_EAGAIN(rv)) { > -/* Output everything prior to this bucket, and then > - * do a blocking read on the next batch. > - */ > -if (e != APR_BRIGADE_FIRST(b)) { > -apr_bucket *flush; > -apr_brigade_split_ex(b, e, ctx->tmpbb); > -flush = > apr_bucket_flush_create(r->connection->bucket_alloc); > - > -APR_BRIGADE_INSERT_TAIL(b, flush); > -rv = ap_pass_brigade(f->next, b); > -if (rv != APR_SUCCESS || f->c->aborted) { > -return rv; > -} > -apr_brigade_cleanup(b); > -APR_BRIGADE_CONCAT(b, ctx->tmpbb); > -e = APR_BRIGADE_FIRST(b); > +apr_bucket *flush; > > -ctx->data_sent = 1; > -} > +/* Next read must
Re: SSLSrvConfigRec shared
I added the 'flags' getter in r1809311, much cleaner, thanks! On Fri, Sep 22, 2017 at 2:48 PM, Eric Covenerwrote: > Whoops I see you already folllowed it up. > > On Fri, Sep 22, 2017 at 8:46 AM, Eric Covener wrote: >> On Fri, Sep 22, 2017 at 8:11 AM, Yann Ylavic wrote: >>> On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: > On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: >> >> IIUC it should be safe to extend module_struct with a minor bump to >> add 'int flags' to the bottom, but when you check the value you'd need >> to check the MMN first. In the module you'd then just have some flags >> or'ed together after register_hooks. > > Something like the attached patch might do it (untested, no MMN minor > bump). > >> >> (hopefully someone will check my work) > > Since modules (module_struct) are déclared globally, unspecified > fields at the end of the struct should be initialized to zero, so it > should be safe. I was thinking about modules compiled against the previous definition / out of tree. >>> >>> Hmm, I'm not sure my commits address this. >>> >>> The modules would be run by the latest core without being recompiled >>> against it, that's the case? >> >> Yes, I think it not yet handled because you're checking the cores MMN >> # at compile time. >> >> I think we need an accessor or macro to retrieve the flags that looks >> at the module_struct being evaluated which I think also has their >> compile-time MMN baked in. Probably best to have this be a simple >> function rather than a macro. >> >> >> -- >> Eric Covener >> cove...@gmail.com > > > > -- > Eric Covener > cove...@gmail.com
Re: SSLSrvConfigRec shared
posticipate - realizing, while one writes a reply, that Yann has probably already implemented it. X-) > Am 22.09.2017 um 14:48 schrieb Eric Covener: > > Whoops I see you already folllowed it up. > > On Fri, Sep 22, 2017 at 8:46 AM, Eric Covener wrote: >> On Fri, Sep 22, 2017 at 8:11 AM, Yann Ylavic wrote: >>> On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: > On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: >> >> IIUC it should be safe to extend module_struct with a minor bump to >> add 'int flags' to the bottom, but when you check the value you'd need >> to check the MMN first. In the module you'd then just have some flags >> or'ed together after register_hooks. > > Something like the attached patch might do it (untested, no MMN minor > bump). > >> >> (hopefully someone will check my work) > > Since modules (module_struct) are déclared globally, unspecified > fields at the end of the struct should be initialized to zero, so it > should be safe. I was thinking about modules compiled against the previous definition / out of tree. >>> >>> Hmm, I'm not sure my commits address this. >>> >>> The modules would be run by the latest core without being recompiled >>> against it, that's the case? >> >> Yes, I think it not yet handled because you're checking the cores MMN >> # at compile time. >> >> I think we need an accessor or macro to retrieve the flags that looks >> at the module_struct being evaluated which I think also has their >> compile-time MMN baked in. Probably best to have this be a simple >> function rather than a macro. >> >> >> -- >> Eric Covener >> cove...@gmail.com > > > > -- > Eric Covener > cove...@gmail.com
Re: SSLSrvConfigRec shared
Whoops I see you already folllowed it up. On Fri, Sep 22, 2017 at 8:46 AM, Eric Covenerwrote: > On Fri, Sep 22, 2017 at 8:11 AM, Yann Ylavic wrote: >> On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: >>> On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: > > IIUC it should be safe to extend module_struct with a minor bump to > add 'int flags' to the bottom, but when you check the value you'd need > to check the MMN first. In the module you'd then just have some flags > or'ed together after register_hooks. Something like the attached patch might do it (untested, no MMN minor bump). > > (hopefully someone will check my work) Since modules (module_struct) are déclared globally, unspecified fields at the end of the struct should be initialized to zero, so it should be safe. >>> >>> I was thinking about modules compiled against the previous definition >>> / out of tree. >> >> Hmm, I'm not sure my commits address this. >> >> The modules would be run by the latest core without being recompiled >> against it, that's the case? > > Yes, I think it not yet handled because you're checking the cores MMN > # at compile time. > > I think we need an accessor or macro to retrieve the flags that looks > at the module_struct being evaluated which I think also has their > compile-time MMN baked in. Probably best to have this be a simple > function rather than a macro. > > > -- > Eric Covener > cove...@gmail.com -- Eric Covener cove...@gmail.com
Re: SSLSrvConfigRec shared
On Fri, Sep 22, 2017 at 8:11 AM, Yann Ylavicwrote: > On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: >> On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: >>> On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: IIUC it should be safe to extend module_struct with a minor bump to add 'int flags' to the bottom, but when you check the value you'd need to check the MMN first. In the module you'd then just have some flags or'ed together after register_hooks. >>> >>> Something like the attached patch might do it (untested, no MMN minor bump). >>> (hopefully someone will check my work) >>> >>> Since modules (module_struct) are déclared globally, unspecified >>> fields at the end of the struct should be initialized to zero, so it >>> should be safe. >> >> I was thinking about modules compiled against the previous definition >> / out of tree. > > Hmm, I'm not sure my commits address this. > > The modules would be run by the latest core without being recompiled > against it, that's the case? Yes, I think it not yet handled because you're checking the cores MMN # at compile time. I think we need an accessor or macro to retrieve the flags that looks at the module_struct being evaluated which I think also has their compile-time MMN baked in. Probably best to have this be a simple function rather than a macro. -- Eric Covener cove...@gmail.com
Re: svn commit: r1809302 - in /httpd/httpd/trunk: include/ap_mmn.h include/http_config.h server/config.c
On Fri, Sep 22, 2017 at 2:31 PM, Joe Ortonwrote: > On Fri, Sep 22, 2017 at 11:58:53AM -, yla...@apache.org wrote: >> --- httpd/httpd/trunk/include/ap_mmn.h (original) >> +++ httpd/httpd/trunk/include/ap_mmn.h Fri Sep 22 11:58:53 2017 > ... >> @@ -562,7 +563,7 @@ >> #ifndef MODULE_MAGIC_NUMBER_MAJOR >> #define MODULE_MAGIC_NUMBER_MAJOR 20161018 >> #endif >> -#define MODULE_MAGIC_NUMBER_MINOR 6 /* 0...n */ >> +#define MODULE_MAGIC_NUMBER_MINOR 7 /* 0...n */ > > Since this adds fields to module_struct, not sure how it's anything but > a major bump (not backwards-compatible)? We started to discuss it in the "SSLSrvConfigRec shared" thread, but it's indeed more appriopriate here I think. Usually a new field *at the end*" of structs is a minor bump only... For module_struct, I don't think it's that sensible because modules are usually declared globally, hence existing modules recompiled against latest core will have their 'flags' field initialized to zero. I also took care (hopefully, r1809305) in the core to not try to access the 'flags' field if the module was not compiled against latest core. So it should be safe, unless e.g. one managed to create an array of module_structs... But ISTM that we're allowed to minor bump only here. Regards, Yann.
Re: svn commit: r1809302 - in /httpd/httpd/trunk: include/ap_mmn.h include/http_config.h server/config.c
On Fri, Sep 22, 2017 at 11:58:53AM -, yla...@apache.org wrote: > --- httpd/httpd/trunk/include/ap_mmn.h (original) > +++ httpd/httpd/trunk/include/ap_mmn.h Fri Sep 22 11:58:53 2017 ... > @@ -562,7 +563,7 @@ > #ifndef MODULE_MAGIC_NUMBER_MAJOR > #define MODULE_MAGIC_NUMBER_MAJOR 20161018 > #endif > -#define MODULE_MAGIC_NUMBER_MINOR 6 /* 0...n */ > +#define MODULE_MAGIC_NUMBER_MINOR 7 /* 0...n */ Since this adds fields to module_struct, not sure how it's anything but a major bump (not backwards-compatible)? Regards, Joe
Re: SSLSrvConfigRec shared
On Thu, Sep 21, 2017 at 2:51 PM, Eric Covenerwrote: > On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: >> On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: >>> >>> IIUC it should be safe to extend module_struct with a minor bump to >>> add 'int flags' to the bottom, but when you check the value you'd need >>> to check the MMN first. In the module you'd then just have some flags >>> or'ed together after register_hooks. >> >> Something like the attached patch might do it (untested, no MMN minor bump). >> >>> >>> (hopefully someone will check my work) >> >> Since modules (module_struct) are déclared globally, unspecified >> fields at the end of the struct should be initialized to zero, so it >> should be safe. > > I was thinking about modules compiled against the previous definition > / out of tree. Hmm, I'm not sure my commits address this. The modules would be run by the latest core without being recompiled against it, that's the case?
Re: Time for 2.4.28 ?
STATUS looks clean. Hoping to do a T this afternoon, eastern, unless I hear any objections or concerns re: timing. Cheers!
Re: SSLSrvConfigRec shared
The patches look great! Will test on next occasion! Thanks! :) > Am 22.09.2017 um 14:02 schrieb Yann Ylavic: > > On Thu, Sep 21, 2017 at 2:54 PM, Yann Ylavic wrote: >> On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: >>> On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: > > IIUC it should be safe to extend module_struct with a minor bump to > add 'int flags' to the bottom, but when you check the value you'd need > to check the MMN first. In the module you'd then just have some flags > or'ed together after register_hooks. Something like the attached patch might do it (untested, no MMN minor bump). > > (hopefully someone will check my work) Since modules (module_struct) are déclared globally, unspecified fields at the end of the struct should be initialized to zero, so it should be safe. >>> >>> I was thinking about modules compiled against the previous definition >>> / out of tree. >> >> Right, it's missing some #ifdefs MMN too :) > > r1809302 and r1809303, does it work for you Stefan? > > We have to change the AP_MODULE_HAS_FLAGS macro with the appropriate > MMN if we'd backport...
Re: SSLSrvConfigRec shared
On Thu, Sep 21, 2017 at 2:54 PM, Yann Ylavicwrote: > On Thu, Sep 21, 2017 at 2:51 PM, Eric Covener wrote: >> On Thu, Sep 21, 2017 at 8:44 AM, Yann Ylavic wrote: >>> On Thu, Sep 21, 2017 at 2:11 PM, Eric Covener wrote: IIUC it should be safe to extend module_struct with a minor bump to add 'int flags' to the bottom, but when you check the value you'd need to check the MMN first. In the module you'd then just have some flags or'ed together after register_hooks. >>> >>> Something like the attached patch might do it (untested, no MMN minor bump). >>> (hopefully someone will check my work) >>> >>> Since modules (module_struct) are déclared globally, unspecified >>> fields at the end of the struct should be initialized to zero, so it >>> should be safe. >> >> I was thinking about modules compiled against the previous definition >> / out of tree. > > Right, it's missing some #ifdefs MMN too :) r1809302 and r1809303, does it work for you Stefan? We have to change the AP_MODULE_HAS_FLAGS macro with the appropriate MMN if we'd backport...
Re: mod_authz_core: More control over the authz failed response
On 22 Sep 2017, at 12:12 PM, Yann Ylavicwrote: > I think: > ErrorDocument 403 https://somewhere/ > should work. It does indeed! https://httpd.apache.org/docs/2.4/mod/core.html#errordocument Regards, Graham — smime.p7s Description: S/MIME cryptographic signature
Re: mod_authz_core: More control over the authz failed response
On Fri, Sep 22, 2017 at 12:05 PM, Graham Leggettwrote: > On 22 Sep 2017, at 12:04 PM, Yann Ylavic wrote: > >>> So. I want to be able to send a 302 Temporary Redirect on authz failure, >>> rather than a 403. >> >> Doesn't ErrorDocument work? > > I don’t follow, how would ErrorDocument change the response code from 403 to > 302? I think: ErrorDocument 403 https://somewhere/ should work. > > Regards, > Graham > — >
Re: mod_authz_core: More control over the authz failed response
On 22 Sep 2017, at 12:04 PM, Yann Ylavicwrote: >> So. I want to be able to send a 302 Temporary Redirect on authz failure, >> rather than a 403. > > Doesn't ErrorDocument work? I don’t follow, how would ErrorDocument change the response code from 403 to 302? Regards, Graham — smime.p7s Description: S/MIME cryptographic signature
Re: mod_authz_core: More control over the authz failed response
Hi Graham, On Fri, Sep 22, 2017 at 11:57 AM, Graham Leggettwrote: > > So. I want to be able to send a 302 Temporary Redirect on authz failure, > rather than a 403. Doesn't ErrorDocument work? Regards, Yann.
mod_authz_core: More control over the authz failed response
Hi all, I am currently struggling with Safari’s behaviour where it re-asks for a user certificate if the server accepted optional certificates but returned 403 Forbidden. I want the server to send the end user something sensible to explain what they should do, rather than just have their browser ask for a certificate they don’t have over and over (or they do have but they aren’t authorized). So. I want to be able to send a 302 Temporary Redirect on authz failure, rather than a 403. Looking at mod_authz_core, we have the option to change a 401 response to a 403 response using AuthzSendForbiddenOnFailure, but I’d like more than that. I’m imagining a AuthzForbiddenResponse directive, which would override default behaviour as follows: AuthzForbiddenResponse 401 AuthzForbiddenResponse unauthorized AuthzForbiddenResponse 403 AuthzForbiddenResponse forbidden AuthzForbiddenResponse 302 [url-expression] AuthzForbiddenResponse redirect [url-expression] Does this sound sensible? Regards, Graham — smime.p7s Description: S/MIME cryptographic signature